You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@crail.apache.org by at...@apache.org on 2018/02/21 14:32:15 UTC
[2/2] incubator-crail-website git commit: Publishing from
6097c98ba5c163a131f6bd834e11b76e99ab7418
Publishing from 6097c98ba5c163a131f6bd834e11b76e99ab7418
Project: http://git-wip-us.apache.org/repos/asf/incubator-crail-website/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-crail-website/commit/77142dc2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-crail-website/tree/77142dc2
Diff: http://git-wip-us.apache.org/repos/asf/incubator-crail-website/diff/77142dc2
Branch: refs/heads/asf-site
Commit: 77142dc2f7427ef75403c34fae5866224a77b5c2
Parents: 060c22e
Author: Animesh Trivedi <at...@zurich.ibm.com>
Authored: Wed Feb 21 15:31:54 2018 +0100
Committer: Animesh Trivedi <at...@zurich.ibm.com>
Committed: Wed Feb 21 15:31:54 2018 +0100
----------------------------------------------------------------------
content/404.html | 2 +-
content/Gemfile.lock | 256 ++++--
content/blog/2017/01/sorting.html | 47 +-
content/blog/2017/08/crail-memory.html | 58 +-
content/blog/2017/08/crail-nvme-fabrics-v1.html | 63 +-
content/blog/2017/11/crail-metadata.html | 62 +-
content/blog/2017/11/rdmashuffle.html | 12 +-
content/blog/index.html | 6 +-
content/blog/page2/index.html | 6 +-
content/blog/page3/index.html | 4 +-
content/blog/page4/index.html | 4 +-
content/community/index.html | 6 +-
content/documentation/index.html | 152 ++--
content/feed.xml | 823 +++++++++++++++++++
content/index.html | 2 +-
content/news/index.html | 4 +-
content/overview/index.html | 55 +-
17 files changed, 1230 insertions(+), 332 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/404.html
----------------------------------------------------------------------
diff --git a/content/404.html b/content/404.html
index cff4636..2bfbc7f 100644
--- a/content/404.html
+++ b/content/404.html
@@ -71,7 +71,7 @@
<p>Please consider trying to find what you need from <a href="http://crail.incubator.apache.org/">the home page</a>.</p>
-<p>If you think something might be broken, please notify the developers at the <a href="mailto:dev@crail.incubator.apache.org">dev mailing list</a>. </p>
+<p>If you think something might be broken, please notify the developers at the <a href="mailto:dev@crail.incubator.apache.org">dev mailing list</a>.</p>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/Gemfile.lock
----------------------------------------------------------------------
diff --git a/content/Gemfile.lock b/content/Gemfile.lock
index 52305d8..ad43d03 100644
--- a/content/Gemfile.lock
+++ b/content/Gemfile.lock
@@ -1,128 +1,219 @@
GEM
remote: https://rubygems.org/
specs:
- RedCloth (4.2.9)
- activesupport (5.1.4)
- concurrent-ruby (~> 1.0, >= 1.0.2)
+ activesupport (4.2.9)
i18n (~> 0.7)
minitest (~> 5.1)
+ thread_safe (~> 0.3, >= 0.3.4)
tzinfo (~> 1.1)
- addressable (2.4.0)
- blankslate (2.1.2.4)
- classifier-reborn (2.2.0)
- fast-stemmer (~> 1.0)
+ addressable (2.5.2)
+ public_suffix (>= 2.0.2, < 4.0)
coffee-script (2.4.1)
coffee-script-source
execjs
- coffee-script-source (1.12.2)
- colorator (0.1)
+ coffee-script-source (1.11.1)
+ colorator (1.1.0)
+ commonmarker (0.17.7.1)
+ ruby-enum (~> 0.5)
concurrent-ruby (1.0.5)
ethon (0.11.0)
ffi (>= 1.3.0)
execjs (2.7.0)
faraday (0.14.0)
multipart-post (>= 1.2, < 3)
- fast-stemmer (1.0.2)
- ffi (1.9.21)
- gemoji (2.1.0)
- github-pages (39)
- RedCloth (= 4.2.9)
- github-pages-health-check (~> 0.2)
- jekyll (= 2.4.0)
- jekyll-coffeescript (= 1.0.1)
- jekyll-feed (= 0.3.1)
- jekyll-mentions (= 0.2.1)
- jekyll-redirect-from (= 0.8.0)
- jekyll-sass-converter (= 1.3.0)
- jekyll-sitemap (= 0.8.1)
- jemoji (= 0.5.0)
- kramdown (= 1.5.0)
- liquid (= 2.6.2)
- maruku (= 0.7.0)
+ ffi (1.9.18)
+ forwardable-extended (2.6.0)
+ gemoji (3.0.0)
+ github-pages (172)
+ activesupport (= 4.2.9)
+ github-pages-health-check (= 1.3.5)
+ jekyll (= 3.6.2)
+ jekyll-avatar (= 0.5.0)
+ jekyll-coffeescript (= 1.0.2)
+ jekyll-commonmark-ghpages (= 0.1.3)
+ jekyll-default-layout (= 0.1.4)
+ jekyll-feed (= 0.9.2)
+ jekyll-gist (= 1.4.1)
+ jekyll-github-metadata (= 2.9.3)
+ jekyll-mentions (= 1.2.0)
+ jekyll-optional-front-matter (= 0.3.0)
+ jekyll-paginate (= 1.1.0)
+ jekyll-readme-index (= 0.2.0)
+ jekyll-redirect-from (= 0.12.1)
+ jekyll-relative-links (= 0.5.2)
+ jekyll-remote-theme (= 0.2.3)
+ jekyll-sass-converter (= 1.5.0)
+ jekyll-seo-tag (= 2.3.0)
+ jekyll-sitemap (= 1.1.1)
+ jekyll-swiss (= 0.4.0)
+ jekyll-theme-architect (= 0.1.0)
+ jekyll-theme-cayman (= 0.1.0)
+ jekyll-theme-dinky (= 0.1.0)
+ jekyll-theme-hacker (= 0.1.0)
+ jekyll-theme-leap-day (= 0.1.0)
+ jekyll-theme-merlot (= 0.1.0)
+ jekyll-theme-midnight (= 0.1.0)
+ jekyll-theme-minimal (= 0.1.0)
+ jekyll-theme-modernist (= 0.1.0)
+ jekyll-theme-primer (= 0.5.2)
+ jekyll-theme-slate (= 0.1.0)
+ jekyll-theme-tactile (= 0.1.0)
+ jekyll-theme-time-machine (= 0.1.0)
+ jekyll-titles-from-headings (= 0.5.0)
+ jemoji (= 0.8.1)
+ kramdown (= 1.14.0)
+ liquid (= 4.0.0)
+ listen (= 3.0.6)
mercenary (~> 0.3)
- pygments.rb (= 0.6.3)
- rdiscount (= 2.1.7)
- redcarpet (= 3.3.2)
+ minima (= 2.1.1)
+ rouge (= 2.2.1)
terminal-table (~> 1.4)
- github-pages-health-check (0.6.1)
+ github-pages-health-check (1.3.5)
addressable (~> 2.3)
net-dns (~> 0.8)
- public_suffix (~> 1.4)
+ octokit (~> 4.0)
+ public_suffix (~> 2.0)
typhoeus (~> 0.7)
- html-pipeline (1.9.0)
+ html-pipeline (2.7.1)
activesupport (>= 2)
- nokogiri (~> 1.4)
- i18n (0.9.4)
+ nokogiri (>= 1.4)
+ i18n (0.9.1)
concurrent-ruby (~> 1.0)
- jekyll (2.4.0)
- classifier-reborn (~> 2.0)
- colorator (~> 0.1)
- jekyll-coffeescript (~> 1.0)
- jekyll-gist (~> 1.0)
- jekyll-paginate (~> 1.0)
+ jekyll (3.6.2)
+ addressable (~> 2.4)
+ colorator (~> 1.0)
jekyll-sass-converter (~> 1.0)
jekyll-watch (~> 1.1)
- kramdown (~> 1.3)
- liquid (~> 2.6.1)
+ kramdown (~> 1.14)
+ liquid (~> 4.0)
mercenary (~> 0.3.3)
- pygments.rb (~> 0.6.0)
- redcarpet (~> 3.1)
+ pathutil (~> 0.9)
+ rouge (>= 1.7, < 3)
safe_yaml (~> 1.0)
- toml (~> 0.1.0)
- jekyll-coffeescript (1.0.1)
+ jekyll-avatar (0.5.0)
+ jekyll (~> 3.0)
+ jekyll-coffeescript (1.0.2)
coffee-script (~> 2.2)
- jekyll-feed (0.3.1)
- jekyll-gist (1.5.0)
+ coffee-script-source (~> 1.11.1)
+ jekyll-commonmark (1.1.0)
+ commonmarker (~> 0.14)
+ jekyll (>= 3.0, < 4.0)
+ jekyll-commonmark-ghpages (0.1.3)
+ commonmarker (~> 0.17.6)
+ jekyll-commonmark (~> 1)
+ rouge (~> 2)
+ jekyll-default-layout (0.1.4)
+ jekyll (~> 3.0)
+ jekyll-feed (0.9.2)
+ jekyll (~> 3.3)
+ jekyll-gist (1.4.1)
octokit (~> 4.2)
- jekyll-mentions (0.2.1)
- html-pipeline (~> 1.9.0)
- jekyll (~> 2.0)
+ jekyll-github-metadata (2.9.3)
+ jekyll (~> 3.1)
+ octokit (~> 4.0, != 4.4.0)
+ jekyll-mentions (1.2.0)
+ activesupport (~> 4.0)
+ html-pipeline (~> 2.3)
+ jekyll (~> 3.0)
jekyll-oembed (0.0.1)
jekyll
ruby-oembed (= 0.8.8)
+ jekyll-optional-front-matter (0.3.0)
+ jekyll (~> 3.0)
jekyll-paginate (1.1.0)
- jekyll-redirect-from (0.8.0)
- jekyll (>= 2.0)
- jekyll-sass-converter (1.3.0)
- sass (~> 3.2)
- jekyll-sitemap (0.8.1)
+ jekyll-readme-index (0.2.0)
+ jekyll (~> 3.0)
+ jekyll-redirect-from (0.12.1)
+ jekyll (~> 3.3)
+ jekyll-relative-links (0.5.2)
+ jekyll (~> 3.3)
+ jekyll-remote-theme (0.2.3)
+ jekyll (~> 3.5)
+ rubyzip (>= 1.2.1, < 3.0)
+ typhoeus (>= 0.7, < 2.0)
+ jekyll-sass-converter (1.5.0)
+ sass (~> 3.4)
+ jekyll-seo-tag (2.3.0)
+ jekyll (~> 3.3)
+ jekyll-sitemap (1.1.1)
+ jekyll (~> 3.3)
+ jekyll-swiss (0.4.0)
+ jekyll-theme-architect (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-cayman (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-dinky (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-hacker (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-leap-day (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-merlot (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-midnight (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-minimal (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-modernist (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-primer (0.5.2)
+ jekyll (~> 3.5)
+ jekyll-github-metadata (~> 2.9)
+ jekyll-seo-tag (~> 2.2)
+ jekyll-theme-slate (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-tactile (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-theme-time-machine (0.1.0)
+ jekyll (~> 3.5)
+ jekyll-seo-tag (~> 2.0)
+ jekyll-titles-from-headings (0.5.0)
+ jekyll (~> 3.3)
jekyll-watch (1.5.1)
listen (~> 3.0)
- jemoji (0.5.0)
- gemoji (~> 2.0)
- html-pipeline (~> 1.9)
- jekyll (>= 2.0)
- kramdown (1.5.0)
- liquid (2.6.2)
- listen (3.1.5)
- rb-fsevent (~> 0.9, >= 0.9.4)
- rb-inotify (~> 0.9, >= 0.9.7)
- ruby_dep (~> 1.2)
- maruku (0.7.0)
+ jemoji (0.8.1)
+ activesupport (~> 4.0, >= 4.2.9)
+ gemoji (~> 3.0)
+ html-pipeline (~> 2.2)
+ jekyll (>= 3.0)
+ kramdown (1.14.0)
+ liquid (4.0.0)
+ listen (3.0.6)
+ rb-fsevent (>= 0.9.3)
+ rb-inotify (>= 0.9.7)
mercenary (0.3.6)
mini_portile2 (2.3.0)
- minitest (5.11.3)
+ minima (2.1.1)
+ jekyll (~> 3.3)
+ minitest (5.11.1)
multipart-post (2.0.0)
net-dns (0.8.0)
- nokogiri (1.8.2)
+ nokogiri (1.8.1)
mini_portile2 (~> 2.3.0)
octokit (4.8.0)
sawyer (~> 0.8.0, >= 0.5.3)
- parslet (1.5.0)
- blankslate (~> 2.0)
- posix-spawn (0.3.13)
- public_suffix (1.5.3)
- pygments.rb (0.6.3)
- posix-spawn (~> 0.3.6)
- yajl-ruby (~> 1.2.0)
+ pathutil (0.16.1)
+ forwardable-extended (~> 2.6)
+ public_suffix (2.0.5)
rb-fsevent (0.10.2)
rb-inotify (0.9.10)
ffi (>= 0.5.0, < 2)
- rdiscount (2.1.7)
- redcarpet (3.3.2)
- rouge (3.1.1)
+ rouge (2.2.1)
+ ruby-enum (0.7.1)
+ i18n
ruby-oembed (0.8.8)
- ruby_dep (1.5.0)
+ rubyzip (1.2.1)
safe_yaml (1.0.4)
sass (3.5.5)
sass-listen (~> 4.0.0)
@@ -135,14 +226,11 @@ GEM
terminal-table (1.8.0)
unicode-display_width (~> 1.1, >= 1.1.1)
thread_safe (0.3.6)
- toml (0.1.2)
- parslet (~> 1.5.0)
typhoeus (0.8.0)
ethon (>= 0.8.0)
- tzinfo (1.2.5)
+ tzinfo (1.2.4)
thread_safe (~> 0.1)
unicode-display_width (1.3.0)
- yajl-ruby (1.2.3)
PLATFORMS
ruby
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/blog/2017/01/sorting.html
----------------------------------------------------------------------
diff --git a/content/blog/2017/01/sorting.html b/content/blog/2017/01/sorting.html
index e48bb5a..6b955ac 100644
--- a/content/blog/2017/01/sorting.html
+++ b/content/blog/2017/01/sorting.html
@@ -182,9 +182,9 @@ A Spark sorting job consists of two phases. The first phase is a mapping or cla
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/terasort_pipeline.png" width="490" /></div>
-<br /><br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/terasort_pipeline.png" width="490" /></div>
+<p><br /><br /></p>
<h3 id="using-vanilla-spark">Using Vanilla Spark</h3>
@@ -194,9 +194,9 @@ The first question we are interested in is to what extent such a sorting benchma
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/net_vanilla.svg" /></div>
-<br /><br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/net_vanilla.svg" /></div>
+<p><br /><br /></p>
<div style="text-align: justify">
<p>
@@ -204,9 +204,9 @@ The poor network usage matches with the general observation we made in our previ
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/cpu_network.svg" /></div>
-<br /><br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/cpu_network.svg" /></div>
+<p><br /><br /></p>
<div style="text-align: justify">
<p>
@@ -222,9 +222,9 @@ An overview of the Crail shuffler is provided in the <a href="http://crail.incub
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/shuffle_rdma.png" width="470" /></div>
-<br /><br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/shuffle_rdma.png" width="470" /></div>
+<p><br /><br /></p>
<div style="text-align: justify">
<p>
@@ -247,17 +247,17 @@ The figure below shows the overall performance of Spark/Crail vs Spark/Vanilla o
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/performance_overall.png" width="470" /></div>
-<br /><br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/performance_overall.png" width="470" /></div>
+<p><br /><br /></p>
<p>One key question of interest is about the network usage of the Crail shuffler during the sorting benchmark. In the figure below, we show the data rate at which the different reduce tasks fetch data from the network. Each point in the figure corresponds to one reduce task. In our configuration, we run 3 Spark executors per node and 5 Spark cores per executor. Thus, 1920 reduce tasks are running concurrently (out of 6400 reduce tasks in total) generating a cluster-wide all-to-all traffic of about 70Gbit/s per node during that phase.</p>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/multiread.svg" /></div>
-<br /><br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/sort/multiread.svg" /></div>
+<p><br /><br /></p>
-<p>In this blog post, we have shown that Crail successfully manages to translate the raw network performance into actual workload level gains. The exercise with TeraSort as an application validates the design decisions we made in Crail. Stay tuned for more results with different workloads and hardware configurations. </p>
+<p>In this blog post, we have shown that Crail successfully manages to translate the raw network performance into actual workload level gains. The exercise with TeraSort as an application validates the design decisions we made in Crail. Stay tuned for more results with different workloads and hardware configurations.</p>
<h3 id="how-to-run-sorting-with-sparkcrail">How to run Sorting with Spark/Crail</h3>
@@ -266,19 +266,18 @@ The figure below shows the overall performance of Spark/Crail vs Spark/Vanilla o
<ul>
<li>Build and deploy Crail using the instructions at <a href="http://crail.incubator.apache.org/documentation#crail">documentation</a></li>
<li>Enable the Crail shuffler for Spark by building Spark-IO using the instructions at <a href="http://crail.incubator.apache.org/documentation#spark">documentation</a></li>
- <li>Configure the DRAM storage tier of Crail so that all the shuffle data fits into the DRAM tier. </li>
+ <li>Configure the DRAM storage tier of Crail so that all the shuffle data fits into the DRAM tier.</li>
<li>Build the sorting benchmark using the instructions on <a href="https://github.com/zrlio/crail-terasort">GitHub</a></li>
<li>Make sure you have the custom serializer and sorter specified in spark-defaults.conf</li>
- <li>Run Hadoop TeraGen to produce a valid data set. We used standard HDFS for both input and output data. </li>
+ <li>Run Hadoop TeraGen to produce a valid data set. We used standard HDFS for both input and output data.</li>
<li>Run the Crail-TeraSort on your Spark cluster. The command line we have used on the 128 node cluster is the following:</li>
</ul>
-<p><code>
-./bin/spark-submit -v --num-executors 384 --executor-cores 5 --executor-memory 64G
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/spark-submit -v --num-executors 384 --executor-cores 5 --executor-memory 64G
--driver-memory 64G --master yarn
--class com.ibm.crail.terasort.TeraSort path/to/crail-terasort-2.0.jar
-i /terasort-input-1280g -o /terasort-output-1280g
-</code></p>
+</code></pre></div></div>
<p>Have questions or comments? Feel free to discuss at the dev mailing list at <a href="mailto:dev@crail.incubator.apache.org">dev@crail.incubator.apache.org</a></p>
<hr />
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/blog/2017/08/crail-memory.html
----------------------------------------------------------------------
diff --git a/content/blog/2017/08/crail-memory.html b/content/blog/2017/08/crail-memory.html
index 69e2dd2..0f50c2f 100644
--- a/content/blog/2017/08/crail-memory.html
+++ b/content/blog/2017/08/crail-memory.html
@@ -92,7 +92,7 @@ I recently read the <a href="https://www.usenix.org/conference/atc17/technical-s
</li>
<li>OpenPower Node configuration
<ul>
- <li>CPU: 2x OpenPOWER Power8 10-core @2.9Ghz </li>
+ <li>CPU: 2x OpenPOWER Power8 10-core @2.9Ghz</li>
<li>DRAM: 512GB DDR4</li>
<li>Network: 1x100Gbit/s Ethernet Mellanox ConnectX-4 EN (Ethernet/RoCE)
<ul>
@@ -119,8 +119,7 @@ I recently read the <a href="https://www.usenix.org/conference/atc17/technical-s
Data operations in Crail -- such as the reading or writing of files -- are internally composed of metadata operations and actual data transfers. Let's look at a simple Crail application that opens a file and reads the file sequentially:
</p>
</div>
-<p><code>
-CrailConfiguration conf = new CrailConfiguration();
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailConfiguration conf = new CrailConfiguration();
CrailFS fs = CrailFS.newInstance(conf);
CrailFile file = fs.lookup(filename).get().asFile();
CrailInputStream stream = file.getDirectInputStream();
@@ -131,23 +130,23 @@ while(stream.available() > 0){
//Await completion of operation
future.get();
}
-</code> <br />
-<div style="text-align: justify"> </p>
+</code></pre></div></div>
+<div style="text-align: justify">
<p>
One challenge with file read/write operations is to avoid blocking in case block metadata information is missing. Crail caches block metadata at the client, but caching is ineffective for both random reads and write-once read-once data. To avoid blocking for sequential read/write operations, Crail interleaves metadata operations and actual data transfers. Each read operation always triggers the lookup of block metadata for the next block immediately after issuing the RDMA read operation for the current block. The asynchronous and non-blocking nature of RDMA allows both operations to be executed in the process context of the application, without context switching or any additional background threads. The figure illustrates the case of one outstanding operation a time. The asynchronous Crail storage API, however, permits any number of outstanding operations.
</p>
-<p></div>
-<br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/anatomy.png" width="420" /></div>
-<br />
-<div style="text-align: justify"> </p>
+</div>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/anatomy.png" width="420" /></div>
+<p><br /></p>
+<div style="text-align: justify">
<p>
As a side note, it's also worth mentioning that Crail does not actually use RPCs for the data transfers but uses RDMA one-sided read/write operations instead. Moreover, Crail is designed from ground up for byte-addressable storage and memory. For instance, files in Crail are essentially a sequence of virtual memory windows on different hosts which allows for a very effective handling of small data operations. As shown in the figure, during the last operation, with only a few bytes left to be read, the byte-granular nature of Crail's block access protocol makes sure that only the relevant bytes are transmitted over the network, as opposed to transmitting the entire block.
</p>
<p>
The basic read/write logic shown in the figure above is common to all storage tiers in Crail, including the NVMe flash tier. In the remainder of this post, we specificially look at the performance of Crail's DRAM storage tier though.
</p>
-<p></div></p>
+</div>
<h3 id="sequential-readwrite-throughput">Sequential Read/Write Throughput</h3>
@@ -156,37 +155,35 @@ The basic read/write logic shown in the figure above is common to all storage ti
Let's start by looking at sequential read/write performance. These benchmarks can be run easily from the command line. Below is an example for a sequential write experiment issuing 100M write operations of size 1K to produce a file of roughly 100GB size. The -w switch indicates that we are using 32 warmup operations.
</p>
</div>
-<p><code>
-./bin/crail iobench -t write -s 1024 -k 100000000 -w 32 -f /tmp.dat
-</code> <br />
-<div style="text-align: justify"> </p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t write -s 1024 -k 100000000 -w 32 -f /tmp.dat
+</code></pre></div></div>
+<div style="text-align: justify">
<p>
Crail offers direct I/O streams as well as buffered streams. For sequential operations it is important to use the buffered streams. Even though the buffered streams impose one extra copy (from the Crail stream to the application buffer) they are typically more effective for sequential access as they make sure that at least one network operation is in-flight at any time. The buffer size in a Crail buffered stream and the number of oustanding operations can be controlled by setting the buffersize and the slicesize properties in crail-site.conf. For our experiments we used a 1MB buffer per stream sliced up into two slices of 512K each which eventually leads to two operations in flight.
</p>
-<p></div></p>
+</div>
-<p><code>
-crail.buffersize 1048576
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.buffersize 1048576
crail.slicesize 524288
-</code> </p>
+</code></pre></div></div>
<div style="text-align: justify">
<p>
The figure below illustrates the sequential write (top) and read (bottom) performance of Crail (DRAM tier) for different application buffer sizes (not to be mixed up with crail.buffersize used within streams) and shows a comparison to other systems. As of now, we only show a comparison with Alluxio, an in-memory file system for caching data in Spark or Hadoop applications. We are, however, working on including results for other storage systems such as Apache Ignite and GlusterFS and we plan to update the blog post accordingly soon. If there is a particular storage system that is not included but you would like to see included as a comparison, please write us. And <b>important</b>: if you find that the results we show for a particular storage system do not match your experience, please write to us too, we are happy to revisit the configuration.
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/write.svg" width="550" /></div>
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/read.svg" width="550" /></div>
-<br /><br />
-<div style="text-align: justify"> </p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/write.svg" width="550" /></div>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/read.svg" width="550" /></div>
+<p><br /><br /></p>
+<div style="text-align: justify">
<p>
One first observation from the figure is that there is almost no difference in performance for write and read operations. Second, at a buffer size of around 1K Crail reaches a bandwidth close to 95Gbit/s (for read), which is approaching the network hardware limit of 100Gbps. And third, Crail performs significantly faster than other in-memory storage systems, in this case Alluxio. This because Crail is built on of user-level networking and thereby avoids the overheads of both the Linux network stack (memory copies, context switches, etc.) and the Java runtime.
</p>
<p>
Note that both figures show single-client performance numbers. With Crail being a user-level storage system executing I/O operations directly within the application context this means the entire benchmark is truly runninig on one single core. Often, systems that perform poorly in single-client experiments are being defended saying that nobody cares about the single-client performance. Especially throughput problems can easily be fixed by adding more cores. This is, however, not at all cloudy to say the least. At the level hardware is multiplexed and priced in today's cloud computing data centers every core counts. The figure below shows a simple Spark group-by experiment on the same 8-node cluster. As can be seen, with Crail the benchmark executes faster using a single core per machine than with default Spark using 8 cores per machine, which is a direct consequence from Crail's superb single-core I/O performance.
</p>
-<p></div></p>
+</div>
<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/crail-groupby.svg" width="550" /></div>
<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/spark-groupby.svg" width="550" /></div>
@@ -198,15 +195,14 @@ Note that both figures show single-client performance numbers. With Crail being
Typically, distributed storage systems are either built for sequential access to large data sets (e.g., HDFS) or they are optimized for random access to small data sets (e.g., key/value stores). We have already shown that Crail performs well for large sequentially accessed data sets, let's now look at the latencies of small random read operations. For this, we mimic the behavior of a key/value store by storing key/value pairs in Crail files with the key being the filename. We then measure the time it takes to open the file and read its content. Again, the benchmark can easily be executed from the command line. The following example issues 1M get() operations on a small file filled with a 4 byte value.
</p>
</div>
-<p><code>
-./bin/crail iobench -t getkey -s 4 -k 1000000 -f /tmp.dat -w 32
-</code> <br />
-<div style="text-align: justify"> </p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t getkey -s 4 -k 1000000 -f /tmp.dat -w 32
+</code></pre></div></div>
+<div style="text-align: justify">
<p>
The figure below illustrates the latencies of get() operations for different key/value sizes and compares them to the latencies we obtained with RAMCloud for the same type of operations (measured using RAMClouds C and Java APIs). RAMCloud is a low-latency key/value store implemented using RDMA. RAMCloud actually provides durable storage by asynchronously replicating data onto backup devices. However, at any point in time all the data is held in DRAM and read requests will be served from DRAM directly. Up to our knowledge, RAMCloud is the fastest key/value store that is (a) available open source and (b) can be deployed in practice as a storage platform for applications. Other similar RDMA-based storage systems we looked at, like FaRM or HERD, are either not open source or they do not provide a clean separation between storage system, API and clients.
</p>
-<p></div>
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/latency.svg" width="550" /></div></p>
+</div>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/latency.svg" width="550" /></div>
<div style="text-align: justify">
<p>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/blog/2017/08/crail-nvme-fabrics-v1.html
----------------------------------------------------------------------
diff --git a/content/blog/2017/08/crail-nvme-fabrics-v1.html b/content/blog/2017/08/crail-nvme-fabrics-v1.html
index 3f7ae13..b29c6a2 100644
--- a/content/blog/2017/08/crail-nvme-fabrics-v1.html
+++ b/content/blog/2017/08/crail-nvme-fabrics-v1.html
@@ -111,14 +111,13 @@ This is part II of our series of posts discussing Crail's raw storage performanc
Crail is a framework that allows arbitrary storage backends to be added by implementing the Crail storage interface. A storage backend manages the point-to-point data transfers on a per block granularity between a Crail client and a set of storage servers. The Crail storage interface essentially consists of three virtual functions, which simplified look like this:
</p>
</div>
-<p><code>
-//Server-side interface: donate storage resources to Crail
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>//Server-side interface: donate storage resources to Crail
StorageResource allocateResource();
//Client-side interface: read/write remote/local storage resources
writeBlock(BlockInfo, ByteBuffer);
readBlock(BlockInfo, ByteBuffer);
-</code>
-<div style="text-align: justify"> </p>
+</code></pre></div></div>
+<div style="text-align: justify">
<p>
A specific implementation of this interface provides an efficient mapping of Crail storage operations to the actual storage and network hardware the backend is exporting. Crail comes with two native storage backends, an RDMA-based DRAM backend and an RDMA-based NVMe backend, but other storage backends are available as well (e.g., Netty) and we plan to provide more custom backends in the future as new storage and network technologies are emerging.
</p>
@@ -134,7 +133,7 @@ The Crail client runtime invokes the NVMf client interface during file read/writ
<p>
One downside of the NVMe interface is that byte level access is prohibited. Instead data operations have to be issued for entire drive sectors which are typically 512Byte or 4KB large (we used 512Byte sector size in all the experiments shown in this blog). As we wanted to use the standard NVMf protocol (and Crail has a client driven philosophy) we needed to implement byte level access at the client side. For reads this can be achieved in a straight forward way by reading the whole sector and copying out the requested part. For writes that modify a certain subrange of a sector that has already been written before we need to do a read modify write operation.
</p>
-<p></div></p>
+</div>
<h3 id="performance-comparison-to-native-spdk-nvmf">Performance comparison to native SPDK NVMf</h3>
@@ -143,18 +142,16 @@ One downside of the NVMe interface is that byte level access is prohibited. Inst
We perform latency and throughput measurement of our Crail NVMf storage tier against a native SPDK NVMf benchmark to determine how much overhead our implementation adds. The first plot shows random read latency on a single 512GB Samsung 960Pro accessed remotely through SPDK. For Crail we also show the time it takes to perform a metadata operations. You can run the Crail benchmark from the command line like this:
</p>
</div>
-<p><code>
-./bin/crail iobench -t readRandom -b false -s <size> -k <iterations> -w 32 -f /tmp.dat
-</code>
-and SPDK:
-<code>
-./perf -q 1 -s <size> -w randread -r 'trtype:RDMA adrfam:IPv4 traddr:<ip> trsvcid:<port>' -t <time in seconds>
-</code>
-<div style="text-align: justify"> </p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t readRandom -b false -s <size> -k <iterations> -w 32 -f /tmp.dat
+</code></pre></div></div>
+<p>and SPDK:</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./perf -q 1 -s <size> -w randread -r 'trtype:RDMA adrfam:IPv4 traddr:<ip> trsvcid:<port>' -t <time in seconds>
+</code></pre></div></div>
+<div style="text-align: justify">
<p>
The main take away from this plot is that the time it takes to perform a random read operation on a NVMe-backed file in Crail takes only about 7 microseconds more time than fetching the same amount of data over a point-to-point SPDK connection. This is impressive because it shows that using Crail a bunch of NVMe drives can be turned into a fully distributed storage space at almost no extra cost. The 7 microseconds are due to Crail having to look up the specific NVMe storage node that holdes the data -- an operation which requires one extra network roundtrip (client to namenode). The experiment represents an extreme case where no metadata is cached at the client. In practice, file blocks are often accessed multiple times in which case the read latency is further reduced. Also note that unlike SPDK which is a native library, Crail delivers data directly into Java off-heap memory.
</p>
-<p></div></p>
+</div>
<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-nvmf/latency.svg" width="550" /></div>
<p><br /></p>
@@ -164,19 +161,17 @@ The main take away from this plot is that the time it takes to perform a random
The second plot shows sequential read and write throughput with a transfer size of 64KB and 128 outstanding operations. The Crail throughput benchmark can be run like this:
</p>
</div>
-<p><code>
-./bin/crail iobench -t readAsync -s 65536 -k <iterations> -b 128 -w 32 -f /tmp.dat
-</code>
-and SPDK:
-<code>
-./perf -q 128 -s 65536 -w read -r 'trtype:RDMA adrfam:IPv4 traddr:<ip> trsvcid:<port>' -t <time in seconds>
-</code>
-<div style="text-align: justify"> </p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t readAsync -s 65536 -k <iterations> -b 128 -w 32 -f /tmp.dat
+</code></pre></div></div>
+<p>and SPDK:</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./perf -q 128 -s 65536 -w read -r 'trtype:RDMA adrfam:IPv4 traddr:<ip> trsvcid:<port>' -t <time in seconds>
+</code></pre></div></div>
+<div style="text-align: justify">
<p>
For sequential operations in Crail, metadata fetching is inlined with data operations as described in the <a href="http://crail.incubator.apache.org/blog/2017/08/crail-memory.html">DRAM</a> blog. This is possible as long as the data transfer has a lower latency than the metadata RPC, which is typically the case. As a consequence, our NVMf storage tier reaches the same throughput as the native SPDK benchmark (device limit).
</p>
-<p></div>
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-nvmf/throughput.svg" width="550" /></div></p>
+</div>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-nvmf/throughput.svg" width="550" /></div>
<h3 id="sequential-throughput">Sequential Throughput</h3>
@@ -185,13 +180,11 @@ For sequential operations in Crail, metadata fetching is inlined with data opera
Let us look at the sequential read and write throughput for buffered and direct streams and compare them to a buffered Crail stream on DRAM. All benchmarks are single thread/client performed against 8 storage nodes with 4 drives each, cf. configuration above. In this benchmark we use 32 outstanding operations for the NVMf storage tier buffered stream experiments by using a buffer size of 16MB and a slice size of 512KB, cf. <a href="http://crail.incubator.apache.org/blog/2017/07/crail-memory.html">part I</a>. The buffered stream reaches line speed at a transfer size of around 1KB and shows only slightly slower performance when compared to the DRAM tier buffered stream. However we are only using 2 outstanding operations with the DRAM tier to achieve these results. Basically for sizes smaller than 1KB the buffered stream is limited by the copy speed to fill the application buffer. The direct stream reaches line speed at around 128KB with 128 outstanding operations. Here no copy operati
on is performed for transfer size greater than 512Byte (sector size). The command to run the Crail buffered stream benchmark:
</p>
</div>
-<p><code>
-./bin/crail iobench -t read -s <size> -k <iterations> -w 32 -f /tmp.dat
-</code>
-The direct stream benchmark:
-<code>
-./bin/crail iobench -t readAsync -s <size> -k <iterations> -b 128 -w 32 -f /tmp.dat
-</code></p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t read -s <size> -k <iterations> -w 32 -f /tmp.dat
+</code></pre></div></div>
+<p>The direct stream benchmark:</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t readAsync -s <size> -k <iterations> -b 128 -w 32 -f /tmp.dat
+</code></pre></div></div>
<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-nvmf/throughput2.svg" width="550" /></div>
@@ -213,16 +206,16 @@ In this paragraph we show how Crail can leverage flash memory when there is not
</p>
</div>
<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-nvmf/crail_tiering.png" width="500" vspace="10" /></div>
-<p><br />
-<div style="text-align: justify"> </p>
+<p><br /></p>
+<div style="text-align: justify">
<p>
In the following 200G Terasort experiment we gradually limit the DRAM resources in Crail while adding more flash to the Crail NVMf storage tier. Note that here Crail is used for both input/output as well as shuffle data. The figure shows that by putting all the data in flash we only increase the sorting time by around 48% compared to the configuration where all the data resides in DRAM. Considering the cost of DRAM and the advances in technology described above we believe cheaper NVM storage can replace DRAM for most of the applications with only a minor performance decrease. Also, note that even with 100% of the data in NVMe, Spark/Crail is still faster than vanilla Spark with all the data in memory. The vanilla Spark experiment uses Alluxio for input/output and RamFS for the shuffle data.
</p>
-<p></div></p>
+</div>
<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-nvmf/tiering.svg" width="550" /></div>
-<p>To summarize, in this blog we have shown that the NVMf storage backend for Crail – due to its efficient user-level implementation – offers latencies and throughput very close to the hardware speed. The Crail NVMf storage tier can be used conveniently in combination with the Crail DRAM tier to either save cost or to handle situations where the available DRAM is not sufficient to store the working set of a data processing workload. </p>
+<p>To summarize, in this blog we have shown that the NVMf storage backend for Crail – due to its efficient user-level implementation – offers latencies and throughput very close to the hardware speed. The Crail NVMf storage tier can be used conveniently in combination with the Crail DRAM tier to either save cost or to handle situations where the available DRAM is not sufficient to store the working set of a data processing workload.</p>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/blog/2017/11/crail-metadata.html
----------------------------------------------------------------------
diff --git a/content/blog/2017/11/crail-metadata.html b/content/blog/2017/11/crail-metadata.html
index cc12f0e..beecf61 100644
--- a/content/blog/2017/11/crail-metadata.html
+++ b/content/blog/2017/11/crail-metadata.html
@@ -128,10 +128,9 @@ In two of the previous blogs (<a href="http://crail.incubator.apache.org/blog/20
An important metadata operation is ''getFile()'', which is used by clients to lookup the status of a file (whether the file exists, what size it has, etc.). The ''getFile()'' operation is served by Crail's fast lock-free map and in spirit is very similar to the ''getBlock()'' metadata operation (used by clients to query which storage nodes holds a particular block). In a typical Crail use case, ''getFile()'' and ''getBlock()'' are responsible for the peak metadata load at a namenode. In this experiment, we measure the achievable IOPS on the server side in an artificial configuration with many clients distributed across the cluster issuing ''getFile()'' in a tight loop. Note that the client side RPC interface in Crail is asynchronous, thus, clients can issue multiple metadata operations without blocking while asynchronously waiting for the result. In the experiments below, each client may have a maximum of 128 ''getFile()'' operations outstanding at any point in time. In a practical
scenario, Crail clients may also have multiple metadata operations in flight either because clients are shared by different cores, or because Crail interleaves metadata and data operations (see <a href="http://crail.incubator.apache.org/blog/2017/08/crail-memory.html">DRAM</a>). What makes the benchmark artificial is that clients exclusively focus on generating load for the namenode and thereby are neither performing data operations nor are they doing any compute. The basic command of the benchmark as executed by each of the individual clients is given by the following command:
</p>
</div>
-<p><code>
-./bin/crail iobench -t getMultiFileAsync -f / -k 10000000 -b 128
-</code>
-<div style="text-align: justify"> </p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t getMultiFileAsync -f / -k 10000000 -b 128
+</code></pre></div></div>
+<div style="text-align: justify">
<p>
Where ''-t'' specifies the benchmark to run, ''-f'' specifies the path on the
Crail file system to be used for the benchmark, ''-k'' specifies the number of
@@ -139,7 +138,7 @@ iterations to be performed by the benchmark
(how many times will the benchmark execute ''getFile()'') and
''-b'' specifies the maximum number of requests in flight.
</p>
-<p></div></p>
+</div>
<h3 id="single-namenode-scalability">Single Namenode Scalability</h3>
@@ -151,10 +150,9 @@ In the first experiment, we measure the aggregated number of metadata operations
As comparison, we measure the raw number of IOPS, which can be executed on the RDMA network. We measure the raw number using ib_send_bw. We configured ib_send_bw with the same parameters in terms of RDMA configuration as the namenode. This means, we instructed ib_send_bw not to do CQ moderation, and to use a receive queue and a send queue of length 32, which equals the length of the namenode queues. Note that the default configuration of ib_send_bw uses CQ moderation and does preposting of send operations, which can only be done, if the operation is known in advance. This is not the case in a real system, like crail's namenode. The basic ib_send_bw command is given below:
</p>
</div>
-<p><code>
-ib_send_bw -s 1 -Q 1 -r 32 -t 32 -n 10000000
-</code>
-<div style="text-align: justify"> </p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ib_send_bw -s 1 -Q 1 -r 32 -t 32 -n 10000000
+</code></pre></div></div>
+<div style="text-align: justify">
<p>
Where ''-s 1'' specifies to send packets with a payload of 1 (we don't want to
measure the transmission time of data, just the number of I/O operations),
@@ -163,21 +161,21 @@ queue length to be 32, ''-t 32'' specifies the send queue length to be 32
and ''-n'' specifies the number of
iterations to be performed by ib_send_bw.
</p>
-<p></div>
-<div style="text-align: justify"> </p>
+</div>
+<div style="text-align: justify">
<p>
The line of the raw number of IOPS, labeled ''ib send'' is shown in the same graph. With this measurement we show that Crail's namenode IOPS are similar to the raw ib_send_bw IOPS with the same configuration.
</p>
-<p></div>
-<br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/namenode_ibsend_iops64.svg" width="550" /></div>
-<br />
-<div style="text-align: justify"> </p>
+</div>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/namenode_ibsend_iops64.svg" width="550" /></div>
+<p><br /></p>
+<div style="text-align: justify">
<p>
If one starts ib_send_bw without specifying the queue sizes or whether or not to use CQ moderation, the raw number of IOPS might be higher. This is due to the fact, that the default values of ib_send_bw use a receive queue of 512, a send queue of 128 and CQ moderation of 100, meaning that a new completion is generated only after 100 sends. As comparison, we did this
measurement too and show the result, labeled 'ib_send CQ mod', in the same graph. Fine tuning of receive and send queue sizes, CQ moderation size, postlists etc might lead to a higher number of IOPS.
</p>
-<p></div></p>
+</div>
<h3 id="multiple-namenode-scalability">Multiple Namenode Scalability</h3>
@@ -186,9 +184,9 @@ measurement too and show the result, labeled 'ib_send CQ mod', in the same graph
To increase the number of IOPS the overall system can handle, we allow starting multiple namenode instances. Hot metadata operations, such as ''getFile()'', are distributed over all running instances of the namenode. ''getFile()'' is implemented such that no synchronization among the namenodes is required. As such, we expect good scalability. The graph below compares the overall IOPS of a system with one namenode to a system with two namenodes and four namenodes.
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/namenode_multi64.svg" width="550" /></div>
-<br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/namenode_multi64.svg" width="550" /></div>
+<p><br /></p>
<div style="text-align: justify">
<p>
@@ -248,9 +246,9 @@ namenode over the elapsed runtime of the TeraSort application.
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/terasort_iops.svg" width="550" /></div>
-<br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/terasort_iops.svg" width="550" /></div>
+<p><br /></p>
<div style="text-align: justify">
<p>
@@ -412,9 +410,9 @@ plot shows the number of IOPS relative to the number of clients.
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/namenode_hdfs_iops.svg" width="550" /></div>
-<br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/namenode_hdfs_iops.svg" width="550" /></div>
+<p><br /></p>
<div style="text-align: justify">
<p>
@@ -466,9 +464,9 @@ the blog with the latest numbers as soon as the bug is fixed.
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/ramcloud_iops.svg" width="550" /></div>
-<br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/ramcloud_iops.svg" width="550" /></div>
+<p><br /></p>
<div style="text-align: justify">
<p>
@@ -497,9 +495,9 @@ of 30Mio/s with 4 namenodes).
</p>
</div>
-<p><br />
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/max_iops_crail_hdfs_ramcloud.svg" width="550" /></div>
-<br /></p>
+<p><br /></p>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-metadata/max_iops_crail_hdfs_ramcloud.svg" width="550" /></div>
+<p><br /></p>
<div style="text-align: justify">
<p>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/blog/2017/11/rdmashuffle.html
----------------------------------------------------------------------
diff --git a/content/blog/2017/11/rdmashuffle.html b/content/blog/2017/11/rdmashuffle.html
index 4456790..4482592 100644
--- a/content/blog/2017/11/rdmashuffle.html
+++ b/content/blog/2017/11/rdmashuffle.html
@@ -97,7 +97,7 @@ This blog is comparing the shuffle performance of Crail with SparkRDMA, an alter
<ul>
<li>Ubuntu 16.04.3 LTS (Xenial Xerus) with Linux kernel version 4.10.0-33-generic</li>
<li><a href="https://github.com/zrlio/crail">Crail 1.0</a>, commit a45c8382050f471e9342e1c6cf25f9f2001af6b5</li>
- <li><a href="">Crail Shuffle plugin</a>, commit 2273b5dd53405cab3389f5c1fc2ee4cd30f02ae6 </li>
+ <li><a href="">Crail Shuffle plugin</a>, commit 2273b5dd53405cab3389f5c1fc2ee4cd30f02ae6</li>
<li><a href="https://github.com/Mellanox/SparkRDMA">SparkRDMA</a>, commit d95ce3e370a8e3b5146f4e0ab5e67a19c6f405a5 (latest master on 8th of November 2017)</li>
</ul>
</li>
@@ -119,17 +119,17 @@ In contrast, the Crail shuffler plugin takes a more holistic approach and levera
</p>
</div>
<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/rdma-shuffle/terasort.svg" width="550" /></div>
-<p><br />
-<div style="text-align: justify"></p>
+<p><br /></p>
+<div style="text-align: justify">
<p>
First we run <a href="https://github.com/zrlio/crail-spark-terasort">terasort</a> on our 8+1 machine cluster (see above). We sort 200GB, thus, each node gets 25GB of data (equal distribution). We further did a basic search of the parameter space for each of the systems to find the best possible configuration. In all the experiments we use 8 executors with 12 cores each. Note that in a typical Spark run more CPU cores than assigned are engaged because of garbabge collection, etc. In our test runs assigning 12 cores lead to the best performance.
</p>
<p>
The plot above shows runtimes of the various configuration we run with terasort. SparkRDMA with the Wrapper shuffle writer performance slightly better (3-4%) than vanilla Spark whereas the Chunked shuffle writer shows a 30% overhead. On a quick inspection we found that this overhead stems from memory allocation and registration for the shuffle data that is kept in memory between the stages. Compared to vanilla Spark, Crail's shuffle plugin shows performance improvement of around 235%.
</p>
-<p></div>
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/rdma-shuffle/sql.svg" width="550" /></div>
-<br /></p>
+</div>
+<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/rdma-shuffle/sql.svg" width="550" /></div>
+<p><br /></p>
<div style="text-align: justify">
<p>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/blog/index.html
----------------------------------------------------------------------
diff --git a/content/blog/index.html b/content/blog/index.html
index b87edc9..732428f 100644
--- a/content/blog/index.html
+++ b/content/blog/index.html
@@ -9,8 +9,8 @@
<link rel="alternate" type="application/atom+xml" title="Atom"
href="http://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
- <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
+ <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/blog-summary.png" />
+ <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/blog-summary.png" />
</head>
<body>
@@ -34,7 +34,7 @@
</li>
- <li >
+ <li class="active">
<a href="http://crail.incubator.apache.org/blog/">
Blog
</a>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/blog/page2/index.html
----------------------------------------------------------------------
diff --git a/content/blog/page2/index.html b/content/blog/page2/index.html
index 00f7857..9796b04 100644
--- a/content/blog/page2/index.html
+++ b/content/blog/page2/index.html
@@ -9,8 +9,8 @@
<link rel="alternate" type="application/atom+xml" title="Atom"
href="http://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
- <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
+ <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/page2-summary.png" />
+ <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/page2-summary.png" />
</head>
<body>
@@ -129,7 +129,7 @@
<ul class="pager">
<li class="previous">
- <a href="/blog/index.html">← Newer Posts</a>
+ <a href="/blog/">← Newer Posts</a>
</li>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/blog/page3/index.html
----------------------------------------------------------------------
diff --git a/content/blog/page3/index.html b/content/blog/page3/index.html
index 72c7552..dbd2d51 100644
--- a/content/blog/page3/index.html
+++ b/content/blog/page3/index.html
@@ -9,8 +9,8 @@
<link rel="alternate" type="application/atom+xml" title="Atom"
href="http://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
- <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
+ <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/page3-summary.png" />
+ <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/page3-summary.png" />
</head>
<body>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/blog/page4/index.html
----------------------------------------------------------------------
diff --git a/content/blog/page4/index.html b/content/blog/page4/index.html
index 16fd84a..aabfb8b 100644
--- a/content/blog/page4/index.html
+++ b/content/blog/page4/index.html
@@ -9,8 +9,8 @@
<link rel="alternate" type="application/atom+xml" title="Atom"
href="http://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
- <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
+ <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/page4-summary.png" />
+ <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/page4-summary.png" />
</head>
<body>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/community/index.html
----------------------------------------------------------------------
diff --git a/content/community/index.html b/content/community/index.html
index 93b6c6b..3584720 100644
--- a/content/community/index.html
+++ b/content/community/index.html
@@ -9,8 +9,8 @@
<link rel="alternate" type="application/atom+xml" title="Atom"
href="http://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
- <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
+ <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/community-summary.png" />
+ <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/community-summary.png" />
</head>
<body>
@@ -41,7 +41,7 @@
</li>
- <li >
+ <li class="active">
<a href="http://crail.incubator.apache.org/community/">
Community
</a>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/77142dc2/content/documentation/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/index.html b/content/documentation/index.html
index 3ab4ff6..68ee8c8 100644
--- a/content/documentation/index.html
+++ b/content/documentation/index.html
@@ -9,8 +9,8 @@
<link rel="alternate" type="application/atom+xml" title="Atom"
href="http://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
- <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/index-summary.png" />
+ <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/documentation-summary.png" />
+ <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/documentation-summary.png" />
</head>
<body>
@@ -48,7 +48,7 @@
</li>
- <li >
+ <li class="active">
<a href="http://crail.incubator.apache.org/documentation/">
Documentation
</a>
@@ -70,7 +70,7 @@
<p>Apache Crail (Incubating) is a fast multi-tiered distributed storage system designed from ground up for high-performance network and storage hardware. The unique features of Crail include:</p>
<ul>
- <li>Zero-copy network access from userspace </li>
+ <li>Zero-copy network access from userspace</li>
<li>Integration of multiple storage tiers such DRAM, flash and disaggregated shared storage</li>
<li>Ultra-low latencies for both meta data and data operations. For instance: opening, reading and closing a small file residing in the distributed DRAM tier less than 10 microseconds, which is in the same ballpark as some of the fastest RDMA-based key/value stores</li>
<li>High-performance sequential read/write operations: For instance: read operations on large files residing in the distributed DRAM tier are typically limited only by the performance of the network</li>
@@ -79,7 +79,7 @@
<li>Extensible plugin architecture: new storage tiers tailored to specific hardware can be added easily</li>
</ul>
-<p>Crail is implemented in Java offering a Java API which integrates directly with the Java off-heap memory. Crail is designed for performance critical temporary data within a scope of a rack or two. </p>
+<p>Crail is implemented in Java offering a Java API which integrates directly with the Java off-heap memory. Crail is designed for performance critical temporary data within a scope of a rack or two.</p>
<h2 id="requirements">Requirements</h2>
@@ -99,25 +99,25 @@
<li>Copy tarball to the cluster and unpack it using tar xvfz crail-1.0-bin.tar.gz</li>
</ol>
-<p>Note: later, when deploying Crail, make sure libdisni.so is part of your LD_LIBRARY_PATH. The easiest way to make it work is to copy libdisni.so into crail-1.0/lib </p>
+<p>Note: later, when deploying Crail, make sure libdisni.so is part of your LD_LIBRARY_PATH. The easiest way to make it work is to copy libdisni.so into crail-1.0/lib</p>
<h2 id="configuration">Configuration</h2>
-<p>To configure Crail use crail-site.conf.template as a basis and modify it to match your environment. </p>
+<p>To configure Crail use crail-site.conf.template as a basis and modify it to match your environment.</p>
-<pre><code>cd crail-1.0/conf
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cd crail-1.0/conf
mv crail-site.conf.template crail-site.conf
-</code></pre>
+</code></pre></div></div>
<p>There are a general file system properties and specific properties for the different storage tiers. A typical configuration for the general file system section may look as follows:</p>
-<pre><code>crail.namenode.address crail://namenode:9060
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.namenode.address crail://namenode:9060
crail.storage.types org.apache.crail.storage.rdma.RdmaStorageTier
crail.cachepath /dev/hugepages/cache
crail.cachelimit 12884901888
crail.blocksize 1048576
crail.buffersize 1048576
-</code></pre>
+</code></pre></div></div>
<p>In this configuration the namenode is configured to run using port 9060 on host ‘namenode’, which must be a valid host in the cluster. We further configure a single storage tier, in this case the RDMA-based DRAM tier. The cachepath property needs to point to a directory that is used by the file system to allocate memory for the client cache. Up to cachelimit size, all the memory that is used by Crail will be allocated via mmap from this location. Ideally, the directory specified in cachepath points to a hugetlbfs mountpoint. Aside from the general properties, each storage tier needs to be configured separately.</p>
@@ -125,41 +125,41 @@ crail.buffersize 1048576
<p>For the RDMA/DRAM tier we need to specify the interface that should be used by the storage nodes.</p>
-<pre><code>crail.storage.rdma.interface eth0
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.storage.rdma.interface eth0
+</code></pre></div></div>
<p>The datapath property specifies a path from which the storage nodes will allocate blocks of memory via mmap. Again, that path best points to a hugetlbfs mountpoint.</p>
-<pre><code>crail.storage.rdma.datapath /memory/data
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.storage.rdma.datapath /memory/data
+</code></pre></div></div>
-<p>You want to specify how much DRAM each datanode should donate into the file system pool using the <code>storagelimit</code> property. DRAM is allocated in chunks of <code>allocationsize</code>, which needs to be a multiple of <code>crail.blocksize</code>.</p>
+<p>You want to specify how much DRAM each datanode should donate into the file system pool using the <code class="highlighter-rouge">storagelimit</code> property. DRAM is allocated in chunks of <code class="highlighter-rouge">allocationsize</code>, which needs to be a multiple of <code class="highlighter-rouge">crail.blocksize</code>.</p>
-<pre><code>crail.storage.rdma.allocationsize 1073741824
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.storage.rdma.allocationsize 1073741824
crail.storage.rdma.storagelimit 75161927680
-</code></pre>
+</code></pre></div></div>
<p>Crail supports optimized local operations via memcpy (instead of RDMA) in case a given file operation is backed by a local storage node. The indexpath specifies where Crail will store the necessary metadata that make these optimizations possible. Important: the indexpath must NOT point to a hugetlbfs mountpoint because index files will be updated which not possible in hugetlbfs.</p>
-<pre><code>crail.storage.rdma.localmap true
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.storage.rdma.localmap true
crail.storage.rdma.indexpath /index
-</code></pre>
+</code></pre></div></div>
<h3 id="nvmfflash-storage">NVMf/Flash Storage</h3>
<p>Crail is a multi-tiered storage system. Additinoal tiers can be enabled by adding them to the configuration as follows.</p>
-<pre><code>crail.storage.types org.apache.crail.storage.rdma.RdmaStorageTier,org.apache.crail.storage.nvmf.NvmfStorageTier
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.storage.types org.apache.crail.storage.rdma.RdmaStorageTier,org.apache.crail.storage.nvmf.NvmfStorageTier
+</code></pre></div></div>
-<p>For the NVMf storage tier we need to configure the server IP that is used when listening for new connections. We also need to configure the PCI address of the flash device we want to use, as well as the huge page mount point to be used for allocating memory. </p>
+<p>For the NVMf storage tier we need to configure the server IP that is used when listening for new connections. We also need to configure the PCI address of the flash device we want to use, as well as the huge page mount point to be used for allocating memory.</p>
-<pre><code>crail.storage.nvmf.bindip 10.40.0.XX
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.storage.nvmf.bindip 10.40.0.XX
crail.storage.nvmf.pcieaddr 0000:11:00.0
crail.storage.nvmf.hugedir /dev/hugepages
crail.storage.nvmf.servermempool 512
crail.storage.nvmf.clientmempool 512
-</code></pre>
+</code></pre></div></div>
<h2 id="deploying">Deploying</h2>
@@ -169,40 +169,40 @@ crail.storage.nvmf.clientmempool 512
<p>The simplest way to run Crail is to start it manually on just a handful nodes. You will need to start the Crail namenode, plus at least one datanode. To start the namenode execute the following command on the host that is configured to be the namenode:</p>
-<pre><code>cd crail-1.0/
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cd crail-1.0/
./bin/crail namenode
-</code></pre>
+</code></pre></div></div>
<p>To start a datanode run the following command on a host in the cluster (ideally this is a different physical machine than the one running the namenode):</p>
-<pre><code>./bin/crail datanode
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail datanode
+</code></pre></div></div>
<p>Now you should have a small deployment up with just one datanode. In this case the datanode is of type RDMA/DRAM, which is the default datnode. If you want to start a different storage tier you can do so by passing a specific datanode class as follows:</p>
-<pre><code>./bin/crail datanode -t org.apache.crail.storage.nvmf.NvmfStorageTier
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail datanode -t org.apache.crail.storage.nvmf.NvmfStorageTier
+</code></pre></div></div>
-<p>This would start the shared storage datanode. Note that configuration in crail-site.conf needs to have the specific properties set of this type of datanode, in order for this to work. </p>
+<p>This would start the shared storage datanode. Note that configuration in crail-site.conf needs to have the specific properties set of this type of datanode, in order for this to work.</p>
<h3 id="larger-deployments">Larger deployments</h3>
-<p>To run larger deployments start Crail using </p>
+<p>To run larger deployments start Crail using</p>
-<pre><code>./bin/start-crail.sh
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/start-crail.sh
+</code></pre></div></div>
-<p>Similarly, Crail can be stopped by using </p>
+<p>Similarly, Crail can be stopped by using</p>
-<pre><code>./bin/stop-crail.sh
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/stop-crail.sh
+</code></pre></div></div>
<p>For this to work include the list of machines to start datanodes in conf/slaves. You can start multiple datanode of different types on the same host as follows:</p>
-<pre><code>host02-ib
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>host02-ib
host02-ib -t org.apache.crail.storage.nvmf.NvmfStorageTier
host03-ib
-</code></pre>
+</code></pre></div></div>
<p>In this example, we are configuring a Crail cluster with 2 physical hosts but 3 datanodes and two different storage tiers.</p>
@@ -210,20 +210,20 @@ host03-ib
<p>Crail provides an contains an HDFS adaptor, thus, you can interact with Crail using the HDFS shell:</p>
-<pre><code>./bin/crail fs
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail fs
+</code></pre></div></div>
<p>Crail, however, does not implement the full HDFS shell functionality. The basic commands to copy file to/from Crail, or to move and delete files, will work.</p>
-<pre><code>./bin/crail fs -mkdir /test
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail fs -mkdir /test
./bin/crail fs -ls /
./bin/crail fs -copyFromLocal <path-to-local-file> /test
./bin/crail fs -cat /test/<file-name>
-</code></pre>
+</code></pre></div></div>
<p>For the Crail shell to work properly, the HDFS configuration in crail-1.0/conf/core-site.xml needs to be configured accordingly:</p>
-<pre><code><configuration>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><configuration>
<property>
<name>fs.crail.impl</name>
<value>org.apache.crail.hdfs.CrailHadoopFileSystem</value>
@@ -237,57 +237,57 @@ host03-ib
<value>org.apache.crail.hdfs.CrailHDFS</value>
</property>
</configuration>
-</code></pre>
+</code></pre></div></div>
-<p>Note that the Crail HDFS interface currently cannot provide the full performance of Crail due to limitations of the HDFS API. In particular, the HDFS <code>FSDataOutputStream</code> API only support heap-based <code>byte[]</code> arrays which requires a data copy. Moreover, HDFS operations are synchronous preventing efficient pipelining of operations. Instead, applications that seek the best performance should use the Crail interface directly, as shown next.</p>
+<p>Note that the Crail HDFS interface currently cannot provide the full performance of Crail due to limitations of the HDFS API. In particular, the HDFS <code class="highlighter-rouge">FSDataOutputStream</code> API only support heap-based <code class="highlighter-rouge">byte[]</code> arrays which requires a data copy. Moreover, HDFS operations are synchronous preventing efficient pipelining of operations. Instead, applications that seek the best performance should use the Crail interface directly, as shown next.</p>
<h2 id="programming-against-crail">Programming against Crail</h2>
<p>The best way to program against Crail is to use Maven. Make sure you have the Crail dependency specified in your application pom.xml file:</p>
-<pre><code><dependency>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><dependency>
<groupId>org.apache.crail</groupId>
<artifactId>crail-client</artifactId>
<version>1.0</version>
</dependency>
-</code></pre>
+</code></pre></div></div>
<p>Then, create a Crail client as follows:</p>
-<pre><code>CrailConfiguration conf = new CrailConfiguration();
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailConfiguration conf = new CrailConfiguration();
CrailStore store = CrailStore.newInstance(conf);
-</code></pre>
+</code></pre></div></div>
-<p>Make sure the crail-1.0/conf directory is part of the classpath. </p>
+<p>Make sure the crail-1.0/conf directory is part of the classpath.</p>
<p>Crail supports different file types. The simplest way to create a file in Crail is as follows:</p>
-<pre><code>CrailFile file = store.create(filename, CrailNodeType.DATAFILE, CrailStorageClass.DEFAULT, CrailLocationClass.DEFAULT).get().syncDir();
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailFile file = store.create(filename, CrailNodeType.DATAFILE, CrailStorageClass.DEFAULT, CrailLocationClass.DEFAULT).get().syncDir();
+</code></pre></div></div>
<p>Aside from the actual filename, the ‘create()’ call takes as input the storage and location classes which are preferences for the storage tier and physical location that this file should be created in. Crail tries to satisfy these preferences later when the file is written. In the example we do not request any particular storage or location affinity.</p>
-<p>This ‘create()’ command is non-blocking, calling ‘get()’ on the returning future object awaits the completion of the call. At that time, the file has been created, but its directory entry may not be visible. Therefore, the file may not yet show up in a file enumeration of the given parent directory. Calling ‘syncDir()’ waits to for the directory entry to be completed. Both the ‘get()’ and the ‘syncDir()’ operation can be deffered to a later time at which they may become non-blocking operations. </p>
+<p>This ‘create()’ command is non-blocking, calling ‘get()’ on the returning future object awaits the completion of the call. At that time, the file has been created, but its directory entry may not be visible. Therefore, the file may not yet show up in a file enumeration of the given parent directory. Calling ‘syncDir()’ waits to for the directory entry to be completed. Both the ‘get()’ and the ‘syncDir()’ operation can be deffered to a later time at which they may become non-blocking operations.</p>
<p>Once the file is created, a file stream can be obtained for writing:</p>
-<pre><code>CrailBufferedOutputStream outstream = file.getBufferedOutputStream(1024);
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailBufferedOutputStream outstream = file.getBufferedOutputStream(1024);
+</code></pre></div></div>
<p>Here, we create a buffered stream so that we can pass heap byte arrays as well. We could also create a non-buffered stream using</p>
-<pre><code>CrailOutputStream outstream = file.getDirectOutputStream(1024);
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailOutputStream outstream = file.getDirectOutputStream(1024);
+</code></pre></div></div>
-<p>In both cases, we pass a write hint (1024 in the example) that indicates to Crail how much data we are intending to write. This allows Crail to optimize metadatanode lookups. Crail never prefetches data, but it may fetch the metadata of the very next operation concurrently with the current data operation if the write hint allows to do so. </p>
+<p>In both cases, we pass a write hint (1024 in the example) that indicates to Crail how much data we are intending to write. This allows Crail to optimize metadatanode lookups. Crail never prefetches data, but it may fetch the metadata of the very next operation concurrently with the current data operation if the write hint allows to do so.</p>
<p>Once the stream has been obtained, there exist various ways to write a file. The code snippet below shows the use of the asynchronous interface:</p>
-<pre><code>CrailBuffer dataBuf = fs.allocateBuffer();
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailBuffer dataBuf = fs.allocateBuffer();
Future<DataResult> future = outputStream.write(dataBuf);
...
future.get();
-</code></pre>
+</code></pre></div></div>
<p>Reading files works very similar to writing. There exist various examples in org.apache.crail.tools.CrailBenchmark.</p>
@@ -295,32 +295,32 @@ future.get();
<p>Crail is designed for user-level networking and storage. It does, however, also provide plain TCP-based storage backends for storage and RPC and, thus, can be run easily on any machine without requiring spspecial hardware support. The TCP storage backend can be enabled as follows:</p>
-<pre><code>crail.storage.types org.apache.crail.storage.tcp.TcpStorageTier
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.storage.types org.apache.crail.storage.tcp.TcpStorageTier
+</code></pre></div></div>
<p>The TCP RPC binding can be enabled as follows:</p>
-<pre><code>crail.namenode.rpctype org.apache.crail.namenode.rpc.tcp.TcpNameNode
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.namenode.rpctype org.apache.crail.namenode.rpc.tcp.TcpNameNode
+</code></pre></div></div>
<h2 id="benchmarks">Benchmarks</h2>
<p>Crail provides a set of benchmark tools to measure the performance. Type</p>
-<pre><code>./bin/crail iobench
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench
+</code></pre></div></div>
<p>to get an overview of the available benchmarks. For instance, to benchmark the sequential write performance, type</p>
-<pre><code>./bin/crail iobench -t write -s 1048576 -k 102400 -f /tmp.dat
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t write -s 1048576 -k 102400 -f /tmp.dat
+</code></pre></div></div>
-<p>This will create a file of size 100G, written sequentially in a sequence of 1MB operations. </p>
+<p>This will create a file of size 100G, written sequentially in a sequence of 1MB operations.</p>
<p>To read a file sequentially, type</p>
-<pre><code>./bin/crail iobench -t read -s 1048576 -k 102400 -f /tmp.dat
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t read -s 1048576 -k 102400 -f /tmp.dat
+</code></pre></div></div>
<p>This command issues 102400 read operations of 1MB each.</p>
@@ -328,16 +328,16 @@ future.get();
<h2 id="applications">Applications</h2>
-<p>Crail is used by <a href="https://github.com/zrlio/crail-spark-io">Crail-Spark-IO</a>, a high-performance shuffle engine for Spark. <a href="https://github.com/zrlio/crail-terasort">Crail-Terasort</a> is a fast sorting benchmark for Spark based on Crail. </p>
+<p>Crail is used by <a href="https://github.com/zrlio/crail-spark-io">Crail-Spark-IO</a>, a high-performance shuffle engine for Spark. <a href="https://github.com/zrlio/crail-terasort">Crail-Terasort</a> is a fast sorting benchmark for Spark based on Crail.</p>
<h2 id="contributions">Contributions</h2>
<p>PRs are always welcome. Please fork, and make necessary modifications
-you propose, and let us know. </p>
+you propose, and let us know.</p>
<h2 id="contact">Contact</h2>
-<p>Please join the Crail developer mailing list for discussions and notifications. The list is at: </p>
+<p>Please join the Crail developer mailing list for discussions and notifications. The list is at:</p>
<p>dev@crail.incubator.apache.org.</p>