You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@crail.apache.org by pe...@apache.org on 2018/06/05 08:45:15 UTC
incubator-crail-website git commit: Publishing from
2a99a565c586f0cdab46aad9b10113e54e63676a
Repository: incubator-crail-website
Updated Branches:
refs/heads/asf-site 0d9ceb872 -> 0e5a2e88c
Publishing from 2a99a565c586f0cdab46aad9b10113e54e63676a
Project: http://git-wip-us.apache.org/repos/asf/incubator-crail-website/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-crail-website/commit/0e5a2e88
Tree: http://git-wip-us.apache.org/repos/asf/incubator-crail-website/tree/0e5a2e88
Diff: http://git-wip-us.apache.org/repos/asf/incubator-crail-website/diff/0e5a2e88
Branch: refs/heads/asf-site
Commit: 0e5a2e88c245a664b4725bc35625072d203b2d61
Parents: 0d9ceb8
Author: Jonas Pfefferle <pe...@apache.org>
Authored: Tue Jun 5 10:44:58 2018 +0200
Committer: Jonas Pfefferle <pe...@apache.org>
Committed: Tue Jun 5 10:44:58 2018 +0200
----------------------------------------------------------------------
content/blog/2018/06/apache-release.html | 86 ++++++++++++++
content/blog/2018/06/dataworks.html | 86 ++++++++++++++
content/blog/2018/06/sparksummit.html | 86 ++++++++++++++
content/blog/page4/index.html | 4 +
content/blog/page5/index.html | 154 +++++++++++++++++++++++++
content/feed.xml | 157 +-------------------------
content/index.html | 20 ++--
content/news/index.html | 24 ++++
8 files changed, 452 insertions(+), 165 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/0e5a2e88/content/blog/2018/06/apache-release.html
----------------------------------------------------------------------
diff --git a/content/blog/2018/06/apache-release.html b/content/blog/2018/06/apache-release.html
new file mode 100644
index 0000000..3c0dc23
--- /dev/null
+++ b/content/blog/2018/06/apache-release.html
@@ -0,0 +1,86 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title>The Apache Crail (Incubating) Project: Apache Release</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <link href="http://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="http://crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Atom"
+ href="http://crail.incubator.apache.org/blog/blog.xml">
+
+ <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/apache-release-summary.png" />
+ <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/apache-release-summary.png" />
+ </head>
+
+ <body>
+ <div class="container">
+ <div class="header">
+ <ul class="nav nav-pills pull-right">
+
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/">
+ Home
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/overview/">
+ Overview
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/blog/">
+ Blog
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/community/">
+ Community
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/documentation/">
+ Documentation
+ </a>
+ </li>
+
+ </ul>
+ <a href="http://crail.incubator.apache.org/">
+ <img src="http://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="http://crail.incubator.apache.org/img/crail_logo.png"
+ alt="Crail" id="logo">
+ </a>
+ </div>
+
+
+
+ <h2>Apache Release</h2>
+
+
+ <p>Apache Crail 1.0 incubator <a href="https://dist.apache.org/repos/dist/release/incubator/crail/1.0-incubating/">release</a></p>
+
+
+ <br>
+ <br>
+ <div class="footer">
+ <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/">incubation</a> at <a href="https://www.apache.org/">The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
+ </p>
+ </div>
+
+ </div> <!-- /container -->
+
+ <!-- Support retina images. -->
+ <script type="text/javascript"
+ src="http://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/0e5a2e88/content/blog/2018/06/dataworks.html
----------------------------------------------------------------------
diff --git a/content/blog/2018/06/dataworks.html b/content/blog/2018/06/dataworks.html
new file mode 100644
index 0000000..5c738f5
--- /dev/null
+++ b/content/blog/2018/06/dataworks.html
@@ -0,0 +1,86 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title>The Apache Crail (Incubating) Project: Dataworks</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <link href="http://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="http://crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Atom"
+ href="http://crail.incubator.apache.org/blog/blog.xml">
+
+ <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/dataworks-summary.png" />
+ <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/dataworks-summary.png" />
+ </head>
+
+ <body>
+ <div class="container">
+ <div class="header">
+ <ul class="nav nav-pills pull-right">
+
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/">
+ Home
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/overview/">
+ Overview
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/blog/">
+ Blog
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/community/">
+ Community
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/documentation/">
+ Documentation
+ </a>
+ </li>
+
+ </ul>
+ <a href="http://crail.incubator.apache.org/">
+ <img src="http://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="http://crail.incubator.apache.org/img/crail_logo.png"
+ alt="Crail" id="logo">
+ </a>
+ </div>
+
+
+
+ <h2>Dataworks</h2>
+
+
+ <p>We are presenting Crail at the <a href="https://dataworkssummit.com/san-jose-2018/session/data-processing-at-the-speed-of-100-gbpsapache-crail-incubating/">Data Works Summit</a> on June 21st</p>
+
+
+ <br>
+ <br>
+ <div class="footer">
+ <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/">incubation</a> at <a href="https://www.apache.org/">The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
+ </p>
+ </div>
+
+ </div> <!-- /container -->
+
+ <!-- Support retina images. -->
+ <script type="text/javascript"
+ src="http://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/0e5a2e88/content/blog/2018/06/sparksummit.html
----------------------------------------------------------------------
diff --git a/content/blog/2018/06/sparksummit.html b/content/blog/2018/06/sparksummit.html
new file mode 100644
index 0000000..914b2ef
--- /dev/null
+++ b/content/blog/2018/06/sparksummit.html
@@ -0,0 +1,86 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title>The Apache Crail (Incubating) Project: Sparksummit</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <link href="http://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="http://crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Atom"
+ href="http://crail.incubator.apache.org/blog/blog.xml">
+
+ <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/sparksummit-summary.png" />
+ <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/sparksummit-summary.png" />
+ </head>
+
+ <body>
+ <div class="container">
+ <div class="header">
+ <ul class="nav nav-pills pull-right">
+
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/">
+ Home
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/overview/">
+ Overview
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/blog/">
+ Blog
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/community/">
+ Community
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/documentation/">
+ Documentation
+ </a>
+ </li>
+
+ </ul>
+ <a href="http://crail.incubator.apache.org/">
+ <img src="http://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="http://crail.incubator.apache.org/img/crail_logo.png"
+ alt="Crail" id="logo">
+ </a>
+ </div>
+
+
+
+ <h2>Sparksummit</h2>
+
+
+ <p>We are presenting Crail at the <a href="https://databricks.com/session/serverless-machine-learning-on-modern-hardware-using-apache-spark">Spark Summit</a> in San Francisco today</p>
+
+
+ <br>
+ <br>
+ <div class="footer">
+ <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/">incubation</a> at <a href="https://www.apache.org/">The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
+ </p>
+ </div>
+
+ </div> <!-- /container -->
+
+ <!-- Support retina images. -->
+ <script type="text/javascript"
+ src="http://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/0e5a2e88/content/blog/page4/index.html
----------------------------------------------------------------------
diff --git a/content/blog/page4/index.html b/content/blog/page4/index.html
index aabfb8b..606469c 100644
--- a/content/blog/page4/index.html
+++ b/content/blog/page4/index.html
@@ -133,6 +133,10 @@
</li>
+ <li class="next">
+ <a href="/blog/page5">Older Posts →</a>
+ </li>
+
</ul>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/0e5a2e88/content/blog/page5/index.html
----------------------------------------------------------------------
diff --git a/content/blog/page5/index.html b/content/blog/page5/index.html
new file mode 100644
index 0000000..bc1e12f
--- /dev/null
+++ b/content/blog/page5/index.html
@@ -0,0 +1,154 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title>The Apache Crail (Incubating) Project: Latest posts</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <link href="http://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="http://crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Atom"
+ href="http://crail.incubator.apache.org/blog/blog.xml">
+
+ <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/page5-summary.png" />
+ <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/page5-summary.png" />
+ </head>
+
+ <body>
+ <div class="container">
+ <div class="header">
+ <ul class="nav nav-pills pull-right">
+
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/">
+ Home
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/overview/">
+ Overview
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/blog/">
+ Blog
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/community/">
+ Community
+ </a>
+ </li>
+
+
+ <li >
+ <a href="http://crail.incubator.apache.org/documentation/">
+ Documentation
+ </a>
+ </li>
+
+ </ul>
+ <a href="http://crail.incubator.apache.org/">
+ <img src="http://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="http://crail.incubator.apache.org/img/crail_logo.png"
+ alt="Crail" id="logo">
+ </a>
+ </div>
+
+
+
+ <h2>Latest posts</h2>
+
+
+
+<div class="post-preview">
+ <a href="/blog/2017/11/crail-metadata.html">
+ <h2 class="post-title">
+ Crail Storage Performance -- Part III: Metadata
+ </h2>
+
+ </a>
+ <p class="post-meta">Posted by Adrian Schuepbach and Patrick Stuedi on November 21, 2017</p>
+</div>
+<hr>
+
+<div class="post-preview">
+ <a href="/blog/2017/11/rdmashuffle.html">
+ <h2 class="post-title">
+ Spark Shuffle: SparkRDMA vs Crail
+ </h2>
+
+ </a>
+ <p class="post-meta">Posted by Jonas Pfefferle, Patrick Stuedi, Animesh Trivedi, Bernard Metzler, Adrian Schuepbach on November 17, 2017</p>
+</div>
+<hr>
+
+<div class="post-preview">
+ <a href="/blog/2017/08/crail-nvme-fabrics-v1.html">
+ <h2 class="post-title">
+ Crail Storage Performance -- Part II: NVMf
+ </h2>
+
+ </a>
+ <p class="post-meta">Posted by Jonas Pfefferle on August 22, 2017</p>
+</div>
+<hr>
+
+<div class="post-preview">
+ <a href="/blog/2017/08/crail-memory.html">
+ <h2 class="post-title">
+ Crail Storage Performance -- Part I: DRAM
+ </h2>
+
+ </a>
+ <p class="post-meta">Posted by Patrick Stuedi on August 18, 2017</p>
+</div>
+<hr>
+
+<div class="post-preview">
+ <a href="/blog/2017/01/sorting.html">
+ <h2 class="post-title">
+ Sorting on a 100Gbit/s Cluster using Spark/Crail
+ </h2>
+
+ </a>
+ <p class="post-meta">Posted by Patrick Stuedi on January 17, 2017</p>
+</div>
+<hr>
+
+
+<!-- Pager -->
+
+<ul class="pager">
+
+ <li class="previous">
+ <a href="/blog/page4">← Newer Posts</a>
+ </li>
+
+
+</ul>
+
+
+
+
+ <br>
+ <br>
+ <div class="footer">
+ <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/">incubation</a> at <a href="https://www.apache.org/">The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
+ </p>
+ </div>
+
+ </div> <!-- /container -->
+
+ <!-- Support retina images. -->
+ <script type="text/javascript"
+ src="http://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/0e5a2e88/content/feed.xml
----------------------------------------------------------------------
diff --git a/content/feed.xml b/content/feed.xml
index 11a1189..77da4c0 100644
--- a/content/feed.xml
+++ b/content/feed.xml
@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.6.2">Jekyll</generator><link href="http://crail.incubator.apache.org//feed.xml" rel="self" type="application/atom+xml" /><link href="http://crail.incubator.apache.org//" rel="alternate" type="text/html" /><updated>2018-03-09T13:02:04+01:00</updated><id>http://crail.incubator.apache.org//</id><title type="html">The Apache Crail (Incubating) Project</title><entry><title type="html">Apache</title><link href="http://crail.incubator.apache.org//blog/2018/01/apache.html" rel="alternate" type="text/html" title="Apache" /><published>2018-01-22T00:00:00+01:00</published><updated>2018-01-22T00:00:00+01:00</updated><id>http://crail.incubator.apache.org//blog/2018/01/apache</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2018/01/apache.html"><p>Crail is now an Apache Incubator project!</p></content><author><name></name></author
><category term="news" /><summary type="html">Crail is now an Apache Incubator project!</summary></entry><entry><title type="html">Iops</title><link href="http://crail.incubator.apache.org//blog/2017/11/iops.html" rel="alternate" type="text/html" title="Iops" /><published>2017-11-23T00:00:00+01:00</published><updated>2017-11-23T00:00:00+01:00</updated><id>http://crail.incubator.apache.org//blog/2017/11/iops</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2017/11/iops.html"><p>New blog <a href="http://crail.incubator.apache.org/blog/2017/11/crail-metadata.html">post</a> about Crail’s metadata performance and scalability</p></content><author><name></name></author><category term="news" /><summary type="html">New blog post about Crail’s metadata performance and scalability</summary></entry><entry><title type="html">Crail Storage Performance – Part III: Metadata</title><link href="http://crail.incubator.apache.org//blog/2
017/11/crail-metadata.html" rel="alternate" type="text/html" title="Crail Storage Performance -- Part III: Metadata" /><published>2017-11-21T00:00:00+01:00</published><updated>2017-11-21T00:00:00+01:00</updated><id>http://crail.incubator.apache.org//blog/2017/11/crail-metadata</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2017/11/crail-metadata.html"><div style="text-align: justify">
+<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.6.2">Jekyll</generator><link href="http://crail.incubator.apache.org//feed.xml" rel="self" type="application/atom+xml" /><link href="http://crail.incubator.apache.org//" rel="alternate" type="text/html" /><updated>2018-06-05T10:44:56+02:00</updated><id>http://crail.incubator.apache.org//</id><title type="html">The Apache Crail (Incubating) Project</title><entry><title type="html">Sparksummit</title><link href="http://crail.incubator.apache.org//blog/2018/06/sparksummit.html" rel="alternate" type="text/html" title="Sparksummit" /><published>2018-06-05T00:00:00+02:00</published><updated>2018-06-05T00:00:00+02:00</updated><id>http://crail.incubator.apache.org//blog/2018/06/sparksummit</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2018/06/sparksummit.html"><p>We are presenting Crail at the <a href="https://databrick
s.com/session/serverless-machine-learning-on-modern-hardware-using-apache-spark">Spark Summit</a> in San Francisco today</p></content><author><name></name></author><category term="news" /><summary type="html">We are presenting Crail at the Spark Summit in San Francisco today</summary></entry><entry><title type="html">Dataworks</title><link href="http://crail.incubator.apache.org//blog/2018/06/dataworks.html" rel="alternate" type="text/html" title="Dataworks" /><published>2018-06-05T00:00:00+02:00</published><updated>2018-06-05T00:00:00+02:00</updated><id>http://crail.incubator.apache.org//blog/2018/06/dataworks</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2018/06/dataworks.html"><p>We are presenting Crail at the <a href="https://dataworkssummit.com/san-jose-2018/session/data-processing-at-the-speed-of-100-gbpsapache-crail-incubating/">Data Works Summit</a> on June 21st</p></content><author><name></na
me></author><category term="news" /><summary type="html">We are presenting Crail at the Data Works Summit on June 21st</summary></entry><entry><title type="html">Apache Release</title><link href="http://crail.incubator.apache.org//blog/2018/06/apache-release.html" rel="alternate" type="text/html" title="Apache Release" /><published>2018-06-04T00:00:00+02:00</published><updated>2018-06-04T00:00:00+02:00</updated><id>http://crail.incubator.apache.org//blog/2018/06/apache-release</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2018/06/apache-release.html"><p>Apache Crail 1.0 incubator <a href="https://dist.apache.org/repos/dist/release/incubator/crail/1.0-incubating/">release</a></p></content><author><name></name></author><category term="news" /><summary type="html">Apache Crail 1.0 incubator release</summary></entry><entry><title type="html">Apache</title><link href="http://crail.incubator.apache.org//blog/2018/01/apache.htm
l" rel="alternate" type="text/html" title="Apache" /><published>2018-01-22T00:00:00+01:00</published><updated>2018-01-22T00:00:00+01:00</updated><id>http://crail.incubator.apache.org//blog/2018/01/apache</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2018/01/apache.html"><p>Crail is now an Apache Incubator project!</p></content><author><name></name></author><category term="news" /><summary type="html">Crail is now an Apache Incubator project!</summary></entry><entry><title type="html">Iops</title><link href="http://crail.incubator.apache.org//blog/2017/11/iops.html" rel="alternate" type="text/html" title="Iops" /><published>2017-11-23T00:00:00+01:00</published><updated>2017-11-23T00:00:00+01:00</updated><id>http://crail.incubator.apache.org//blog/2017/11/iops</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2017/11/iops.html"><p>New blog <a href="http://crail.incubator.apache.org/blog/2017/11/crail-metad
ata.html">post</a> about Crail’s metadata performance and scalability</p></content><author><name></name></author><category term="news" /><summary type="html">New blog post about Crail’s metadata performance and scalability</summary></entry><entry><title type="html">Crail Storage Performance – Part III: Metadata</title><link href="http://crail.incubator.apache.org//blog/2017/11/crail-metadata.html" rel="alternate" type="text/html" title="Crail Storage Performance -- Part III: Metadata" /><published>2017-11-21T00:00:00+01:00</published><updated>2017-11-21T00:00:00+01:00</updated><id>http://crail.incubator.apache.org//blog/2017/11/crail-metadata</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2017/11/crail-metadata.html"><div style="text-align: justify">
<p>
This is part III of our series of posts discussing Crail's raw storage performance. This part is about Crail's metadata performance and scalability.
</p>
@@ -667,157 +667,4 @@ In the following 200G Terasort experiment we gradually limit the DRAM resources
<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-nvmf/tiering.svg" width="550" /></div>
-<p>To summarize, in this blog we have shown that the NVMf storage backend for Crail – due to its efficient user-level implementation – offers latencies and throughput very close to the hardware speed. The Crail NVMf storage tier can be used conveniently in combination with the Crail DRAM tier to either save cost or to handle situations where the available DRAM is not sufficient to store the working set of a data processing workload.</p></content><author><name>Jonas Pfefferle</name></author><category term="blog" /><summary type="html">This is part II of our series of posts discussing Crail's raw storage performance. This part is about Crail's NVMe storage tier, a low-latency flash storage backend for Crail completely based on user-level storage access.</summary></entry><entry><title type="html">Crail Storage Performance – Part I: DRAM</title><link href="http://crail.incubator.apache.org//blog/2017/08/crail-memory.html" rel="alternate" type="text/html" title="Crail S
torage Performance -- Part I: DRAM" /><published>2017-08-18T00:00:00+02:00</published><updated>2017-08-18T00:00:00+02:00</updated><id>http://crail.incubator.apache.org//blog/2017/08/crail-memory</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2017/08/crail-memory.html"><div style="text-align: justify">
-<p>
-It's summer and there is some time to blog about things. This blog post is the first in a series of three posts where we illustrate Crail's raw storage performance on our 100Gbps cluster. In part I we cover Crail's DRAM storage tier, part II will be about Crail's NVMe flash storage tier, and part III will be about Crail's metadata performance.
-</p>
-<p>
-I recently read the <a href="https://www.usenix.org/conference/atc17/technical-sessions/presentation/lu">Octopus file system</a> Usenix'17 paper, where the authors show Crail performance numbers that do not match the performance we measure on our clusters. Like many other distributed systems, Crail also requires a careful system configuration and wrong or mismatching configuration settings can easily lead to poor performance. Therefore, in this blog we try to point out the key parameter settings that are necessary to obtain proper performance numbers with Crail.
-</p>
-</div>
-
-<h3 id="hardware-configuration">Hardware Configuration</h3>
-
-<p>The specific cluster configuration used for the experiments in this blog:</p>
-
-<ul>
- <li>Cluster
- <ul>
- <li>8 node OpenPower cluster (for Crail)</li>
- <li>2 node X86 cluster (for RAMCloud)</li>
- </ul>
- </li>
- <li>OpenPower Node configuration
- <ul>
- <li>CPU: 2x OpenPOWER Power8 10-core @2.9Ghz</li>
- <li>DRAM: 512GB DDR4</li>
- <li>Network: 1x100Gbit/s Ethernet Mellanox ConnectX-4 EN (Ethernet/RoCE)
- <ul>
- <li>RDMA send/recv latency, ib_send_lat (RTT): 3.1us</li>
- <li>RDMA read latency, ib_read_lat (RTT): 2.3us</li>
- </ul>
- </li>
- </ul>
- </li>
- <li>Software
- <ul>
- <li>RedHat 7.2 with Linux kernel version 4.10.13</li>
- <li>Crail 1.0, internal version 2842</li>
- <li>Alluxio 1.4</li>
- <li>RAMCloud commit f53202398b4720f20b0cdc42732edf48b928b8d7</li>
- </ul>
- </li>
-</ul>
-
-<h3 id="anatomy-of-a-crail-data-operation">Anatomy of a Crail Data Operation</h3>
-
-<div style="text-align: justify">
-<p>
-Data operations in Crail -- such as the reading or writing of files -- are internally composed of metadata operations and actual data transfers. Let's look at a simple Crail application that opens a file and reads the file sequentially:
-</p>
-</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailConfiguration conf = new CrailConfiguration();
-CrailFS fs = CrailFS.newInstance(conf);
-CrailFile file = fs.lookup(filename).get().asFile();
-CrailInputStream stream = file.getDirectInputStream();
-while(stream.available() &gt; 0){
- Future&lt;Buffer&gt; future = stream.read(buf);
- //Do something useful
- ...
- //Await completion of operation
- future.get();
-}
-</code></pre></div></div>
-<div style="text-align: justify">
-<p>
-One challenge with file read/write operations is to avoid blocking in case block metadata information is missing. Crail caches block metadata at the client, but caching is ineffective for both random reads and write-once read-once data. To avoid blocking for sequential read/write operations, Crail interleaves metadata operations and actual data transfers. Each read operation always triggers the lookup of block metadata for the next block immediately after issuing the RDMA read operation for the current block. The asynchronous and non-blocking nature of RDMA allows both operations to be executed in the process context of the application, without context switching or any additional background threads. The figure illustrates the case of one outstanding operation a time. The asynchronous Crail storage API, however, permits any number of outstanding operations.
-</p>
-</div>
-<p><br /></p>
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/anatomy.png" width="420" /></div>
-<p><br /></p>
-<div style="text-align: justify">
-<p>
-As a side note, it's also worth mentioning that Crail does not actually use RPCs for the data transfers but uses RDMA one-sided read/write operations instead. Moreover, Crail is designed from ground up for byte-addressable storage and memory. For instance, files in Crail are essentially a sequence of virtual memory windows on different hosts which allows for a very effective handling of small data operations. As shown in the figure, during the last operation, with only a few bytes left to be read, the byte-granular nature of Crail's block access protocol makes sure that only the relevant bytes are transmitted over the network, as opposed to transmitting the entire block.
-</p>
-<p>
-The basic read/write logic shown in the figure above is common to all storage tiers in Crail, including the NVMe flash tier. In the remainder of this post, we specificially look at the performance of Crail's DRAM storage tier though.
-</p>
-</div>
-
-<h3 id="sequential-readwrite-throughput">Sequential Read/Write Throughput</h3>
-
-<div style="text-align: justify">
-<p>
-Let's start by looking at sequential read/write performance. These benchmarks can be run easily from the command line. Below is an example for a sequential write experiment issuing 100M write operations of size 1K to produce a file of roughly 100GB size. The -w switch indicates that we are using 32 warmup operations.
-</p>
-</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t write -s 1024 -k 100000000 -w 32 -f /tmp.dat
-</code></pre></div></div>
-<div style="text-align: justify">
-<p>
-Crail offers direct I/O streams as well as buffered streams. For sequential operations it is important to use the buffered streams. Even though the buffered streams impose one extra copy (from the Crail stream to the application buffer) they are typically more effective for sequential access as they make sure that at least one network operation is in-flight at any time. The buffer size in a Crail buffered stream and the number of oustanding operations can be controlled by setting the buffersize and the slicesize properties in crail-site.conf. For our experiments we used a 1MB buffer per stream sliced up into two slices of 512K each which eventually leads to two operations in flight.
-</p>
-</div>
-
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.buffersize 1048576
-crail.slicesize 524288
-</code></pre></div></div>
-
-<div style="text-align: justify">
-<p>
-The figure below illustrates the sequential write (top) and read (bottom) performance of Crail (DRAM tier) for different application buffer sizes (not to be mixed up with crail.buffersize used within streams) and shows a comparison to other systems. As of now, we only show a comparison with Alluxio, an in-memory file system for caching data in Spark or Hadoop applications. We are, however, working on including results for other storage systems such as Apache Ignite and GlusterFS and we plan to update the blog post accordingly soon. If there is a particular storage system that is not included but you would like to see included as a comparison, please write us. And <b>important</b>: if you find that the results we show for a particular storage system do not match your experience, please write to us too, we are happy to revisit the configuration.
-</p>
-</div>
-<p><br /></p>
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/write.svg" width="550" /></div>
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/read.svg" width="550" /></div>
-<p><br /><br /></p>
-<div style="text-align: justify">
-<p>
-One first observation from the figure is that there is almost no difference in performance for write and read operations. Second, at a buffer size of around 1K Crail reaches a bandwidth close to 95Gbit/s (for read), which is approaching the network hardware limit of 100Gbps. And third, Crail performs significantly faster than other in-memory storage systems, in this case Alluxio. This because Crail is built on of user-level networking and thereby avoids the overheads of both the Linux network stack (memory copies, context switches, etc.) and the Java runtime.
-</p>
-<p>
-Note that both figures show single-client performance numbers. With Crail being a user-level storage system executing I/O operations directly within the application context this means the entire benchmark is truly runninig on one single core. Often, systems that perform poorly in single-client experiments are being defended saying that nobody cares about the single-client performance. Especially throughput problems can easily be fixed by adding more cores. This is, however, not at all cloudy to say the least. At the level hardware is multiplexed and priced in today's cloud computing data centers every core counts. The figure below shows a simple Spark group-by experiment on the same 8-node cluster. As can be seen, with Crail the benchmark executes faster using a single core per machine than with default Spark using 8 cores per machine, which is a direct consequence from Crail's superb single-core I/O performance.
-</p>
-</div>
-
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/crail-groupby.svg" width="550" /></div>
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/spark-groupby.svg" width="550" /></div>
-
-<h3 id="random-read-latency">Random Read Latency</h3>
-
-<div style="text-align: justify">
-<p>
-Typically, distributed storage systems are either built for sequential access to large data sets (e.g., HDFS) or they are optimized for random access to small data sets (e.g., key/value stores). We have already shown that Crail performs well for large sequentially accessed data sets, let's now look at the latencies of small random read operations. For this, we mimic the behavior of a key/value store by storing key/value pairs in Crail files with the key being the filename. We then measure the time it takes to open the file and read its content. Again, the benchmark can easily be executed from the command line. The following example issues 1M get() operations on a small file filled with a 4 byte value.
-</p>
-</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t getkey -s 4 -k 1000000 -f /tmp.dat -w 32
-</code></pre></div></div>
-<div style="text-align: justify">
-<p>
-The figure below illustrates the latencies of get() operations for different key/value sizes and compares them to the latencies we obtained with RAMCloud for the same type of operations (measured using RAMClouds C and Java APIs). RAMCloud is a low-latency key/value store implemented using RDMA. RAMCloud actually provides durable storage by asynchronously replicating data onto backup devices. However, at any point in time all the data is held in DRAM and read requests will be served from DRAM directly. Up to our knowledge, RAMCloud is the fastest key/value store that is (a) available open source and (b) can be deployed in practice as a storage platform for applications. Other similar RDMA-based storage systems we looked at, like FaRM or HERD, are either not open source or they do not provide a clean separation between storage system, API and clients.
-</p>
-</div>
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/latency.svg" width="550" /></div>
-
-<div style="text-align: justify">
-<p>
-As can be seen from the figure, Crail's latencies for reading small files range from 10us to 20us for files smaller than 256K. The first observation is that these latency numbers are very close to the RAMCloud get() latencies obtained using the RAMCloud C API. Mainly, the latency difference between the two systems comes from the extra network roundtrip that is required in Crail to open the file, an operation which involves the Crail namenode. Once the file size reaches 64K, the cost for the extra roundtrip is amortized and the Crail latencies start to match the RAMCloud latencies. The second observation from the figure is that Crail offers lower latencies than the RAMCloud Java API for key/value sizes of 16K and bigger. This is because Crail, which is implemented in Java itself, integrates natively with the Java memory system. For instance, Crail's raw stream APIs permits clients to pass Java off-heap ByteBuffers which can be accessed by the network interface directly, avoiding data
copies along the way. That being said we also understand that the Java API is not RAMCloud's primary API and could probably be optimized further.
-</p>
-<p>
-All in all the main take away here is that -- despite Crail offering a fully hierchical storage namespace and high-performance operations on large data sets -- the latencies for looking up and reading small data sets are in the same ballpark as the get() latencies of some of the fastest key/value stores out there.
-</p>
-<p>
-The latency advantages of Crail are beneficial also at the application level. The figure below illustrates this in a Spark broadcast experiment. Broadcast objects in Spark are typically small read-only variables that are shared across the cluster. The Crail broadcast module for Spark uses Crail as a storage backend to make broadcast variables accessible by the different tasks. As can be seen, using Crail broadcast objects can be accessed in just a few microseconds, while the same operation in default Spark takes milliseconds.
-</p>
-</div>
-
-<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/crail-memory/cdf-broadcast-128-read.svg" width="550" /></div>
-
-<div style="text-align: justify">
-<p>
-To summarize, in this blog post we have shown that Crail's DRAM storage tier provides both throughput and latency close to the hardware limits. These performance benefits enable high-level data processing operations like shuffle or broadcast to be implemented faster and/or more efficient.
-</p>
-
-</div></content><author><name>Patrick Stuedi</name></author><category term="blog" /><summary type="html">It's summer and there is some time to blog about things. This blog post is the first in a series of three posts where we illustrate Crail's raw storage performance on our 100Gbps cluster. In part I we cover Crail's DRAM storage tier, part II will be about Crail's NVMe flash storage tier, and part III will be about Crail's metadata performance. I recently read the Octopus file system Usenix'17 paper, where the authors show Crail performance numbers that do not match the performance we measure on our clusters. Like many other distributed systems, Crail also requires a careful system configuration and wrong or mismatching configuration settings can easily lead to poor performance. Therefore, in this blog we try to point out the key parameter settings that are necessary to obtain proper performance numbers with Crail.</summary></entry><entry><title type="html">Openpower</title>
<link href="http://crail.incubator.apache.org//blog/2017/08/openpower.html" rel="alternate" type="text/html" title="Openpower" /><published>2017-08-04T00:00:00+02:00</published><updated>2017-08-04T00:00:00+02:00</updated><id>http://crail.incubator.apache.org//blog/2017/08/openpower</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2017/08/openpower.html"><p>Crail on OpenPower discussed by Peter Hofstee on <a href="https://www.youtube.com/watch?v=f-pgMaEmqn4&amp;feature=youtu.be&amp;platform=hootsuite">Youtube</a></p></content><author><name></name></author><category term="news" /><summary type="html">Crail on OpenPower discussed by Peter Hofstee on Youtube</summary></entry><entry><title type="html">Disni</title><link href="http://crail.incubator.apache.org//blog/2017/06/disni.html" rel="alternate" type="text/html" title="Disni" /><published>2017-06-17T00:00:00+02:00</published><updated>2017-06-17T00:00:00+02:00</updat
ed><id>http://crail.incubator.apache.org//blog/2017/06/disni</id><content type="html" xml:base="http://crail.incubator.apache.org//blog/2017/06/disni.html"><p>DiSNI, the RDMA and NVMe user-level stack used in Crail is now available on <a href="https://search.maven.org/">Maven Central</a></p></content><author><name></name></author><category term="news" /><summary type="html">DiSNI, the RDMA and NVMe user-level stack used in Crail is now available on Maven Central</summary></entry></feed>
\ No newline at end of file
+<p>To summarize, in this blog we have shown that the NVMf storage backend for Crail – due to its efficient user-level implementation – offers latencies and throughput very close to the hardware speed. The Crail NVMf storage tier can be used conveniently in combination with the Crail DRAM tier to either save cost or to handle situations where the available DRAM is not sufficient to store the working set of a data processing workload.</p></content><author><name>Jonas Pfefferle</name></author><category term="blog" /><summary type="html">This is part II of our series of posts discussing Crail's raw storage performance. This part is about Crail's NVMe storage tier, a low-latency flash storage backend for Crail completely based on user-level storage access.</summary></entry></feed>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/0e5a2e88/content/index.html
----------------------------------------------------------------------
diff --git a/content/index.html b/content/index.html
index 12cfe33..ef52c74 100644
--- a/content/index.html
+++ b/content/index.html
@@ -100,40 +100,40 @@
<li class="shortnews">
- <span class="date">January 22, 2018</span>
- <p>Crail is now an Apache Incubator project!</p>
+ <span class="date">June 5, 2018</span>
+ <p>We are presenting Crail at the <a href="https://databricks.com/session/serverless-machine-learning-on-modern-hardware-using-apache-spark">Spark Summit</a> in San Francisco today</p>
</li>
<li class="shortnews">
- <span class="date">November 23, 2017</span>
- <p>New blog <a href="http://crail.incubator.apache.org/blog/2017/11/crail-metadata.html">post</a> about Crail’s metadata performance and scalability</p>
+ <span class="date">June 5, 2018</span>
+ <p>We are presenting Crail at the <a href="https://dataworkssummit.com/san-jose-2018/session/data-processing-at-the-speed-of-100-gbpsapache-crail-incubating/">Data Works Summit</a> on June 21st</p>
</li>
<li class="shortnews">
- <span class="date">November 17, 2017</span>
- <p>Crail features in the <a href="https://twit.tv/shows/floss-weekly/episodes/458?autostart=false">FLOSS weekly podcast</a></p>
+ <span class="date">June 4, 2018</span>
+ <p>Apache Crail 1.0 incubator <a href="https://dist.apache.org/repos/dist/release/incubator/crail/1.0-incubating/">release</a></p>
</li>
<li class="shortnews">
- <span class="date">November 17, 2017</span>
- <p>New blog <a href="http://crail.incubator.apache.org/blog/2017/11/rdmashuffle.html">post</a> about SparkRDMA and Crail shuffle plugins</p>
+ <span class="date">January 22, 2018</span>
+ <p>Crail is now an Apache Incubator project!</p>
</li>
<li class="shortnews">
- <span class="date">August 4, 2017</span>
- <p>Crail on OpenPower discussed by Peter Hofstee on <a href="https://www.youtube.com/watch?v=f-pgMaEmqn4&feature=youtu.be&platform=hootsuite">Youtube</a></p>
+ <span class="date">November 23, 2017</span>
+ <p>New blog <a href="http://crail.incubator.apache.org/blog/2017/11/crail-metadata.html">post</a> about Crail’s metadata performance and scalability</p>
</li>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/0e5a2e88/content/news/index.html
----------------------------------------------------------------------
diff --git a/content/news/index.html b/content/news/index.html
index 7fbef87..9d0668a 100644
--- a/content/news/index.html
+++ b/content/news/index.html
@@ -71,6 +71,30 @@
<li class="shortnews">
+ <span class="date">June 5, 2018</span>
+ <p>We are presenting Crail at the <a href="https://databricks.com/session/serverless-machine-learning-on-modern-hardware-using-apache-spark">Spark Summit</a> in San Francisco today</p>
+
+ </li>
+
+
+
+ <li class="shortnews">
+ <span class="date">June 5, 2018</span>
+ <p>We are presenting Crail at the <a href="https://dataworkssummit.com/san-jose-2018/session/data-processing-at-the-speed-of-100-gbpsapache-crail-incubating/">Data Works Summit</a> on June 21st</p>
+
+ </li>
+
+
+
+ <li class="shortnews">
+ <span class="date">June 4, 2018</span>
+ <p>Apache Crail 1.0 incubator <a href="https://dist.apache.org/repos/dist/release/incubator/crail/1.0-incubating/">release</a></p>
+
+ </li>
+
+
+
+ <li class="shortnews">
<span class="date">January 22, 2018</span>
<p>Crail is now an Apache Incubator project!</p>