You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@crail.apache.org by pe...@apache.org on 2021/09/20 13:36:53 UTC
[incubator-crail-website] branch asf-site updated: Publishing from
b5b58adde938e2555bd021df2f93178521be1af4
This is an automated email from the ASF dual-hosted git repository.
pepperjo pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-crail-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new be12485 Publishing from b5b58adde938e2555bd021df2f93178521be1af4
be12485 is described below
commit be12485b527941456e46e56cfbbb807d9ca3d3c3
Author: Jonas Pfefferle <pe...@apache.org>
AuthorDate: Mon Sep 20 15:36:40 2021 +0200
Publishing from b5b58adde938e2555bd021df2f93178521be1af4
---
content/404.html | 32 ++++----
content/Gemfile | 6 --
content/Gemfile.lock | 84 --------------------
content/blog/2017/01/firstrelease.html | 30 +++----
content/blog/2017/01/sorting-post.html | 32 ++++----
content/blog/2017/01/sorting.html | 52 ++++++------
content/blog/2017/01/spark-io.html | 30 +++----
content/blog/2017/03/dataworkssummit.html | 30 +++----
content/blog/2017/03/nvmf.html | 30 +++----
content/blog/2017/03/nvmw.html | 30 +++----
content/blog/2017/03/ofa.html | 30 +++----
content/blog/2017/03/sparksummit.html | 30 +++----
content/blog/2017/06/disni.html | 30 +++----
content/blog/2017/08/crail-memory.html | 52 ++++++------
content/blog/2017/08/crail-nvme-fabrics-v1.html | 62 +++++++--------
content/blog/2017/08/openpower.html | 30 +++----
content/blog/2017/11/blog.html | 32 ++++----
content/blog/2017/11/crail-metadata.html | 56 ++++++-------
content/blog/2017/11/floss.html | 30 +++----
content/blog/2017/11/iops.html | 32 ++++----
content/blog/2017/11/rdmashuffle.html | 34 ++++----
content/blog/2018/01/apache.html | 30 +++----
content/blog/2018/06/apache-release.html | 32 ++++----
content/blog/2018/06/dataworks.html | 30 +++----
content/blog/2018/06/sparksummit.html | 30 +++----
content/blog/2018/08/sql-p1-news.html | 32 ++++----
content/blog/2018/08/sql-p1.html | 40 +++++-----
content/blog/2019/01/python.html | 56 ++++++-------
content/blog/2019/03/deployment.html | 44 +++++------
content/blog/2019/03/disaggregation.html | 50 ++++++------
content/blog/2019/04/strata.html | 30 +++----
content/blog/2019/06/ycsb.html | 30 +++----
content/blog/2019/08/atc.html | 30 +++----
content/blog/2019/10/ycsb.html | 54 ++++++-------
content/blog/2020/01/release_v1.2.html | 30 +++----
content/blog/index.html | 30 +++----
content/blog/page2/index.html | 30 +++----
content/blog/page3/index.html | 30 +++----
content/blog/page4/index.html | 30 +++----
content/blog/page5/index.html | 30 +++----
content/blog/page6/index.html | 30 +++----
content/blog/page7/index.html | 30 +++----
content/community/index.html | 30 +++----
content/documentation/index.html | 32 ++++----
content/download/index.html | 40 +++++-----
content/feed.xml | 100 ++++++++++++------------
content/index.html | 36 ++++-----
content/news/index.html | 40 +++++-----
content/overview/index.html | 44 +++++------
49 files changed, 867 insertions(+), 957 deletions(-)
diff --git a/content/404.html b/content/404.html
index 9b93dbb..9f73387 100644
--- a/content/404.html
+++ b/content/404.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: 404 Not Found</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/404-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/404-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/404-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/404-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -76,7 +76,7 @@
<p>Sorry! The server can’t find that page.</p>
-<p>Please consider trying to find what you need from <a href="//crail.incubator.apache.org/">the home page</a>.</p>
+<p>Please consider trying to find what you need from <a href="https://crail.incubator.apache.org/">the home page</a>.</p>
<p>If you think something might be broken, please notify the developers at the <a href="mailto:dev@crail.incubator.apache.org">dev mailing list</a>.</p>
@@ -93,6 +93,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/Gemfile b/content/Gemfile
deleted file mode 100644
index d204088..0000000
--- a/content/Gemfile
+++ /dev/null
@@ -1,6 +0,0 @@
-source 'https://rubygems.org'
-gem 'rouge'
-gem 'jekyll-paginate'
-gem 'jekyll-feed'
-gem 'jekyll-gist'
-gem 'jekyll-oembed', :require => 'jekyll_oembed'
diff --git a/content/Gemfile.lock b/content/Gemfile.lock
deleted file mode 100644
index 89d0b51..0000000
--- a/content/Gemfile.lock
+++ /dev/null
@@ -1,84 +0,0 @@
-GEM
- remote: https://rubygems.org/
- specs:
- addressable (2.5.2)
- public_suffix (>= 2.0.2, < 4.0)
- colorator (1.1.0)
- concurrent-ruby (1.1.3)
- em-websocket (0.5.1)
- eventmachine (>= 0.12.9)
- http_parser.rb (~> 0.6.0)
- eventmachine (1.2.7)
- faraday (0.15.4)
- multipart-post (>= 1.2, < 3)
- ffi (1.9.25)
- forwardable-extended (2.6.0)
- http_parser.rb (0.6.0)
- i18n (0.9.5)
- concurrent-ruby (~> 1.0)
- jekyll (3.8.5)
- addressable (~> 2.4)
- colorator (~> 1.0)
- em-websocket (~> 0.5)
- i18n (~> 0.7)
- jekyll-sass-converter (~> 1.0)
- jekyll-watch (~> 2.0)
- kramdown (~> 1.14)
- liquid (~> 4.0)
- mercenary (~> 0.3.3)
- pathutil (~> 0.9)
- rouge (>= 1.7, < 4)
- safe_yaml (~> 1.0)
- jekyll-feed (0.11.0)
- jekyll (~> 3.3)
- jekyll-gist (1.5.0)
- octokit (~> 4.2)
- jekyll-oembed (0.0.1)
- jekyll
- ruby-oembed (= 0.8.8)
- jekyll-paginate (1.1.0)
- jekyll-sass-converter (1.5.2)
- sass (~> 3.4)
- jekyll-watch (2.1.2)
- listen (~> 3.0)
- kramdown (1.17.0)
- liquid (4.0.1)
- listen (3.1.5)
- rb-fsevent (~> 0.9, >= 0.9.4)
- rb-inotify (~> 0.9, >= 0.9.7)
- ruby_dep (~> 1.2)
- mercenary (0.3.6)
- multipart-post (2.0.0)
- octokit (4.13.0)
- sawyer (~> 0.8.0, >= 0.5.3)
- pathutil (0.16.2)
- forwardable-extended (~> 2.6)
- public_suffix (3.0.3)
- rb-fsevent (0.10.3)
- rb-inotify (0.9.10)
- ffi (>= 0.5.0, < 2)
- rouge (3.3.0)
- ruby-oembed (0.8.8)
- ruby_dep (1.5.0)
- safe_yaml (1.0.4)
- sass (3.7.2)
- sass-listen (~> 4.0.0)
- sass-listen (4.0.0)
- rb-fsevent (~> 0.9, >= 0.9.4)
- rb-inotify (~> 0.9, >= 0.9.7)
- sawyer (0.8.1)
- addressable (>= 2.3.5, < 2.6)
- faraday (~> 0.8, < 1.0)
-
-PLATFORMS
- ruby
-
-DEPENDENCIES
- jekyll-feed
- jekyll-gist
- jekyll-oembed
- jekyll-paginate
- rouge
-
-BUNDLED WITH
- 1.15.4
diff --git a/content/blog/2017/01/firstrelease.html b/content/blog/2017/01/firstrelease.html
index cc52338..cadcedf 100644
--- a/content/blog/2017/01/firstrelease.html
+++ b/content/blog/2017/01/firstrelease.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Firstrelease</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/firstrelease-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/firstrelease-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/firstrelease-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/firstrelease-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/01/sorting-post.html b/content/blog/2017/01/sorting-post.html
index 9ba4634..3acd36d 100644
--- a/content/blog/2017/01/sorting-post.html
+++ b/content/blog/2017/01/sorting-post.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Sorting Post</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/sorting-post-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/sorting-post-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/sorting-post-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/sorting-post-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -74,7 +74,7 @@
<h2>Sorting Post</h2>
- <p>New blog <a href="//crail.incubator.apache.org/blog/2017/01/sorting.html">post</a> about sorting on a 100Gbit/s cluster using Spark/Crail</p>
+ <p>New blog <a href="https://crail.incubator.apache.org/blog/2017/01/sorting.html">post</a> about sorting on a 100Gbit/s cluster using Spark/Crail</p>
<br>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/01/sorting.html b/content/blog/2017/01/sorting.html
index 89967e5..3cd3a9a 100644
--- a/content/blog/2017/01/sorting.html
+++ b/content/blog/2017/01/sorting.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Sorting on a 100Gbit/s Cluster using Spark/Crail</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/sorting-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/sorting-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/sorting-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/sorting-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -190,7 +190,7 @@ A Spark sorting job consists of two phases. The first phase is a mapping or cla
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sort/terasort_pipeline.png" width="490" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sort/terasort_pipeline.png" width="490" /></div>
<p><br /><br /></p>
<h3 id="using-vanilla-spark">Using Vanilla Spark</h3>
@@ -202,7 +202,7 @@ The first question we are interested in is to what extent such a sorting benchma
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sort/net_vanilla.svg" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sort/net_vanilla.svg" /></div>
<p><br /><br /></p>
<div style="text-align: justify">
@@ -212,7 +212,7 @@ The poor network usage matches with the general observation we made in our previ
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sort/cpu_network.svg" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sort/cpu_network.svg" /></div>
<p><br /><br /></p>
<div style="text-align: justify">
@@ -225,12 +225,12 @@ Consequently, to improve the runtime of the sorting benchmark and to make good u
<div style="text-align: justify">
<p>
-An overview of the Crail shuffler is provided in the <a href="//crail.incubator.apache.org/docs">documentation section</a>. The main difference between the Crail shuffler and the Spark built-in shuffler lies in the way data from the network is processed in a reduce task. The Spark shuffler is based on TCP sockets, thus, many CPU instructions are necessary to bring the data from the networking interface to the buffer inside Spark. In contrast, the Crail shuffler shares shuffle data throug [...]
+An overview of the Crail shuffler is provided in the <a href="https://crail.incubator.apache.org/docs">documentation section</a>. The main difference between the Crail shuffler and the Spark built-in shuffler lies in the way data from the network is processed in a reduce task. The Spark shuffler is based on TCP sockets, thus, many CPU instructions are necessary to bring the data from the networking interface to the buffer inside Spark. In contrast, the Crail shuffler shares shuffle data [...]
</p>
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sort/shuffle_rdma.png" width="470" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sort/shuffle_rdma.png" width="470" /></div>
<p><br /><br /></p>
<div style="text-align: justify">
@@ -239,7 +239,7 @@ During the map phase, the Crail shuffler organizes each key range in a set of Cr
</p>
<p>
-As illustrated in the <a href="//crail.incubator.apache.org/docs">documentation section</a>, the Crail shuffler allows applications to use their own custom serializer and sorter. The recommended serializer for Spark is Kryo, which is a generic serializer. Being generic, however, comes at a cost. Specifically, Kryo requires more type information to be stored along with the serialized data than a custom serializer would need, and also the parsing is more complex for a generic serializer. O [...]
+As illustrated in the <a href="https://crail.incubator.apache.org/docs">documentation section</a>, the Crail shuffler allows applications to use their own custom serializer and sorter. The recommended serializer for Spark is Kryo, which is a generic serializer. Being generic, however, comes at a cost. Specifically, Kryo requires more type information to be stored along with the serialized data than a custom serializer would need, and also the parsing is more complex for a generic seriali [...]
</p>
<p>
As with serialization, the Spark built-in sorter is a general purpose TimSort that can sort arbitrary collections of comparable objects. In our benchmark, we instruct the Crail shuffler to use a Radix sorter instead. The Radix sorter cannot be applied to arbitrary objects but works well for keys of a fixed byte length. The standard pipeline of a reduce task is to first deserialize the data and then sort it. In the particular configuration of the Crail shuffler, we turn these two steps ar [...]
@@ -255,13 +255,13 @@ The figure below shows the overall performance of Spark/Crail vs Spark/Vanilla o
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sort/performance_overall.png" width="470" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sort/performance_overall.png" width="470" /></div>
<p><br /><br /></p>
<p>One key question of interest is about the network usage of the Crail shuffler during the sorting benchmark. In the figure below, we show the data rate at which the different reduce tasks fetch data from the network. Each point in the figure corresponds to one reduce task. In our configuration, we run 3 Spark executors per node and 5 Spark cores per executor. Thus, 1920 reduce tasks are running concurrently (out of 6400 reduce tasks in total) generating a cluster-wide all-to-all traffi [...]
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sort/multiread.svg" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sort/multiread.svg" /></div>
<p><br /><br /></p>
<p>In this blog post, we have shown that Crail successfully manages to translate the raw network performance into actual workload level gains. The exercise with TeraSort as an application validates the design decisions we made in Crail. Stay tuned for more results with different workloads and hardware configurations.</p>
@@ -271,8 +271,8 @@ The figure below shows the overall performance of Spark/Crail vs Spark/Vanilla o
<p>All the components required to run the sorting benchmark using Spark/Crail are open source. Here is some guidance how to run the benchmark:</p>
<ul>
- <li>Build and deploy Crail using the instructions at <a href="//crail.incubator.apache.org/documentation#crail">documentation</a></li>
- <li>Enable the Crail shuffler for Spark by building Spark-IO using the instructions at <a href="//crail.incubator.apache.org/documentation#spark">documentation</a></li>
+ <li>Build and deploy Crail using the instructions at <a href="https://crail.incubator.apache.org/documentation#crail">documentation</a></li>
+ <li>Enable the Crail shuffler for Spark by building Spark-IO using the instructions at <a href="https://crail.incubator.apache.org/documentation#spark">documentation</a></li>
<li>Configure the DRAM storage tier of Crail so that all the shuffle data fits into the DRAM tier.</li>
<li>Build the sorting benchmark using the instructions on <a href="https://github.com/zrlio/crail-terasort">GitHub</a></li>
<li>Make sure you have the custom serializer and sorter specified in spark-defaults.conf</li>
@@ -280,7 +280,7 @@ The figure below shows the overall performance of Spark/Crail vs Spark/Vanilla o
<li>Run the Crail-TeraSort on your Spark cluster. The command line we have used on the 128 node cluster is the following:</li>
</ul>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/spark-submit -v --num-executors 384 --executor-cores 5 --executor-memory 64G
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/spark-submit -v --num-executors 384 --executor-cores 5 --executor-memory 64G
--driver-memory 64G --master yarn
--class com.ibm.crail.terasort.TeraSort path/to/crail-terasort-2.0.jar
-i /terasort-input-1280g -o /terasort-output-1280g
@@ -308,6 +308,6 @@ The figure below shows the overall performance of Spark/Crail vs Spark/Vanilla o
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/01/spark-io.html b/content/blog/2017/01/spark-io.html
index fc9f132..6646135 100644
--- a/content/blog/2017/01/spark-io.html
+++ b/content/blog/2017/01/spark-io.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Spark Io</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/spark-io-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/spark-io-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/spark-io-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/spark-io-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/03/dataworkssummit.html b/content/blog/2017/03/dataworkssummit.html
index 5fc0fee..ab492aa 100644
--- a/content/blog/2017/03/dataworkssummit.html
+++ b/content/blog/2017/03/dataworkssummit.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Dataworkssummit</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/dataworkssummit-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/dataworkssummit-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/dataworkssummit-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/dataworkssummit-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/03/nvmf.html b/content/blog/2017/03/nvmf.html
index 7b05314..6aa60f9 100644
--- a/content/blog/2017/03/nvmf.html
+++ b/content/blog/2017/03/nvmf.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Nvmf</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/nvmf-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/nvmf-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/nvmf-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/nvmf-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/03/nvmw.html b/content/blog/2017/03/nvmw.html
index 8c9211e..1cfc4ef 100644
--- a/content/blog/2017/03/nvmw.html
+++ b/content/blog/2017/03/nvmw.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Nvmw</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/nvmw-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/nvmw-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/nvmw-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/nvmw-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/03/ofa.html b/content/blog/2017/03/ofa.html
index 498a05b..6eab518 100644
--- a/content/blog/2017/03/ofa.html
+++ b/content/blog/2017/03/ofa.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Ofa</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/ofa-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/ofa-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/ofa-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/ofa-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/03/sparksummit.html b/content/blog/2017/03/sparksummit.html
index d8f203e..6b32200 100644
--- a/content/blog/2017/03/sparksummit.html
+++ b/content/blog/2017/03/sparksummit.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Sparksummit</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/sparksummit-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/sparksummit-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/sparksummit-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/sparksummit-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/06/disni.html b/content/blog/2017/06/disni.html
index 4e2fbdb..4416bb2 100644
--- a/content/blog/2017/06/disni.html
+++ b/content/blog/2017/06/disni.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Disni</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/disni-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/disni-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/disni-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/disni-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/08/crail-memory.html b/content/blog/2017/08/crail-memory.html
index ea9c7a9..7588bba 100644
--- a/content/blog/2017/08/crail-memory.html
+++ b/content/blog/2017/08/crail-memory.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Crail Storage Performance -- Part I: DRAM</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/crail-memory-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/crail-memory-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/crail-memory-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/crail-memory-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -126,7 +126,7 @@ I recently read the <a href="https://www.usenix.org/conference/atc17/technical-s
Data operations in Crail -- such as the reading or writing of files -- are internally composed of metadata operations and actual data transfers. Let's look at a simple Crail application that opens a file and reads the file sequentially:
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailConfiguration conf = new CrailConfiguration();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailConfiguration conf = new CrailConfiguration();
CrailFS fs = CrailFS.newInstance(conf);
CrailFile file = fs.lookup(filename).get().asFile();
CrailInputStream stream = file.getDirectInputStream();
@@ -144,7 +144,7 @@ One challenge with file read/write operations is to avoid blocking in case block
</p>
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-memory/anatomy.png" width="420" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-memory/anatomy.png" width="420" /></div>
<p><br /></p>
<div style="text-align: justify">
<p>
@@ -162,7 +162,7 @@ The basic read/write logic shown in the figure above is common to all storage ti
Let's start by looking at sequential read/write performance. These benchmarks can be run easily from the command line. Below is an example for a sequential write experiment issuing 100M write operations of size 1K to produce a file of roughly 100GB size. The -w switch indicates that we are using 32 warmup operations.
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t write -s 1024 -k 100000000 -w 32 -f /tmp.dat
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t write -s 1024 -k 100000000 -w 32 -f /tmp.dat
</code></pre></div></div>
<div style="text-align: justify">
<p>
@@ -170,7 +170,7 @@ Crail offers direct I/O streams as well as buffered streams. For sequential oper
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.buffersize 1048576
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail.buffersize 1048576
crail.slicesize 524288
</code></pre></div></div>
@@ -180,8 +180,8 @@ The figure below illustrates the sequential write (top) and read (bottom) perfor
</p>
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-memory/write.svg" width="550" /></div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-memory/read.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-memory/write.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-memory/read.svg" width="550" /></div>
<p><br /><br /></p>
<div style="text-align: justify">
<p>
@@ -192,8 +192,8 @@ Note that both figures show single-client performance numbers. With Crail being
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-memory/crail-groupby.svg" width="550" /></div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-memory/spark-groupby.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-memory/crail-groupby.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-memory/spark-groupby.svg" width="550" /></div>
<h3 id="random-read-latency">Random Read Latency</h3>
@@ -202,14 +202,14 @@ Note that both figures show single-client performance numbers. With Crail being
Typically, distributed storage systems are either built for sequential access to large data sets (e.g., HDFS) or they are optimized for random access to small data sets (e.g., key/value stores). We have already shown that Crail performs well for large sequentially accessed data sets, let's now look at the latencies of small random read operations. For this, we mimic the behavior of a key/value store by storing key/value pairs in Crail files with the key being the filename. We then measur [...]
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t getkey -s 4 -k 1000000 -f /tmp.dat -w 32
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t getkey -s 4 -k 1000000 -f /tmp.dat -w 32
</code></pre></div></div>
<div style="text-align: justify">
<p>
The figure below illustrates the latencies of get() operations for different key/value sizes and compares them to the latencies we obtained with RAMCloud for the same type of operations (measured using RAMClouds C and Java APIs). RAMCloud is a low-latency key/value store implemented using RDMA. RAMCloud actually provides durable storage by asynchronously replicating data onto backup devices. However, at any point in time all the data is held in DRAM and read requests will be served from [...]
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-memory/latency.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-memory/latency.svg" width="550" /></div>
<div style="text-align: justify">
<p>
@@ -223,7 +223,7 @@ The latency advantages of Crail are beneficial also at the application level. Th
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-memory/cdf-broadcast-128-read.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-memory/cdf-broadcast-128-read.svg" width="550" /></div>
<div style="text-align: justify">
<p>
@@ -271,6 +271,6 @@ s.setAttribute('data-timestamp', +new Date());
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/08/crail-nvme-fabrics-v1.html b/content/blog/2017/08/crail-nvme-fabrics-v1.html
index 3381d1a..ad58a1c 100644
--- a/content/blog/2017/08/crail-nvme-fabrics-v1.html
+++ b/content/blog/2017/08/crail-nvme-fabrics-v1.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Crail Storage Performance -- Part II: NVMf</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/crail-nvme-fabrics-v1-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/crail-nvme-fabrics-v1-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/crail-nvme-fabrics-v1-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/crail-nvme-fabrics-v1-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -118,7 +118,7 @@ This is part II of our series of posts discussing Crail's raw storage performanc
Crail is a framework that allows arbitrary storage backends to be added by implementing the Crail storage interface. A storage backend manages the point-to-point data transfers on a per block granularity between a Crail client and a set of storage servers. The Crail storage interface essentially consists of three virtual functions, which simplified look like this:
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>//Server-side interface: donate storage resources to Crail
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>//Server-side interface: donate storage resources to Crail
StorageResource allocateResource();
//Client-side interface: read/write remote/local storage resources
writeBlock(BlockInfo, ByteBuffer);
@@ -149,10 +149,10 @@ One downside of the NVMe interface is that byte level access is prohibited. Inst
We perform latency and throughput measurement of our Crail NVMf storage tier against a native SPDK NVMf benchmark to determine how much overhead our implementation adds. The first plot shows random read latency on a single 512GB Samsung 960Pro accessed remotely through SPDK. For Crail we also show the time it takes to perform a metadata operations. You can run the Crail benchmark from the command line like this:
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t readRandom -b false -s <size> -k <iterations> -w 32 -f /tmp.dat
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t readRandom -b false -s <size> -k <iterations> -w 32 -f /tmp.dat
</code></pre></div></div>
<p>and SPDK:</p>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./perf -q 1 -s <size> -w randread -r 'trtype:RDMA adrfam:IPv4 traddr:<ip> trsvcid:<port>' -t <time in seconds>
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./perf -q 1 -s <size> -w randread -r 'trtype:RDMA adrfam:IPv4 traddr:<ip> trsvcid:<port>' -t <time in seconds>
</code></pre></div></div>
<div style="text-align: justify">
<p>
@@ -160,7 +160,7 @@ The main take away from this plot is that the time it takes to perform a random
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-nvmf/latency.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-nvmf/latency.svg" width="550" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -168,32 +168,32 @@ The main take away from this plot is that the time it takes to perform a random
The second plot shows sequential read and write throughput with a transfer size of 64KB and 128 outstanding operations. The Crail throughput benchmark can be run like this:
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t readAsync -s 65536 -k <iterations> -b 128 -w 32 -f /tmp.dat
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t readAsync -s 65536 -k <iterations> -b 128 -w 32 -f /tmp.dat
</code></pre></div></div>
<p>and SPDK:</p>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./perf -q 128 -s 65536 -w read -r 'trtype:RDMA adrfam:IPv4 traddr:<ip> trsvcid:<port>' -t <time in seconds>
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./perf -q 128 -s 65536 -w read -r 'trtype:RDMA adrfam:IPv4 traddr:<ip> trsvcid:<port>' -t <time in seconds>
</code></pre></div></div>
<div style="text-align: justify">
<p>
-For sequential operations in Crail, metadata fetching is inlined with data operations as described in the <a href="//crail.incubator.apache.org/blog/2017/08/crail-memory.html">DRAM</a> blog. This is possible as long as the data transfer has a lower latency than the metadata RPC, which is typically the case. As a consequence, our NVMf storage tier reaches the same throughput as the native SPDK benchmark (device limit).
+For sequential operations in Crail, metadata fetching is inlined with data operations as described in the <a href="https://crail.incubator.apache.org/blog/2017/08/crail-memory.html">DRAM</a> blog. This is possible as long as the data transfer has a lower latency than the metadata RPC, which is typically the case. As a consequence, our NVMf storage tier reaches the same throughput as the native SPDK benchmark (device limit).
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-nvmf/throughput.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-nvmf/throughput.svg" width="550" /></div>
<h3 id="sequential-throughput">Sequential Throughput</h3>
<div style="text-align: justify">
<p>
-Let us look at the sequential read and write throughput for buffered and direct streams and compare them to a buffered Crail stream on DRAM. All benchmarks are single thread/client performed against 8 storage nodes with 4 drives each, cf. configuration above. In this benchmark we use 32 outstanding operations for the NVMf storage tier buffered stream experiments by using a buffer size of 16MB and a slice size of 512KB, cf. <a href="//crail.incubator.apache.org/blog/2017/07/crail-memory.h [...]
+Let us look at the sequential read and write throughput for buffered and direct streams and compare them to a buffered Crail stream on DRAM. All benchmarks are single thread/client performed against 8 storage nodes with 4 drives each, cf. configuration above. In this benchmark we use 32 outstanding operations for the NVMf storage tier buffered stream experiments by using a buffer size of 16MB and a slice size of 512KB, cf. <a href="https://crail.incubator.apache.org/blog/2017/07/crail-me [...]
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t read -s <size> -k <iterations> -w 32 -f /tmp.dat
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t read -s <size> -k <iterations> -w 32 -f /tmp.dat
</code></pre></div></div>
<p>The direct stream benchmark:</p>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t readAsync -s <size> -k <iterations> -b 128 -w 32 -f /tmp.dat
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t readAsync -s <size> -k <iterations> -b 128 -w 32 -f /tmp.dat
</code></pre></div></div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-nvmf/throughput2.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-nvmf/throughput2.svg" width="550" /></div>
<h3 id="random-read-latency">Random Read Latency</h3>
@@ -203,16 +203,16 @@ Random read latency is limited by the flash technology and we currently see arou
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-nvmf/latency2.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-nvmf/latency2.svg" width="550" /></div>
<h3 id="tiering-dram---nvmf">Tiering DRAM - NVMf</h3>
<div style="text-align: justify">
<p>
-In this paragraph we show how Crail can leverage flash memory when there is not sufficient DRAM available in the cluster to hold all the data. As described in the <a href="//crail.incubator.apache.org/overview/">overview</a> section, if you have multiple storage tiers deployed in Crail, e.g. the DRAM tier and the NVMf tier, Crail by default first uses up all available resources of the faster tier. Basically a remote resource of a faster tier (e.g. remote DRAM) is preferred over a slower [...]
+In this paragraph we show how Crail can leverage flash memory when there is not sufficient DRAM available in the cluster to hold all the data. As described in the <a href="https://crail.incubator.apache.org/overview/">overview</a> section, if you have multiple storage tiers deployed in Crail, e.g. the DRAM tier and the NVMf tier, Crail by default first uses up all available resources of the faster tier. Basically a remote resource of a faster tier (e.g. remote DRAM) is preferred over a s [...]
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-nvmf/crail_tiering.png" width="500" vspace="10" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-nvmf/crail_tiering.png" width="500" vspace="10" /></div>
<p><br /></p>
<div style="text-align: justify">
<p>
@@ -220,7 +220,7 @@ In the following 200G Terasort experiment we gradually limit the DRAM resources
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-nvmf/tiering.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-nvmf/tiering.svg" width="550" /></div>
<p>To summarize, in this blog we have shown that the NVMf storage backend for Crail – due to its efficient user-level implementation – offers latencies and throughput very close to the hardware speed. The Crail NVMf storage tier can be used conveniently in combination with the Crail DRAM tier to either save cost or to handle situations where the available DRAM is not sufficient to store the working set of a data processing workload.</p>
@@ -264,6 +264,6 @@ s.setAttribute('data-timestamp', +new Date());
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/08/openpower.html b/content/blog/2017/08/openpower.html
index a5897d2..4d2a006 100644
--- a/content/blog/2017/08/openpower.html
+++ b/content/blog/2017/08/openpower.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Openpower</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/openpower-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/openpower-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/openpower-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/openpower-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/11/blog.html b/content/blog/2017/11/blog.html
index 66a31e4..f2785cc 100644
--- a/content/blog/2017/11/blog.html
+++ b/content/blog/2017/11/blog.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Blog</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/blog-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/blog-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/blog-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/blog-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -74,7 +74,7 @@
<h2>Blog</h2>
- <p>New blog <a href="//crail.incubator.apache.org/blog/2017/11/rdmashuffle.html">post</a> about SparkRDMA and Crail shuffle plugins</p>
+ <p>New blog <a href="https://crail.incubator.apache.org/blog/2017/11/rdmashuffle.html">post</a> about SparkRDMA and Crail shuffle plugins</p>
<br>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/11/crail-metadata.html b/content/blog/2017/11/crail-metadata.html
index 5df113a..3d042dc 100644
--- a/content/blog/2017/11/crail-metadata.html
+++ b/content/blog/2017/11/crail-metadata.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Crail Storage Performance -- Part III: Metadata</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/crail-metadata-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/crail-metadata-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/crail-metadata-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/crail-metadata-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -112,11 +112,11 @@ This is part III of our series of posts discussing Crail's raw storage performan
<div style="text-align: justify">
<p>
-As described in <a href="//crail.incubator.apache.org/blog/2017/08/crail-memory.html">part I</a>, Crail data operations are composed of actual data transfers and metadata operations. Examples of metadata operations are operations for creating or modifying the state of a file, or operations to lookup the storage server that stores a particular range (block) of a file. In Crail, all the metadata is managed by the namenode(s) (as opposed to the data which is managed by the storage nodes). C [...]
+As described in <a href="https://crail.incubator.apache.org/blog/2017/08/crail-memory.html">part I</a>, Crail data operations are composed of actual data transfers and metadata operations. Examples of metadata operations are operations for creating or modifying the state of a file, or operations to lookup the storage server that stores a particular range (block) of a file. In Crail, all the metadata is managed by the namenode(s) (as opposed to the data which is managed by the storage nod [...]
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-metadata/rpc.png" width="480" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-metadata/rpc.png" width="480" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -129,13 +129,13 @@ Crail supports partitioning of metadata across several namenods. Thereby, metada
<div style="text-align: justify">
<p>
-In two of the previous blogs (<a href="//crail.incubator.apache.org/blog/2017/08/crail-memory.html">DRAM</a>, <a href="//crail.incubator.apache.org/blog/2017/08/crail-nvme-fabrics-v1.html">NVMf</a>) we have already shown that Crail metadata operations are very low latency. Essentially a single metadata operation issued by a remote client takes 5-6 microseconds, which is only slightly more than the raw network latency of the RDMA network fabric. In this blog, we want to explore the scalab [...]
+In two of the previous blogs (<a href="https://crail.incubator.apache.org/blog/2017/08/crail-memory.html">DRAM</a>, <a href="https://crail.incubator.apache.org/blog/2017/08/crail-nvme-fabrics-v1.html">NVMf</a>) we have already shown that Crail metadata operations are very low latency. Essentially a single metadata operation issued by a remote client takes 5-6 microseconds, which is only slightly more than the raw network latency of the RDMA network fabric. In this blog, we want to explor [...]
</p>
<p>
-An important metadata operation is ''getFile()'', which is used by clients to lookup the status of a file (whether the file exists, what size it has, etc.). The ''getFile()'' operation is served by Crail's fast lock-free map and in spirit is very similar to the ''getBlock()'' metadata operation (used by clients to query which storage nodes holds a particular block). In a typical Crail use case, ''getFile()'' and ''getBlock()'' are responsible for the peak metadata load at a namenode. In [...]
+An important metadata operation is ''getFile()'', which is used by clients to lookup the status of a file (whether the file exists, what size it has, etc.). The ''getFile()'' operation is served by Crail's fast lock-free map and in spirit is very similar to the ''getBlock()'' metadata operation (used by clients to query which storage nodes holds a particular block). In a typical Crail use case, ''getFile()'' and ''getBlock()'' are responsible for the peak metadata load at a namenode. In [...]
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t getMultiFileAsync -f / -k 10000000 -b 128
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail iobench -t getMultiFileAsync -f / -k 10000000 -b 128
</code></pre></div></div>
<div style="text-align: justify">
<p>
@@ -157,7 +157,7 @@ In the first experiment, we measure the aggregated number of metadata operations
As comparison, we measure the raw number of IOPS, which can be executed on the RDMA network. We measure the raw number using ib_send_bw. We configured ib_send_bw with the same parameters in terms of RDMA configuration as the namenode. This means, we instructed ib_send_bw not to do CQ moderation, and to use a receive queue and a send queue of length 32, which equals the length of the namenode queues. Note that the default configuration of ib_send_bw uses CQ moderation and does preposting [...]
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ib_send_bw -s 1 -Q 1 -r 32 -t 32 -n 10000000
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ib_send_bw -s 1 -Q 1 -r 32 -t 32 -n 10000000
</code></pre></div></div>
<div style="text-align: justify">
<p>
@@ -175,7 +175,7 @@ The line of the raw number of IOPS, labeled ''ib send'' is shown in the same gra
</p>
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-metadata/namenode_ibsend_iops64.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-metadata/namenode_ibsend_iops64.svg" width="550" /></div>
<p><br /></p>
<div style="text-align: justify">
<p>
@@ -192,7 +192,7 @@ To increase the number of IOPS the overall system can handle, we allow starting
</p>
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-metadata/namenode_multi64.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-metadata/namenode_multi64.svg" width="550" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -220,7 +220,7 @@ namenodes happening, which should lead to linear scalability.
<div style="text-align: justify">
<p>
Let us look at a concrete application, which ideally runs on a large cluster:
-TeraSort. In a previous blog, <a href="//crail.incubator.apache.org/blog/2017/01/sorting.html">sorting</a>,
+TeraSort. In a previous blog, <a href="https://crail.incubator.apache.org/blog/2017/01/sorting.html">sorting</a>,
we analyze performance characteristics of TeraSort on Crail on a big cluster
of 128 nodes, where we run 384 executors in total. This already proves that
Crail can at least handle 384 clients. Now we analyze the theoretical number
@@ -254,7 +254,7 @@ namenode over the elapsed runtime of the TeraSort application.
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-metadata/terasort_iops.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-metadata/terasort_iops.svg" width="550" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -418,7 +418,7 @@ plot shows the number of IOPS relative to the number of clients.
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-metadata/namenode_hdfs_iops.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-metadata/namenode_hdfs_iops.svg" width="550" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -472,7 +472,7 @@ the blog with the latest numbers as soon as the bug is fixed.
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-metadata/ramcloud_iops.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-metadata/ramcloud_iops.svg" width="550" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -503,7 +503,7 @@ of 30Mio/s with 4 namenodes).
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/crail-metadata/max_iops_crail_hdfs_ramcloud.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/crail-metadata/max_iops_crail_hdfs_ramcloud.svg" width="550" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -565,6 +565,6 @@ s.setAttribute('data-timestamp', +new Date());
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/11/floss.html b/content/blog/2017/11/floss.html
index 7bdd6cc..3128eb0 100644
--- a/content/blog/2017/11/floss.html
+++ b/content/blog/2017/11/floss.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Floss</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/floss-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/floss-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/floss-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/floss-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/11/iops.html b/content/blog/2017/11/iops.html
index 5b91957..b83ce7e 100644
--- a/content/blog/2017/11/iops.html
+++ b/content/blog/2017/11/iops.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Iops</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/iops-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/iops-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/iops-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/iops-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -74,7 +74,7 @@
<h2>Iops</h2>
- <p>New blog <a href="//crail.incubator.apache.org/blog/2017/11/crail-metadata.html">post</a> about Crail’s metadata performance and scalability</p>
+ <p>New blog <a href="https://crail.incubator.apache.org/blog/2017/11/crail-metadata.html">post</a> about Crail’s metadata performance and scalability</p>
<br>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2017/11/rdmashuffle.html b/content/blog/2017/11/rdmashuffle.html
index 816139f..c5e43ef 100644
--- a/content/blog/2017/11/rdmashuffle.html
+++ b/content/blog/2017/11/rdmashuffle.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Spark Shuffle: SparkRDMA vs Crail</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/rdmashuffle-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/rdmashuffle-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/rdmashuffle-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/rdmashuffle-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -125,7 +125,7 @@ In contrast, the Crail shuffler plugin takes a more holistic approach and levera
<p>Lets start by quantitatively assessing performance gains from the Crail shuffle plugin and SparkRDMA. As described above, SparkRDMA can be operated in two different modes. Users decide which mode to use by selecting a particular type of shuffle writer (spark.shuffle.rdma.shuffleWriterMethod). The Wrapper shuffle writer writes shuffle data to files between the stages, the Chunked shuffle writer stores shuffle data in memory. We evaluate both writer methods for terasort and SQL equijoin.
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/rdma-shuffle/terasort.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/rdma-shuffle/terasort.svg" width="550" /></div>
<p><br /></p>
<div style="text-align: justify">
<p>
@@ -135,7 +135,7 @@ First we run <a href="https://github.com/zrlio/crail-spark-terasort">terasort</a
The plot above shows runtimes of the various configuration we run with terasort. SparkRDMA with the Wrapper shuffle writer performance slightly better (3-4%) than vanilla Spark whereas the Chunked shuffle writer shows a 30% overhead. On a quick inspection we found that this overhead stems from memory allocation and registration for the shuffle data that is kept in memory between the stages. Compared to vanilla Spark, Crail's shuffle plugin shows performance improvement of around 235%.
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/rdma-shuffle/sql.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/rdma-shuffle/sql.svg" width="550" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -176,6 +176,6 @@ These benchmarks validate our belief that a "last-mile" integration cannot deliv
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2018/01/apache.html b/content/blog/2018/01/apache.html
index e3abaf2..1b18b50 100644
--- a/content/blog/2018/01/apache.html
+++ b/content/blog/2018/01/apache.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Apache</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/apache-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/apache-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/apache-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/apache-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2018/06/apache-release.html b/content/blog/2018/06/apache-release.html
index 23f40a2..f949085 100644
--- a/content/blog/2018/06/apache-release.html
+++ b/content/blog/2018/06/apache-release.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Apache Release</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/apache-release-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/apache-release-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/apache-release-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/apache-release-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -74,7 +74,7 @@
<h2>Apache Release</h2>
- <p>Apache Crail 1.0 incubator <a href="//crail.incubator.apache.org/download">release</a></p>
+ <p>Apache Crail 1.0 incubator <a href="https://crail.incubator.apache.org/download">release</a></p>
<br>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2018/06/dataworks.html b/content/blog/2018/06/dataworks.html
index c278119..908e930 100644
--- a/content/blog/2018/06/dataworks.html
+++ b/content/blog/2018/06/dataworks.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Dataworks</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/dataworks-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/dataworks-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/dataworks-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/dataworks-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2018/06/sparksummit.html b/content/blog/2018/06/sparksummit.html
index d6afa64..21168d1 100644
--- a/content/blog/2018/06/sparksummit.html
+++ b/content/blog/2018/06/sparksummit.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Sparksummit</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/sparksummit-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/sparksummit-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/sparksummit-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/sparksummit-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2018/08/sql-p1-news.html b/content/blog/2018/08/sql-p1-news.html
index a3b81cc..cfbae86 100644
--- a/content/blog/2018/08/sql-p1-news.html
+++ b/content/blog/2018/08/sql-p1-news.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Sql P1 News</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/sql-p1-news-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/sql-p1-news-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/sql-p1-news-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/sql-p1-news-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -74,7 +74,7 @@
<h2>Sql P1 News</h2>
- <p>A new blog <a href="//crail.incubator.apache.org/blog/2018/08/sql-p1.html">post</a> discussing file formats performance is now online</p>
+ <p>A new blog <a href="https://crail.incubator.apache.org/blog/2018/08/sql-p1.html">post</a> discussing file formats performance is now online</p>
<br>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2018/08/sql-p1.html b/content/blog/2018/08/sql-p1.html
index 40cea88..dd0069e 100644
--- a/content/blog/2018/08/sql-p1.html
+++ b/content/blog/2018/08/sql-p1.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: SQL Performance: Part 1 - Input File Formats</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/sql-p1-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/sql-p1-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/sql-p1-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/sql-p1-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -114,11 +114,11 @@ This is the first user blog post in a multi-part series where we will focus on r
<p>In a typical cloud-based relational data processing setup, the input data is stored on an external data storage solution like HDFS or AWS S3. Data tables and their associated schema are converted into a storage-friendly format for optimal performance. Examples of some popular and familiar file formats are <a href="https://parquet.apache.org/">Apache Parquet</a>, <a href="https://orc.apache.org/">Apache ORC</a>, <a href="https://avro.apache.org/">Apache Avro</a>, <a href="https://en.wi [...]
-<figure><div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sql-p1/outline.svg" width="550" /><figcaption>Figure 1: The benchmarking setup with HDFS and file formats on a 100 Gbps network with NVMe flash devices. All formats contains routines for compression, encoding, and value materialization with associated I/O buffer management and data copies routines.<p></p></figcaption></div></figure>
+<figure><div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sql-p1/outline.svg" width="550" /><figcaption>Figure 1: The benchmarking setup with HDFS and file formats on a 100 Gbps network with NVMe flash devices. All formats contains routines for compression, encoding, and value materialization with associated I/O buffer management and data copies routines.<p></p></figcaption></div></figure>
<p>To benchmark the performance of file formats, we wrote a set of micro-benchmarks which are available at <a href="https://github.com/zrlio/fileformat-benchmarks">https://github.com/zrlio/fileformat-benchmarks</a>. We cannot use typical SQL micro-benchmarks because every SQL engine has its own favorite file format, on which it performs the best. Hence, in order to ensure parity, we decoupled the performance of reading the input file format from the SQL query processing by writing simple [...]
-<figure><div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sql-p1/performance-all.svg" width="550" /><figcaption>Figure 2: Performance of JSON, Avro, Parquet, ORC, and Arrow on NVMe devices over a 100 Gbps network.<p></p></figcaption></div></figure>
+<figure><div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sql-p1/performance-all.svg" width="550" /><figcaption>Figure 2: Performance of JSON, Avro, Parquet, ORC, and Arrow on NVMe devices over a 100 Gbps network.<p></p></figcaption></div></figure>
<p>We evaluate the performance of the benchmark on a 3 node HDFS cluster connected using 100 Gbps RoCE. One datanode in HDFS contains 4 NVMe devices with a collective aggregate bandwidth of 12.5 GB/sec (equals to 100 Gbps, hence, we have a balanced network and storage performance). Figure 2 shows our results where none of the file formats is able to deliver the full hardware performance for reading input files. One third of the performance is already lost in HDFS (maximum throughput 74.9 [...]
@@ -132,7 +132,7 @@ This is the first user blog post in a multi-part series where we will focus on r
<li>Careful object materialization using a binary API: To optimize the runtime representation in managed runtimes like the JVM, only objects which are necessary for SQL processing are materialized. Otherwise, a 4 byte integer can be passed around as a byte array (using the binary API of Albis).</li>
</ul>
-<figure><div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sql-p1/core-scalability.svg" width="550" /><figcaption>Figure 3: Core scalability of JSON, Avro, Parquet, ORC, Arrow, and Albis on HDFS/NVMe.<p></p></figcaption></div></figure>
+<figure><div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sql-p1/core-scalability.svg" width="550" /><figcaption>Figure 3: Core scalability of JSON, Avro, Parquet, ORC, Arrow, and Albis on HDFS/NVMe.<p></p></figcaption></div></figure>
<p>Using the Albis format, we revise our previous experiment where we read the input store_sales table from HDFS. In the figure above, we show the performance of Albis and other file formats with number of CPU cores involved. At the right hand of the x-axis, we have performance with all 16 cores engaged, hence, representing the peak possible performance. As evident, Albis delivers 59.9 Gbps out of 74.9 Gbps possible bandwidth with HDFS over NVMe. Albis performance is 1.9 - 21.4x better t [...]
@@ -171,9 +171,9 @@ This is the first user blog post in a multi-part series where we will focus on r
<h3 id="apache-crail-incubating-with-albis">Apache Crail (Incubating) with Albis</h3>
-<p>For our final experiment, we try to answer the question what it would take to deliver the full 100 Gbps bandwidth for Albis. Certainly, the first bottleneck is to improve the base storage layer performance. Here we use Apache Crail (Incubating) with its <a href="https://en.wikipedia.org/wiki/NVM_Express#NVMeOF">NVMeF</a> storage tier. This tier uses <a href="https://github.com/zrlio/jNVMf">jNVMf library</a> to implement NVMeF stack in Java. As we have shown in a previous blog <a href= [...]
+<p>For our final experiment, we try to answer the question what it would take to deliver the full 100 Gbps bandwidth for Albis. Certainly, the first bottleneck is to improve the base storage layer performance. Here we use Apache Crail (Incubating) with its <a href="https://en.wikipedia.org/wiki/NVM_Express#NVMeOF">NVMeF</a> storage tier. This tier uses <a href="https://github.com/zrlio/jNVMf">jNVMf library</a> to implement NVMeF stack in Java. As we have shown in a previous blog <a href= [...]
-<figure><div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sql-p1/albis-crail.svg" width="550" /><figcaption>Figure 4: Performance of Albis on Crail.<p></p></figcaption></div></figure>
+<figure><div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sql-p1/albis-crail.svg" width="550" /><figcaption>Figure 4: Performance of Albis on Crail.<p></p></figcaption></div></figure>
<p>The left half of the figure shows the performance scalability of Albis on Crail in a setup with 1 core (8.9 Gbps) to 16 cores (85.5 Gbps). In comparison, the right half of the figure shows the performance of Crail on HDFS/NVMe at 59.9 Gbps, and on Crail/NVMe at 85.5 Gbps. The last bar shows the performance of Albis if the benchmark does not materialize Java object values. In this configuration, Albis on Crail delivers 91.3 Gbps, which is very close to the expected peak of 91.8 Gbps.</p>
@@ -227,6 +227,6 @@ s.setAttribute('data-timestamp', +new Date());
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2019/01/python.html b/content/blog/2019/01/python.html
index 7bb85de..23c12c1 100644
--- a/content/blog/2019/01/python.html
+++ b/content/blog/2019/01/python.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Crail Python API: Python -> C/C++ call overhead</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/python-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/python-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/python-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/python-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -153,14 +153,14 @@ and does not interface with the CPython C-API directly.</li>
<p>In this blog post we focus on the overhead of calling a C/C++ function from Python.
We vary the number of arguments, argument types and the return types. We also
test passing strings to C/C++ since it is part of the Crail API e.g. when
-opening or creating a file. Some frameworks expect <code class="highlighter-rouge">bytes</code> when passing a string
-to a underlying <code class="highlighter-rouge">const char *</code>, some allow to pass a <code class="highlighter-rouge">str</code> and others allow both.
-If C++ is supported by the framework we also test passing a <code class="highlighter-rouge">std::string</code> to a
+opening or creating a file. Some frameworks expect <code class="language-plaintext highlighter-rouge">bytes</code> when passing a string
+to a underlying <code class="language-plaintext highlighter-rouge">const char *</code>, some allow to pass a <code class="language-plaintext highlighter-rouge">str</code> and others allow both.
+If C++ is supported by the framework we also test passing a <code class="language-plaintext highlighter-rouge">std::string</code> to a
C++ function. Note that we perform all benchmarks with CPython version 3.5.2.
We measure the time it takes to call the Python function until it returns.
-The C/C++ functions are empty, except a <code class="highlighter-rouge">return</code> statement where necessary.</p>
+The C/C++ functions are empty, except a <code class="language-plaintext highlighter-rouge">return</code> statement where necessary.</p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/python_c/python_c_foo.svg" width="725" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/python_c/python_c_foo.svg" width="725" /></div>
<p></p>
<p>The plot shows that adding more arguments to a function increases runtime.
@@ -181,26 +181,26 @@ between Cython and our extension module implementation we found that Cython make
better use of the C-API.</p>
<p>Our extension module implementation follows the official tutorial and uses
-<code class="highlighter-rouge">PyArg_ParseTuple</code> to parse the arguments. However as shown below we found that
-manually unpacking the arguments with <code class="highlighter-rouge">PyArg_UnpackTuple</code> already significantly
+<code class="language-plaintext highlighter-rouge">PyArg_ParseTuple</code> to parse the arguments. However as shown below we found that
+manually unpacking the arguments with <code class="language-plaintext highlighter-rouge">PyArg_UnpackTuple</code> already significantly
increased the performance. Although these numbers still do not match Cython’s
performance we did not further investigate possible optimizations
to our code.</p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/python_c/python_c_foo_opt.svg" width="725" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/python_c/python_c_foo_opt.svg" width="725" /></div>
<p></p>
-<p>Let’s take a look at the string performance. <code class="highlighter-rouge">bytes</code> and <code class="highlighter-rouge">str</code> is used whereever
+<p>Let’s take a look at the string performance. <code class="language-plaintext highlighter-rouge">bytes</code> and <code class="language-plaintext highlighter-rouge">str</code> is used whereever
applicable. To pass strings as bytes the ‘b’ prefix is used.</p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/python_c/python_c_foo_str.svg" width="725" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/python_c/python_c_foo_str.svg" width="725" /></div>
<p></p>
<p>Again Cython and the extension module implementation with manual unpacking seem to
-deliver the best performance. Passing a 64bit value in form of a <code class="highlighter-rouge">const char *</code>
+deliver the best performance. Passing a 64bit value in form of a <code class="language-plaintext highlighter-rouge">const char *</code>
pointer seems to be slightly faster than passing an integer argument (up to 20%).
-Passing the string to a C++ function which takes a <code class="highlighter-rouge">std::string</code>
-is ~50% slower than passing a <code class="highlighter-rouge">const char *</code>, probably because of the
+Passing the string to a C++ function which takes a <code class="language-plaintext highlighter-rouge">std::string</code>
+is ~50% slower than passing a <code class="language-plaintext highlighter-rouge">const char *</code>, probably because of the
instantiation of the underlying data buffer and copying however we have not
confirmed this.</p>
@@ -290,6 +290,6 @@ s.setAttribute('data-timestamp', +new Date());
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2019/03/deployment.html b/content/blog/2019/03/deployment.html
index eb64c58..7a97065 100644
--- a/content/blog/2019/03/deployment.html
+++ b/content/blog/2019/03/deployment.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Deployment Options for Tiered Storage Disaggregation</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/deployment-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/deployment-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/deployment-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/deployment-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -98,7 +98,7 @@ Remember that Crail is a tiered storage system where each storage tier consists
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/deployment/three_options.svg" width="580" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/deployment/three_options.svg" width="580" /></div>
<p><br />
<br /></p>
@@ -123,7 +123,7 @@ In all of the previously discussed configurations there is a one-to-one mapping
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/deployment/storage_class.svg" width="400" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/deployment/storage_class.svg" width="400" /></div>
<p><br />
<br /></p>
@@ -133,7 +133,7 @@ Storage classes can easily be defined in the slaves file as follows (see the <a
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ cat $CRAIL_HOME/conf/slaves
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ cat $CRAIL_HOME/conf/slaves
clusternode1 -t org.apache.crail.storage.rdma.RdmaStorageTier -c 0
clusternode2 -t org.apache.crail.storage.rdma.RdmaStorageTier -c 0
clusternode1 -t org.apache.crail.storage.nvmf.NvmfStorageTier -c 1
@@ -147,7 +147,7 @@ One can also manually attach a storage server to a particular storage class:
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clusternode2:~$ $CRAIL_HOME/bin/crail datanode -t org.apache.crail.storage.nvmf.NvmfStorageTier -c 2
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clusternode2:~$ $CRAIL_HOME/bin/crail datanode -t org.apache.crail.storage.nvmf.NvmfStorageTier -c 2
</code></pre></div></div>
<div style="text-align: justify">
@@ -157,7 +157,7 @@ Remember that the storage class ID is implicitly ordering the storage tiers. Dur
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/deployment/timeline.svg" width="300" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/deployment/timeline.svg" width="300" /></div>
<p><br />
<br /></p>
@@ -167,7 +167,7 @@ If applications want to further prioritize the specific local resource of a mach
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailLocationClass local = fs.getLocationClass();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailLocationClass local = fs.getLocationClass();
CrailFile file = fs.create("/tmp.dat", CrailNodeType.DATAFILE, CrailStorageClass.DEFAULT, local).get().asFile();
</code></pre></div></div>
<div style="text-align: justify">
@@ -185,7 +185,7 @@ During the deployment of Crail, one has to decide on the storage capacity of eac
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/deployment/resource_provisioning.svg" width="400" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/deployment/resource_provisioning.svg" width="400" /></div>
<p><br />
<br /></p>
@@ -243,6 +243,6 @@ s.setAttribute('data-timestamp', +new Date());
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2019/03/disaggregation.html b/content/blog/2019/03/disaggregation.html
index 0a6b7a7..b9e1a25 100644
--- a/content/blog/2019/03/disaggregation.html
+++ b/content/blog/2019/03/disaggregation.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Shuffle Disaggregation using RDMA accessible remote DRAM and NVMe Flash</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/disaggregation-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/disaggregation-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/disaggregation-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/disaggregation-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -86,7 +86,7 @@ Today, data processing applications running in the cloud may implicitly use disa
</p>
<br />
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/spark_disagg.svg" width="580" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/spark_disagg.svg" width="580" /></div>
<br />
<br />
@@ -108,7 +108,7 @@ In a traditional shuffle operation, data is exchanged between map and reduce tas
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/overview.svg" width="350" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/overview.svg" width="350" /></div>
<p><br /></p>
<h3 id="challenge-large-number-of-small-files">Challenge: Large Number of Small Files</h3>
@@ -125,7 +125,7 @@ performed a simple experiment where we measured the size distribution of Spark s
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/cdf-plot.svg" width="480" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/cdf-plot.svg" width="480" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -143,7 +143,7 @@ To mitigate the overheads of writing and reading large numbers of small data set
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/optimization_tasks.svg" width="380" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/optimization_tasks.svg" width="380" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -154,7 +154,7 @@ The second optimization we use in our disaggregated Crail shuffler is efficient
Crail Multifiles offer zero-copy parallel reading of large numbers of files in a sequential manner. From the prespective of a map task MultiFiles are flat directories consisting of files belonging to different per-core file sets. From a reducer perspective, a MultFile looks like a large file that can be read sequentially using many in-flight operations. For instance, the following code shows how a reduce task during a Crail shuffle operation reads a partition from remote storage.
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore fs = CrailStore.newInstance();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore fs = CrailStore.newInstance();
CrailMultiFile multiFile = fs.lookup("/shuffle/partition1").get().asMultiFile();
ByteBuffer buffer = Buffer.allocateDirect(multiFile.size());
int batchSize = 16;
@@ -182,7 +182,7 @@ Shuffle operations, being essentially barriers between compute stages, are highl
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/machine_skew.svg" width="520" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/machine_skew.svg" width="520" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -192,7 +192,7 @@ One way to deal with this problem is through weighted fair scheduling of network
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/machine_skew_crail.svg" width="520" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/machine_skew_crail.svg" width="520" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -208,7 +208,7 @@ One may argue that chopping shuffle data up into blocks and transferring them ov
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/loadbalancing.svg" width="520" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/loadbalancing.svg" width="520" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -226,7 +226,7 @@ Note that Crail disaggregated storage may be provided by a few highly dense stor
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/terasort.svg" width="450" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/terasort.svg" width="450" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -244,7 +244,7 @@ Next we look at Spark SQL performance in a disaggregated configuration. Again we
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/sql.svg" width="420" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/sql.svg" width="420" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -300,6 +300,6 @@ s.setAttribute('data-timestamp', +new Date());
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2019/04/strata.html b/content/blog/2019/04/strata.html
index ba59faf..2e6495c 100644
--- a/content/blog/2019/04/strata.html
+++ b/content/blog/2019/04/strata.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Strata</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/strata-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/strata-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/strata-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/strata-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2019/06/ycsb.html b/content/blog/2019/06/ycsb.html
index df90937..a8c9467 100644
--- a/content/blog/2019/06/ycsb.html
+++ b/content/blog/2019/06/ycsb.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Ycsb</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/ycsb-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/ycsb-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/ycsb-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/ycsb-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2019/08/atc.html b/content/blog/2019/08/atc.html
index dc76104..dde3422 100644
--- a/content/blog/2019/08/atc.html
+++ b/content/blog/2019/08/atc.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Atc</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/atc-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/atc-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/atc-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/atc-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2019/10/ycsb.html b/content/blog/2019/10/ycsb.html
index 1f8efdc..d22d877 100644
--- a/content/blog/2019/10/ycsb.html
+++ b/content/blog/2019/10/ycsb.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: YCSB Benchmark with Crail on DRAM, Flash and Optane over RDMA and NVMe-over-Fabrics</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/ycsb-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/ycsb-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/ycsb-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/ycsb-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -92,7 +92,7 @@ Remember that Crail exports a hierarchical storage namespace where individual no
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/storage_namespace.svg" width="400" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/storage_namespace.svg" width="400" /></div>
<p><br />
<br /></p>
@@ -102,7 +102,7 @@ In this blog we focus on Crail's KeyValue API available to users through the Tab
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore crail = CrailStore.newInstance();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore crail = CrailStore.newInstance();
CrailTable table = fs.create("/tab1", CrailNodeType.TABLE, ..., ...).get();
CrailKeyValue kv = fs.create("/tab1/key1", CrailNodeType.KEYVALUE, ..., ...).get();
</code></pre></div></div>
@@ -113,7 +113,7 @@ Here the table's name is "/tab1" and the key of the key-value pair is "key1". Un
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailOutputStream stream = kv.getOutputStream();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailOutputStream stream = kv.getOutputStream();
CrailBuffer buf = CrailBuffer.wrap("data".bytes());
stream.append(buf);
</code></pre></div></div>
@@ -123,7 +123,7 @@ Lookup and reading of a key-value pair is done in a similar fashion.
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore crail = CrailStore.newInstance();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore crail = CrailStore.newInstance();
CrailKeyValue kv = fs.lookup("/tab1/key1").get().asKeyValue();
CrailInputStream stream = kv.getInputStream();
CrailBuffer buf = crail.createBuffer();
@@ -150,7 +150,7 @@ In order to run the benchmark simply clone the YCSB repository and build the Cra
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ git clone http://github.com/brianfrankcooper/YCSB.git
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ git clone http://github.com/brianfrankcooper/YCSB.git
crail@clustermaster:~$ cd YCSB
crail@clustermaster:~$ mvn -pl com.yahoo.ycsb:crail-binding -am clean package
</code></pre></div></div>
@@ -161,7 +161,7 @@ You need to have a Crail deployment up and running to run the YCSB benchmark. Fo
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ ./bin/ycsb load crail -s -P workloads/workloada -P large.dat -p crail.enumeratekeys=true >outputLoad.txt
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ ./bin/ycsb load crail -s -P workloads/workloada -P large.dat -p crail.enumeratekeys=true >outputLoad.txt
</code></pre></div></div>
<div style="text-align: justify">
@@ -176,7 +176,7 @@ So far we have just loaded the data. Let's now run the actual benchmark which co
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ ./bin/ycsb run crail -s -P workloads/workloada
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ ./bin/ycsb run crail -s -P workloads/workloada
</code></pre></div></div>
<h3 id="ycsb-benchmark-performance-for-dram--intel-optane">YCSB Benchmark Performance for DRAM & Intel Optane</h3>
@@ -191,7 +191,7 @@ Looking at small KV pairs first (left figure below), we can see that Crail's DRA
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/ycsb_put.svg" width="700" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/ycsb_put.svg" width="700" /></div>
<p><br />
<br /></p>
@@ -202,7 +202,7 @@ The reason Crail trails RAMCloud for small KV pairs is because Crail's internal
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/crail_put_anatomy.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/crail_put_anatomy.svg" width="550" /></div>
<p><br />
<br /></p>
@@ -219,7 +219,7 @@ Besides update latency, we are also showing read latency in the CDF figure below
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/ycsb_get.svg" width="700" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/ycsb_get.svg" width="700" /></div>
<p><br />
<br /></p>
@@ -235,7 +235,7 @@ We used Crail's asynchronous API to benchmark Crail's key-value performance unde
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/iops_qd1.svg" width="650" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/iops_qd1.svg" width="650" /></div>
<p><br />
<br /></p>
@@ -246,7 +246,7 @@ In order to generate a higher load, we measured throughput and latencies for the
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/iops_qd4.svg" width="650" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/iops_qd4.svg" width="650" /></div>
<p><br />
<br /></p>
@@ -298,6 +298,6 @@ s.setAttribute('data-timestamp', +new Date());
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/2020/01/release_v1.2.html b/content/blog/2020/01/release_v1.2.html
index 0752c75..3361c46 100644
--- a/content/blog/2020/01/release_v1.2.html
+++ b/content/blog/2020/01/release_v1.2.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Release_v1.2</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/release_v1.2-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/release_v1.2-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/release_v1.2-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/release_v1.2-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -88,6 +88,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/index.html b/content/blog/index.html
index a3d3354..49a1364 100644
--- a/content/blog/index.html
+++ b/content/blog/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Latest posts</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/blog-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/blog-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/blog-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/blog-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li class="active">
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -231,6 +231,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/page2/index.html b/content/blog/page2/index.html
index c46dce5..8156ac6 100644
--- a/content/blog/page2/index.html
+++ b/content/blog/page2/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Latest posts</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/page2-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/page2-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/page2-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/page2-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -235,6 +235,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/page3/index.html b/content/blog/page3/index.html
index 68b4ecb..ea29a05 100644
--- a/content/blog/page3/index.html
+++ b/content/blog/page3/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Latest posts</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/page3-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/page3-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/page3-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/page3-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -235,6 +235,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/page4/index.html b/content/blog/page4/index.html
index 3ccf960..d43d0c8 100644
--- a/content/blog/page4/index.html
+++ b/content/blog/page4/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Latest posts</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/page4-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/page4-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/page4-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/page4-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -235,6 +235,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/page5/index.html b/content/blog/page5/index.html
index 5d3b7b4..b98eb05 100644
--- a/content/blog/page5/index.html
+++ b/content/blog/page5/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Latest posts</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/page5-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/page5-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/page5-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/page5-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -235,6 +235,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/page6/index.html b/content/blog/page6/index.html
index d9b2df3..38e44c3 100644
--- a/content/blog/page6/index.html
+++ b/content/blog/page6/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Latest posts</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/page6-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/page6-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/page6-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/page6-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -235,6 +235,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/blog/page7/index.html b/content/blog/page7/index.html
index 49afe95..5fa7388 100644
--- a/content/blog/page7/index.html
+++ b/content/blog/page7/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Latest posts</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/page7-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/page7-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/page7-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/page7-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -231,6 +231,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/community/index.html b/content/community/index.html
index 423593f..81816a0 100644
--- a/content/community/index.html
+++ b/content/community/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Community</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/community-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/community-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/community-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/community-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li class="active">
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -112,6 +112,6 @@ For any potential changes/proposals we recommend that you open a JIRA ticket to
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/documentation/index.html b/content/documentation/index.html
index 4f455c5..71767cf 100644
--- a/content/documentation/index.html
+++ b/content/documentation/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Documentation</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/documentation-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/documentation-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/documentation-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/documentation-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li class="active">
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -80,7 +80,7 @@
<ul>
<li><a href="https://www.usenix.org/conference/atc19/presentation/stuedi">Unification of Temporary Storage in the NodeKernel Architecture</a>, USENIX ATC’19</li>
<li><a href="https://conferences.oreilly.com/strata/strata-ca/public/schedule/detail/71902">Data processing at the Speed of 100 Gbps using Apache Crail</a>, Oreilly Strata, 2019.</li>
- <li><a href="//crail.incubator.apache.org/files/crail-project-web.pdf">Data Processing at the Speed of 100 Gbps using Apache Crail (Incubating)</a>, ETH Zurich, September, 2018.</li>
+ <li><a href="https://crail.incubator.apache.org/files/crail-project-web.pdf">Data Processing at the Speed of 100 Gbps using Apache Crail (Incubating)</a>, ETH Zurich, September, 2018.</li>
<li><a href="https://databricks.com/session/serverless-machine-learning-on-modern-hardware-using-apache-spark">Serverless Machine Learning on Modern Hardware Using Apache Spark</a>, Spark AI Summit, 2018.</li>
<li><a href="https://dataworkssummit.com/san-jose-2018/session/data-processing-at-the-speed-of-100-gbpsapache-crail-incubating/">Data processing at the speed of 100 Gbps@Apache Crail (Incubating)</a>, DataWorks Summit, 2018.</li>
<li><a href="https://databricks.com/session/running-apache-spark-on-a-high-performance-cluster-using-rdma-and-nvme-flash">Running Apache Spark on a High-Performance Cluster Using RDMA and NVMe Flash</a>, Spark Summit 2017.</li>
@@ -106,6 +106,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/download/index.html b/content/download/index.html
index a557c47..ff42d9c 100644
--- a/content/download/index.html
+++ b/content/download/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Downloads</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/download-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/download-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/download-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/download-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li class="active">
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -82,8 +82,8 @@
<tbody>
<tr>
<td><a href="https://www.apache.org/dyn/closer.lua/incubator/crail/1.2-incubating/apache-crail-1.2-incubating-src.tar.gz">tar.gz</a></td>
- <td><a href="https://www.apache.org/dist/incubator/crail/1.2-incubating/apache-crail-1.2-incubating-src.tar.gz.sha512">sha512</a></td>
- <td><a href="https://www.apache.org/dist/incubator/crail/1.2-incubating/apache-crail-1.2-incubating-src.tar.gz.asc">signature</a></td>
+ <td><a href="https://downloads.apache.org/incubator/crail/1.2-incubating/apache-crail-1.2-incubating-src.tar.gz.sha512">sha512</a></td>
+ <td><a href="https://downloads.apache.org/incubator/crail/1.2-incubating/apache-crail-1.2-incubating-src.tar.gz.asc">signature</a></td>
</tr>
</tbody>
</table>
@@ -96,8 +96,8 @@
<tbody>
<tr>
<td><a href="https://www.apache.org/dyn/closer.lua/incubator/crail/1.2-incubating/apache-crail-1.2-incubating-bin.tar.gz">tar.gz</a></td>
- <td><a href="https://www.apache.org/dist/incubator/crail/1.2-incubating/apache-crail-1.2-incubating-bin.tar.gz.sha512">sha512</a></td>
- <td><a href="https://www.apache.org/dist/incubator/crail/1.2-incubating/apache-crail-1.2-incubating-bin.tar.gz.asc">signature</a></td>
+ <td><a href="https://downloads.apache.org/incubator/crail/1.2-incubating/apache-crail-1.2-incubating-bin.tar.gz.sha512">sha512</a></td>
+ <td><a href="https://downloads.apache.org/incubator/crail/1.2-incubating/apache-crail-1.2-incubating-bin.tar.gz.asc">signature</a></td>
</tr>
</tbody>
</table>
@@ -112,7 +112,7 @@
<h3 id="verifying-a-release">Verifying a Release</h3>
-<p>You can find instructions for checking hashes and signatures on the <a href="http://www.apache.org/info/verification.html">Verifying Apache Software Foundation Releases</a> page. The committer’s pgp signatures can be found <a href="https://www.apache.org/dist/incubator/crail/KEYS">here</a>.</p>
+<p>You can find instructions for checking hashes and signatures on the <a href="http://www.apache.org/info/verification.html">Verifying Apache Software Foundation Releases</a> page. The committer’s pgp signatures can be found <a href="https://downloads.apache.org/incubator/crail/KEYS">here</a>.</p>
<p>For fast downloads, current source distributions are hosted on mirror servers; older source distributions are in the archive.
For security, hash and signature files are always hosted at Apache.</p>
@@ -130,6 +130,6 @@ For security, hash and signature files are always hosted at Apache.</p>
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/feed.xml b/content/feed.xml
index 0b77a4c..a352849 100644
--- a/content/feed.xml
+++ b/content/feed.xml
@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.8.5">Jekyll</generator><link href="http://crail.incubator.apache.org//feed.xml" rel="self" type="application/atom+xml" /><link href="http://crail.incubator.apache.org//" rel="alternate" type="text/html" /><updated>2020-01-14T11:20:49+01:00</updated><id>http://crail.incubator.apache.org//feed.xml</id><title type="html">The Apache Crail (Incubating) Project</ti [...]
+<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.2.0">Jekyll</generator><link href="http://crail.incubator.apache.org//feed.xml" rel="self" type="application/atom+xml" /><link href="http://crail.incubator.apache.org//" rel="alternate" type="text/html" /><updated>2021-09-20T15:36:37+02:00</updated><id>http://crail.incubator.apache.org//feed.xml</id><title type="html">The Apache Crail (Incubating) Project</ti [...]
<p>
Recently, suppport for Crail has been added to the <a href="https://github.com/brianfrankcooper/YCSB">YCSB</a> benchmark suite. In this blog we describe how to run the benchmark and briefly show some performance comparisons between Crail and other key-value stores running on DRAM, Flash and Optane such as <a href="https://www.aerospike.com">Aerospike</a> or <a href="https://ramcloud.atlassian.net/wiki/spaces/RAM/overview">RAMClo [...]
</p>
@@ -13,7 +13,7 @@ Remember that Crail exports a hierarchical storage namespace where individual no
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/storage_namespace.svg" width="400" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/storage_namespace.svg" width="400" /></div>
<p><br />
<br /></p>
@@ -23,7 +23,7 @@ In this blog we focus on Crail's KeyValue API available to users through the Tab
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore crail = CrailStore.newInstance();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore crail = CrailStore.newInstance();
CrailTable table = fs.create("/tab1", CrailNodeType.TABLE, ..., ...).get();
CrailKeyValue kv = fs.create("/tab1/key1", CrailNodeType.KEYVALUE, ..., ...).get();
</code></pre></div></div>
@@ -34,7 +34,7 @@ Here the table's name is "/tab1" and the key of the key-value pair is
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailOutputStream stream = kv.getOutputStream();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailOutputStream stream = kv.getOutputStream();
CrailBuffer buf = CrailBuffer.wrap("data".bytes());
stream.append(buf);
</code></pre></div></div>
@@ -44,7 +44,7 @@ Lookup and reading of a key-value pair is done in a similar fashion.
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore crail = CrailStore.newInstance();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore crail = CrailStore.newInstance();
CrailKeyValue kv = fs.lookup("/tab1/key1").get().asKeyValue();
CrailInputStream stream = kv.getInputStream();
CrailBuffer buf = crail.createBuffer();
@@ -71,7 +71,7 @@ In order to run the benchmark simply clone the YCSB repository and build the Cra
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ git clone http://github.com/brianfrankcooper/YCSB.git
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ git clone http://github.com/brianfrankcooper/YCSB.git
crail@clustermaster:~$ cd YCSB
crail@clustermaster:~$ mvn -pl com.yahoo.ycsb:crail-binding -am clean package
</code></pre></div></div>
@@ -82,7 +82,7 @@ You need to have a Crail deployment up and running to run the YCSB benchmark. Fo
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ ./bin/ycsb load crail -s -P workloads/workloada -P large.dat -p crail.enumeratekeys=true &gt;outputLoad.txt
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ ./bin/ycsb load crail -s -P workloads/workloada -P large.dat -p crail.enumeratekeys=true &gt;outputLoad.txt
</code></pre></div></div>
<div style="text-align: justify">
@@ -97,7 +97,7 @@ So far we have just loaded the data. Let's now run the actual benchmark which co
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ ./bin/ycsb run crail -s -P workloads/workloada
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ ./bin/ycsb run crail -s -P workloads/workloada
</code></pre></div></div>
<h3 id="ycsb-benchmark-performance-for-dram--intel-optane">YCSB Benchmark Performance for DRAM &amp; Intel Optane</h3>
@@ -112,7 +112,7 @@ Looking at small KV pairs first (left figure below), we can see that Crail's DRA
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/ycsb_put.svg" width="700" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/ycsb_put.svg" width="700" /></div>
<p><br />
<br /></p>
@@ -123,7 +123,7 @@ The reason Crail trails RAMCloud for small KV pairs is because Crail's internal
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/crail_put_anatomy.svg" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/crail_put_anatomy.svg" width="550" /></div>
<p><br />
<br /></p>
@@ -140,7 +140,7 @@ Besides update latency, we are also showing read latency in the CDF figure below
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/ycsb_get.svg" width="700" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/ycsb_get.svg" width="700" /></div>
<p><br />
<br /></p>
@@ -156,7 +156,7 @@ We used Crail's asynchronous API to benchmark Crail's key-value performance unde
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/iops_qd1.svg" width="650" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/iops_qd1.svg" width="650" /></div>
<p><br />
<br /></p>
@@ -167,7 +167,7 @@ In order to generate a higher load, we measured throughput and latencies for the
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/ycsb/iops_qd4.svg" width="650" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/ycsb/iops_qd4.svg" width="650" /></div>
<p><br />
<br /></p>
@@ -198,7 +198,7 @@ Remember that Crail is a tiered storage system where each storage tier consists
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/deployment/three_options.svg" width="580" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/deployment/three_options.svg" width="580" /></div>
<p><br />
<br /></p>
@@ -223,7 +223,7 @@ In all of the previously discussed configurations there is a one-to-one mapping
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/deployment/storage_class.svg" width="400" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/deployment/storage_class.svg" width="400" /></div>
<p><br />
<br /></p>
@@ -233,7 +233,7 @@ Storage classes can easily be defined in the slaves file as follows (see the <
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ cat $CRAIL_HOME/conf/slaves
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clustermaster:~$ cat $CRAIL_HOME/conf/slaves
clusternode1 -t org.apache.crail.storage.rdma.RdmaStorageTier -c 0
clusternode2 -t org.apache.crail.storage.rdma.RdmaStorageTier -c 0
clusternode1 -t org.apache.crail.storage.nvmf.NvmfStorageTier -c 1
@@ -247,7 +247,7 @@ One can also manually attach a storage server to a particular storage class:
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clusternode2:~$ $CRAIL_HOME/bin/crail datanode -t org.apache.crail.storage.nvmf.NvmfStorageTier -c 2
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>crail@clusternode2:~$ $CRAIL_HOME/bin/crail datanode -t org.apache.crail.storage.nvmf.NvmfStorageTier -c 2
</code></pre></div></div>
<div style="text-align: justify">
@@ -257,7 +257,7 @@ Remember that the storage class ID is implicitly ordering the storage tiers. Dur
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/deployment/timeline.svg" width="300" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/deployment/timeline.svg" width="300" /></div>
<p><br />
<br /></p>
@@ -267,7 +267,7 @@ If applications want to further prioritize the specific local resource of a mach
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailLocationClass local = fs.getLocationClass();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailLocationClass local = fs.getLocationClass();
CrailFile file = fs.create("/tmp.dat", CrailNodeType.DATAFILE, CrailStorageClass.DEFAULT, local).get().asFile();
</code></pre></div></div>
<div style="text-align: justify">
@@ -285,7 +285,7 @@ During the deployment of Crail, one has to decide on the storage capacity of eac
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/deployment/resource_provisioning.svg" width="400" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/deployment/resource_provisioning.svg" width="400" /></div>
<p><br />
<br /></p>
@@ -310,7 +310,7 @@ Today, data processing applications running in the cloud may implicitly use disa
</p>
<br />
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/spark_disagg.svg" width="580" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/spark_disagg.svg" width="580" /></div>
<br />
<br />
@@ -332,7 +332,7 @@ In a traditional shuffle operation, data is exchanged between map and reduce tas
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/overview.svg" width="350" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/overview.svg" width="350" /></div>
<p><br /></p>
<h3 id="challenge-large-number-of-small-files">Challenge: Large Number of Small Files</h3>
@@ -349,7 +349,7 @@ performed a simple experiment where we measured the size distribution of Spark s
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/cdf-plot.svg" width="480" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/cdf-plot.svg" width="480" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -367,7 +367,7 @@ To mitigate the overheads of writing and reading large numbers of small data set
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/optimization_tasks.svg" width="380" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/optimization_tasks.svg" width="380" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -378,7 +378,7 @@ The second optimization we use in our disaggregated Crail shuffler is efficient
Crail Multifiles offer zero-copy parallel reading of large numbers of files in a sequential manner. From the prespective of a map task MultiFiles are flat directories consisting of files belonging to different per-core file sets. From a reducer perspective, a MultFile looks like a large file that can be read sequentially using many in-flight operations. For instance, the following code shows how a reduce task during a Crail shuffle operation reads a partition from remote storage.
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore fs = CrailStore.newInstance();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailStore fs = CrailStore.newInstance();
CrailMultiFile multiFile = fs.lookup("/shuffle/partition1").get().asMultiFile();
ByteBuffer buffer = Buffer.allocateDirect(multiFile.size());
int batchSize = 16;
@@ -406,7 +406,7 @@ Shuffle operations, being essentially barriers between compute stages, are highl
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/machine_skew.svg" width="520" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/machine_skew.svg" width="520" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -416,7 +416,7 @@ One way to deal with this problem is through weighted fair scheduling of network
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/machine_skew_crail.svg" width="520" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/machine_skew_crail.svg" width="520" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -432,7 +432,7 @@ One may argue that chopping shuffle data up into blocks and transferring them ov
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/loadbalancing.svg" width="520" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/loadbalancing.svg" width="520" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -450,7 +450,7 @@ Note that Crail disaggregated storage may be provided by a few highly dense stor
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/terasort.svg" width="450" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/terasort.svg" width="450" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -468,7 +468,7 @@ Next we look at Spark SQL performance in a disaggregated configuration. Again we
</div>
<p><br /></p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/disaggregation/sql.svg" width="420" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/disaggregation/sql.svg" width="420" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -482,7 +482,7 @@ Often, high network bandwidth is assumed to be the key enabling factor for stora
<h3 id="summary">Summary</h3>
-<p>Efficient disaggregation of shuffle data is challenging, requiring shuffle managers and storage systems to be co-designed in order to effectively handle large numbers of small files, machine skew and loadbalancing issues. In this blog post we discussed the basic architecture of the Crail disaggregated shuffle engine and showed that by using Crail we can effectively disaggregate shuffle data in both bandwidth intensive map-reduce jobs as well as in more latency sensitive SQL work [...]
+<p>Efficient disaggregation of shuffle data is challenging, requiring shuffle managers and storage systems to be co-designed in order to effectively handle large numbers of small files, machine skew and loadbalancing issues. In this blog post we discussed the basic architecture of the Crail disaggregated shuffle engine and showed that by using Crail we can effectively disaggregate shuffle data in both bandwidth intensive map-reduce jobs as well as in more latency sensitive SQL work [...]
<p>
With python being used in many machine learning applications, serverless frameworks, etc.
as the go-to language, we believe a Crail client Python API would be a useful tool to
@@ -558,14 +558,14 @@ and does not interface with the CPython C-API directly.</li>
<p>In this blog post we focus on the overhead of calling a C/C++ function from Python.
We vary the number of arguments, argument types and the return types. We also
test passing strings to C/C++ since it is part of the Crail API e.g. when
-opening or creating a file. Some frameworks expect <code class="highlighter-rouge">bytes</code> when passing a string
-to a underlying <code class="highlighter-rouge">const char *</code>, some allow to pass a <code class="highlighter-rouge">str</code> and others allow both.
-If C++ is supported by the framework we also test passing a <code class="highlighter-rouge">std::string</code> to a
+opening or creating a file. Some frameworks expect <code class="language-plaintext highlighter-rouge">bytes</code> when passing a string
+to a underlying <code class="language-plaintext highlighter-rouge">const char *</code>, some allow to pass a <code class="language-plaintext highlighter-rouge">str</code> and others allow both.
+If C++ is supported by the framework we also test passing a <code class="language-plaintext highlighter-rouge">std::string</code> to a
C++ function. Note that we perform all benchmarks with CPython version 3.5.2.
We measure the time it takes to call the Python function until it returns.
-The C/C++ functions are empty, except a <code class="highlighter-rouge">return</code> statement where necessary.</p>
+The C/C++ functions are empty, except a <code class="language-plaintext highlighter-rouge">return</code> statement where necessary.</p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/python_c/python_c_foo.svg" width="725" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/python_c/python_c_foo.svg" width="725" /></div>
<p></p>
<p>The plot shows that adding more arguments to a function increases runtime.
@@ -586,26 +586,26 @@ between Cython and our extension module implementation we found that Cython make
better use of the C-API.</p>
<p>Our extension module implementation follows the official tutorial and uses
-<code class="highlighter-rouge">PyArg_ParseTuple</code> to parse the arguments. However as shown below we found that
-manually unpacking the arguments with <code class="highlighter-rouge">PyArg_UnpackTuple</code> already significantly
+<code class="language-plaintext highlighter-rouge">PyArg_ParseTuple</code> to parse the arguments. However as shown below we found that
+manually unpacking the arguments with <code class="language-plaintext highlighter-rouge">PyArg_UnpackTuple</code> already significantly
increased the performance. Although these numbers still do not match Cython’s
performance we did not further investigate possible optimizations
to our code.</p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/python_c/python_c_foo_opt.svg" width="725" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/python_c/python_c_foo_opt.svg" width="725" /></div>
<p></p>
-<p>Let’s take a look at the string performance. <code class="highlighter-rouge">bytes</code> and <code class="highlighter-rouge">str</code> is used whereever
+<p>Let’s take a look at the string performance. <code class="language-plaintext highlighter-rouge">bytes</code> and <code class="language-plaintext highlighter-rouge">str</code> is used whereever
applicable. To pass strings as bytes the ‘b’ prefix is used.</p>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/python_c/python_c_foo_str.svg" width="725" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/python_c/python_c_foo_str.svg" width="725" /></div>
<p></p>
<p>Again Cython and the extension module implementation with manual unpacking seem to
-deliver the best performance. Passing a 64bit value in form of a <code class="highlighter-rouge">const char *</code>
+deliver the best performance. Passing a 64bit value in form of a <code class="language-plaintext highlighter-rouge">const char *</code>
pointer seems to be slightly faster than passing an integer argument (up to 20%).
-Passing the string to a C++ function which takes a <code class="highlighter-rouge">std::string</code>
-is ~50% slower than passing a <code class="highlighter-rouge">const char *</code>, probably because of the
+Passing the string to a C++ function which takes a <code class="language-plaintext highlighter-rouge">std::string</code>
+is ~50% slower than passing a <code class="language-plaintext highlighter-rouge">const char *</code>, probably because of the
instantiation of the underlying data buffer and copying however we have not
confirmed this.</p>
@@ -653,7 +653,7 @@ this one requires the most glue code.</p>
<li>CPython 3.5.2</li>
</ul>
-<p>The source code is available on <a href="https://github.com/zrlio/Python-c-benchmark">GitHub</a></p></content><author><name>Jonas Pfefferle</name></author><category term="blog" /><summary type="html">With python being used in many machine learning applications, serverless frameworks, etc. as the go-to language, we believe a Crail client Python API would be a useful tool to broaden the use-case for Crail. Since the Crail core is written in Java, perfor [...]
+<p>The source code is available on <a href="https://github.com/zrlio/Python-c-benchmark">GitHub</a></p></content><author><name>Jonas Pfefferle</name></author><category term="blog" /><summary type="html">With python being used in many machine learning applications, serverless frameworks, etc. as the go-to language, we believe a Crail client Python API would be a useful tool to broaden the use-case for Crail. Since the Crail core is written in Java, perfor [...]
<p>
This is the first user blog post in a multi-part series where we will focus on relational data processing performance (e.g., SQL) in presence of high-performance network and storage devices - the kind of devices that Crail targets. Relational data processing is one of the most popular and versatile workloads people run in the cloud. The general idea is that data is stored in tables with a schema, and is processed using a domain specific language like SQL. Examples of some popular system [...]
</p>
@@ -690,11 +690,11 @@ This is the first user blog post in a multi-part series where we will focus on r
<p>In a typical cloud-based relational data processing setup, the input data is stored on an external data storage solution like HDFS or AWS S3. Data tables and their associated schema are converted into a storage-friendly format for optimal performance. Examples of some popular and familiar file formats are <a href="https://parquet.apache.org/">Apache Parquet</a>, <a href="https://orc.apache.org/">Apache ORC</a>, <a href="https:/ [...]
-<figure><div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sql-p1/outline.svg" width="550" /><figcaption>Figure 1: The benchmarking setup with HDFS and file formats on a 100 Gbps network with NVMe flash devices. All formats contains routines for compression, encoding, and value materialization with associated I/O buffer management and data copies routines.<p></p></figcaption></div>&l [...]
+<figure><div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sql-p1/outline.svg" width="550" /><figcaption>Figure 1: The benchmarking setup with HDFS and file formats on a 100 Gbps network with NVMe flash devices. All formats contains routines for compression, encoding, and value materialization with associated I/O buffer management and data copies routines.<p></p></figcaption></div [...]
<p>To benchmark the performance of file formats, we wrote a set of micro-benchmarks which are available at <a href="https://github.com/zrlio/fileformat-benchmarks">https://github.com/zrlio/fileformat-benchmarks</a>. We cannot use typical SQL micro-benchmarks because every SQL engine has its own favorite file format, on which it performs the best. Hence, in order to ensure parity, we decoupled the performance of reading the input file format from the SQL query [...]
-<figure><div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sql-p1/performance-all.svg" width="550" /><figcaption>Figure 2: Performance of JSON, Avro, Parquet, ORC, and Arrow on NVMe devices over a 100 Gbps network.<p></p></figcaption></div></figure>
+<figure><div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sql-p1/performance-all.svg" width="550" /><figcaption>Figure 2: Performance of JSON, Avro, Parquet, ORC, and Arrow on NVMe devices over a 100 Gbps network.<p></p></figcaption></div></figure>
<p>We evaluate the performance of the benchmark on a 3 node HDFS cluster connected using 100 Gbps RoCE. One datanode in HDFS contains 4 NVMe devices with a collective aggregate bandwidth of 12.5 GB/sec (equals to 100 Gbps, hence, we have a balanced network and storage performance). Figure 2 shows our results where none of the file formats is able to deliver the full hardware performance for reading input files. One third of the performance is already lost in HDFS (maximum throughpu [...]
@@ -708,7 +708,7 @@ This is the first user blog post in a multi-part series where we will focus on r
<li>Careful object materialization using a binary API: To optimize the runtime representation in managed runtimes like the JVM, only objects which are necessary for SQL processing are materialized. Otherwise, a 4 byte integer can be passed around as a byte array (using the binary API of Albis).</li>
</ul>
-<figure><div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sql-p1/core-scalability.svg" width="550" /><figcaption>Figure 3: Core scalability of JSON, Avro, Parquet, ORC, Arrow, and Albis on HDFS/NVMe.<p></p></figcaption></div></figure>
+<figure><div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sql-p1/core-scalability.svg" width="550" /><figcaption>Figure 3: Core scalability of JSON, Avro, Parquet, ORC, Arrow, and Albis on HDFS/NVMe.<p></p></figcaption></div></figure>
<p>Using the Albis format, we revise our previous experiment where we read the input store_sales table from HDFS. In the figure above, we show the performance of Albis and other file formats with number of CPU cores involved. At the right hand of the x-axis, we have performance with all 16 cores engaged, hence, representing the peak possible performance. As evident, Albis delivers 59.9 Gbps out of 74.9 Gbps possible bandwidth with HDFS over NVMe. Albis performance is 1.9 - 21.4x be [...]
@@ -747,9 +747,9 @@ This is the first user blog post in a multi-part series where we will focus on r
<h3 id="apache-crail-incubating-with-albis">Apache Crail (Incubating) with Albis</h3>
-<p>For our final experiment, we try to answer the question what it would take to deliver the full 100 Gbps bandwidth for Albis. Certainly, the first bottleneck is to improve the base storage layer performance. Here we use Apache Crail (Incubating) with its <a href="https://en.wikipedia.org/wiki/NVM_Express#NVMeOF">NVMeF</a> storage tier. This tier uses <a href="https://github.com/zrlio/jNVMf">jNVMf library</a> to implement NVMeF stack in [...]
+<p>For our final experiment, we try to answer the question what it would take to deliver the full 100 Gbps bandwidth for Albis. Certainly, the first bottleneck is to improve the base storage layer performance. Here we use Apache Crail (Incubating) with its <a href="https://en.wikipedia.org/wiki/NVM_Express#NVMeOF">NVMeF</a> storage tier. This tier uses <a href="https://github.com/zrlio/jNVMf">jNVMf library</a> to implement NVMeF stack in [...]
-<figure><div style="text-align:center"><img src="//crail.incubator.apache.org/img/blog/sql-p1/albis-crail.svg" width="550" /><figcaption>Figure 4: Performance of Albis on Crail.<p></p></figcaption></div></figure>
+<figure><div style="text-align:center"><img src="https://crail.incubator.apache.org/img/blog/sql-p1/albis-crail.svg" width="550" /><figcaption>Figure 4: Performance of Albis on Crail.<p></p></figcaption></div></figure>
<p>The left half of the figure shows the performance scalability of Albis on Crail in a setup with 1 core (8.9 Gbps) to 16 cores (85.5 Gbps). In comparison, the right half of the figure shows the performance of Crail on HDFS/NVMe at 59.9 Gbps, and on Crail/NVMe at 85.5 Gbps. The last bar shows the performance of Albis if the benchmark does not materialize Java object values. In this configuration, Albis on Crail delivers 91.3 Gbps, which is very close to the expected peak of 91.8 G [...]
diff --git a/content/index.html b/content/index.html
index 47a8ea3..abec634 100644
--- a/content/index.html
+++ b/content/index.html
@@ -4,12 +4,12 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: Overview</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/crail-stack-preview.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/crail-stack-preview.png" />
+ href="https://crail.incubator.apache.org/blog/blog.xml">
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/crail-stack-preview.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/crail-stack-preview.png" />
<meta property="keywords" content="storage disaggregation, RDMA, NVMe over Fabrics, NVMe-oF, Spark, Tensorflow, distributed data processing, serverless computing" />
</head>
@@ -21,50 +21,50 @@
<li class="active">
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -80,7 +80,7 @@
<br>
<div>
<center>
-<img src="//crail.incubator.apache.org/img/crail_stack3.png" align="middle" style="width: 340px; text-align:center;">
+<img src="https://crail.incubator.apache.org/img/crail_stack3.png" align="middle" style="width: 340px; text-align:center;">
</center>
</div>
<br>
@@ -142,7 +142,7 @@
<li class="shortnews">
<span class="date">August 9, 2018</span>
- <p>A new blog <a href="//crail.incubator.apache.org/blog/2018/08/sql-p1.html">post</a> discussing file formats performance is now online</p>
+ <p>A new blog <a href="https://crail.incubator.apache.org/blog/2018/08/sql-p1.html">post</a> discussing file formats performance is now online</p>
</li>
@@ -150,7 +150,7 @@
</ul>
-<p><a href="//crail.incubator.apache.org/news/">Older posts…</a></p>
+<p><a href="https://crail.incubator.apache.org/news/">Older posts…</a></p>
@@ -165,6 +165,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/news/index.html b/content/news/index.html
index 9044e64..fba2269 100644
--- a/content/news/index.html
+++ b/content/news/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: News</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/news-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/news-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/news-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/news-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -111,7 +111,7 @@
<li class="shortnews">
<span class="date">August 9, 2018</span>
- <p>A new blog <a href="//crail.incubator.apache.org/blog/2018/08/sql-p1.html">post</a> discussing file formats performance is now online</p>
+ <p>A new blog <a href="https://crail.incubator.apache.org/blog/2018/08/sql-p1.html">post</a> discussing file formats performance is now online</p>
</li>
@@ -135,7 +135,7 @@
<li class="shortnews">
<span class="date">June 4, 2018</span>
- <p>Apache Crail 1.0 incubator <a href="//crail.incubator.apache.org/download">release</a></p>
+ <p>Apache Crail 1.0 incubator <a href="https://crail.incubator.apache.org/download">release</a></p>
</li>
@@ -151,7 +151,7 @@
<li class="shortnews">
<span class="date">November 23, 2017</span>
- <p>New blog <a href="//crail.incubator.apache.org/blog/2017/11/crail-metadata.html">post</a> about Crail’s metadata performance and scalability</p>
+ <p>New blog <a href="https://crail.incubator.apache.org/blog/2017/11/crail-metadata.html">post</a> about Crail’s metadata performance and scalability</p>
</li>
@@ -167,7 +167,7 @@
<li class="shortnews">
<span class="date">November 17, 2017</span>
- <p>New blog <a href="//crail.incubator.apache.org/blog/2017/11/rdmashuffle.html">post</a> about SparkRDMA and Crail shuffle plugins</p>
+ <p>New blog <a href="https://crail.incubator.apache.org/blog/2017/11/rdmashuffle.html">post</a> about SparkRDMA and Crail shuffle plugins</p>
</li>
@@ -231,7 +231,7 @@
<li class="shortnews">
<span class="date">January 17, 2017</span>
- <p>New blog <a href="//crail.incubator.apache.org/blog/2017/01/sorting.html">post</a> about sorting on a 100Gbit/s cluster using Spark/Crail</p>
+ <p>New blog <a href="https://crail.incubator.apache.org/blog/2017/01/sorting.html">post</a> about sorting on a 100Gbit/s cluster using Spark/Crail</p>
</li>
@@ -266,6 +266,6 @@
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>
diff --git a/content/overview/index.html b/content/overview/index.html
index 8e10d26..2441ad1 100644
--- a/content/overview/index.html
+++ b/content/overview/index.html
@@ -4,13 +4,13 @@
<meta charset="utf-8">
<title>The Apache Crail (Incubating) Project: I/O Challenges</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link href="//crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
- <link href="//crail.incubator.apache.org/css/group.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
+ <link href="https://crail.incubator.apache.org/css/group.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Atom"
- href="//crail.incubator.apache.org/blog/blog.xml">
+ href="https://crail.incubator.apache.org/blog/blog.xml">
- <meta property="og:image" content="//crail.incubator.apache.org/img/blog/preview/overview-summary.png" />
- <meta property="og:image:secure_url" content="//crail.incubator.apache.org/img/blog/preview/overview-summary.png" />
+ <meta property="og:image" content="https://crail.incubator.apache.org/img/blog/preview/overview-summary.png" />
+ <meta property="og:image:secure_url" content="https://crail.incubator.apache.org/img/blog/preview/overview-summary.png" />
</head>
<body>
@@ -21,50 +21,50 @@
<li >
- <a href="//crail.incubator.apache.org/">
+ <a href="https://crail.incubator.apache.org/">
Home
</a>
</li>
<li class="active">
- <a href="//crail.incubator.apache.org/overview/">
+ <a href="https://crail.incubator.apache.org/overview/">
Overview
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/download/">
+ <a href="https://crail.incubator.apache.org/download/">
Downloads
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/blog/">
+ <a href="https://crail.incubator.apache.org/blog/">
Blog
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/community/">
+ <a href="https://crail.incubator.apache.org/community/">
Community
</a>
</li>
<li >
- <a href="//crail.incubator.apache.org/documentation/">
+ <a href="https://crail.incubator.apache.org/documentation/">
Documentation
</a>
</li>
</ul>
- <a href="//crail.incubator.apache.org/">
- <img src="//crail.incubator.apache.org/img/crail_logo.png"
- srcset="//crail.incubator.apache.org/img/crail_logo.png"
+ <a href="https://crail.incubator.apache.org/">
+ <img src="https://crail.incubator.apache.org/img/crail_logo.png"
+ srcset="https://crail.incubator.apache.org/img/crail_logo.png"
alt="Crail" id="logo">
</a>
</div>
@@ -92,7 +92,7 @@ Modern storage and network technologies such as 100Gb/s Ethernet, RDMA, NVMe fla
</li>
</ul>
-<p>In the <a href="//crail.incubator.apache.org/blog">Blog</a> section we discuss each of those challenges in more detail.</p>
+<p>In the <a href="https://crail.incubator.apache.org/blog">Blog</a> section we discuss each of those challenges in more detail.</p>
<h2 id="crail-architecture">Crail Architecture</h2>
@@ -133,7 +133,7 @@ backed with separate files inside the directory. Keyvalue nodes are similar to d
</p>
</div>
-<div style="text-align:center"><img src="//crail.incubator.apache.org/overview/filesystem2.png" width="550" /></div>
+<div style="text-align:center"><img src="https://crail.incubator.apache.org/overview/filesystem2.png" width="550" /></div>
<p><br /></p>
<div style="text-align: justify">
@@ -152,7 +152,7 @@ Crail's top level storage API offers asynchronous non-blocking functions for rea
Crail not only exports a Java API but also is written entirely in Java, which makes it easy to use and allows for a better integration with data processing frameworks like Spark, Flink, Hadoop, etc. A simple example of a Crail write operation is shown below:
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailConfiguration conf = new CrailConfiguration();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CrailConfiguration conf = new CrailConfiguration();
CrailStore store = CrailStore.newInstance(conf);
CrailFile file = store.createFile(filename, 0, 0).get().syncDir();
CrailOutputStream outstream = file.getDirectOutputStream();
@@ -172,7 +172,7 @@ Crail uses <a href="https://github.com/zrlio/disni">DiSNI</a>, a user-level netw
<p>The Crail HDFS adaptor enables users to access Crail using the standard HDFS API. For instance, administrators can interact with Crail using the standard HDFS shell:</p>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail fs -mkdir /test
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./bin/crail fs -mkdir /test
./bin/crail fs -ls /
./bin/crail fs -copyFromLocal <path-to-local-file>
./bin/crail fs -cat /test/<file-name>
@@ -180,7 +180,7 @@ Crail uses <a href="https://github.com/zrlio/disni">DiSNI</a>, a user-level netw
<p>Moreover, regular HDFS-based applications will transparently work with Crail when using fully qualified path names (or when specifying Crail as the default Hadoop file system):</p>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Configuration conf = new Configuration();
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
fs.create("crail://test/hello.txt");
</code></pre></div></div>
@@ -209,7 +209,7 @@ The Crail-based broadcast plugin for Spark stores broadcast variables in Crail f
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>spark.shuffle.manager org.apache.spark.shuffle.crail.CrailShuffleManager
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>spark.shuffle.manager org.apache.spark.shuffle.crail.CrailShuffleManager
spark.broadcast.factory org.apache.spark.broadcast.CrailBroadcastFactory
</code></pre></div></div>
@@ -232,7 +232,7 @@ Application specific serializers and sorters can be defined by setting the follo
</p>
</div>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>spark.crail.shuffle.serializer
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>spark.crail.shuffle.serializer
spark.crail.shuffle.sorter
</code></pre></div></div>
@@ -249,6 +249,6 @@ spark.crail.shuffle.sorter
<!-- Support retina images. -->
<script type="text/javascript"
- src="//crail.incubator.apache.org/js/srcset-polyfill.js"></script>
+ src="https://crail.incubator.apache.org/js/srcset-polyfill.js"></script>
</body>
</html>