You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ozone.apache.org by we...@apache.org on 2022/12/23 01:21:16 UTC
[ozone-site] 02/03: Embed Matomo trace code into 0.4.1-alpha ~ 0.2.1-alpha docs.
This is an automated email from the ASF dual-hosted git repository.
weichiu pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/ozone-site.git
commit dcf8570632cf3cc8b89d7b4a103a2c0776eb01e8
Author: Wei-Chiu Chuang <we...@apache.org>
AuthorDate: Thu Dec 22 17:11:06 2022 -0800
Embed Matomo trace code into 0.4.1-alpha ~ 0.2.1-alpha docs.
---
docs/0.2.1-alpha/OzoneOverview.svg | 2 +-
docs/0.2.1-alpha/bucketcommands.html | 82 +---
docs/0.2.1-alpha/buildingsources.html | 43 +--
docs/0.2.1-alpha/categories/index.xml | 10 +-
docs/0.2.1-alpha/commandshell.html | 60 +--
docs/0.2.1-alpha/concepts.html | 71 +---
docs/0.2.1-alpha/dozone.html | 78 +---
docs/0.2.1-alpha/freon.html | 62 +---
docs/0.2.1-alpha/hdds.html | 33 +-
docs/0.2.1-alpha/index.html | 31 +-
docs/0.2.1-alpha/index.xml | 31 +-
docs/0.2.1-alpha/javaapi.html | 191 ++++------
docs/0.2.1-alpha/keycommands.html | 81 +---
docs/0.2.1-alpha/ozonefs.html | 62 +---
docs/0.2.1-alpha/ozonemanager.html | 53 +--
docs/0.2.1-alpha/realcluster.html | 50 +--
docs/0.2.1-alpha/rest.html | 432 ++++++---------------
docs/0.2.1-alpha/runningviadocker.html | 62 +---
docs/0.2.1-alpha/runningwithhdfs.html | 55 +--
docs/0.2.1-alpha/scmcli.html | 20 +-
docs/0.2.1-alpha/settings.html | 113 ++----
docs/0.2.1-alpha/sitemap.xml | 116 ++----
docs/0.2.1-alpha/tags/index.xml | 10 +-
docs/0.2.1-alpha/volumecommands.html | 76 +---
docs/0.3.0-alpha/OzoneOverview.svg | 2 +-
docs/0.3.0-alpha/bucketcommands.html | 104 ++----
docs/0.3.0-alpha/buildingsources.html | 65 ++--
docs/0.3.0-alpha/categories/index.xml | 10 +-
docs/0.3.0-alpha/commandshell.html | 82 ++--
docs/0.3.0-alpha/concepts.html | 93 ++---
docs/0.3.0-alpha/dozone.html | 100 ++---
docs/0.3.0-alpha/freon.html | 84 ++---
docs/0.3.0-alpha/hdds.html | 55 ++-
docs/0.3.0-alpha/index.html | 53 +--
docs/0.3.0-alpha/index.xml | 31 +-
docs/0.3.0-alpha/javaapi.html | 213 +++++------
docs/0.3.0-alpha/keycommands.html | 103 ++---
docs/0.3.0-alpha/ozonefs.html | 84 ++---
docs/0.3.0-alpha/ozonemanager.html | 75 ++--
docs/0.3.0-alpha/realcluster.html | 72 ++--
docs/0.3.0-alpha/rest.html | 454 +++++++----------------
docs/0.3.0-alpha/runningviadocker.html | 84 ++---
docs/0.3.0-alpha/runningwithhdfs.html | 77 ++--
docs/0.3.0-alpha/s3.html | 120 ++----
docs/0.3.0-alpha/scmcli.html | 42 ++-
docs/0.3.0-alpha/settings.html | 135 +++----
docs/0.3.0-alpha/sitemap.xml | 121 +++---
docs/0.3.0-alpha/tags/index.xml | 10 +-
docs/0.3.0-alpha/volumecommands.html | 98 ++---
docs/0.4.0-alpha/OzoneOverview.svg | 2 +-
docs/0.4.0-alpha/auditparser.html | 63 ++--
docs/0.4.0-alpha/bucketcommands.html | 110 ++----
docs/0.4.0-alpha/buildingsources.html | 75 ++--
docs/0.4.0-alpha/categories/index.xml | 10 +-
docs/0.4.0-alpha/commandshell.html | 81 ++--
docs/0.4.0-alpha/concepts.html | 93 ++---
docs/0.4.0-alpha/css/ozonedoc.css | 2 +-
docs/0.4.0-alpha/dozone.html | 98 ++---
docs/0.4.0-alpha/freon.html | 84 ++---
docs/0.4.0-alpha/genconf.html | 42 ++-
docs/0.4.0-alpha/hdds.html | 55 ++-
docs/0.4.0-alpha/index.html | 54 +--
docs/0.4.0-alpha/index.xml | 44 +--
docs/0.4.0-alpha/javaapi.html | 208 +++++------
docs/0.4.0-alpha/keycommands.html | 115 ++----
docs/0.4.0-alpha/ozonefs.html | 128 +++----
docs/0.4.0-alpha/ozonemanager.html | 75 ++--
docs/0.4.0-alpha/ozonesecurityarchitecture.html | 88 ++---
docs/0.4.0-alpha/prometheus.html | 116 +++---
docs/0.4.0-alpha/realcluster.html | 72 ++--
docs/0.4.0-alpha/runningviadocker.html | 85 ++---
docs/0.4.0-alpha/runningwithhdfs.html | 77 ++--
docs/0.4.0-alpha/s3.html | 136 +++----
docs/0.4.0-alpha/scmcli.html | 42 ++-
docs/0.4.0-alpha/settings.html | 135 +++----
docs/0.4.0-alpha/setupsecureozone.html | 96 ++---
docs/0.4.0-alpha/sitemap.xml | 148 +++-----
docs/0.4.0-alpha/sparkozonefsk8s.html | 153 +++-----
docs/0.4.0-alpha/tags/index.xml | 10 +-
docs/0.4.0-alpha/volumecommands.html | 98 ++---
docs/0.4.1-alpha/beyond.html | 54 +--
docs/0.4.1-alpha/beyond/containers.html | 209 +++--------
docs/0.4.1-alpha/beyond/dockercheatsheet.html | 85 ++---
docs/0.4.1-alpha/beyond/index.xml | 23 +-
docs/0.4.1-alpha/beyond/runningwithhdfs.html | 74 ++--
docs/0.4.1-alpha/concept.html | 55 +--
docs/0.4.1-alpha/concept/datanodes.html | 59 ++-
docs/0.4.1-alpha/concept/hdds.html | 45 +--
docs/0.4.1-alpha/concept/index.xml | 24 +-
docs/0.4.1-alpha/concept/overview.html | 74 ++--
docs/0.4.1-alpha/concept/ozonemanager.html | 90 +++--
docs/0.4.1-alpha/index.html | 65 ++--
docs/0.4.1-alpha/index.xml | 144 ++-----
docs/0.4.1-alpha/interface.html | 55 ++-
docs/0.4.1-alpha/interface/index.xml | 25 +-
docs/0.4.1-alpha/interface/javaapi.html | 207 +++++------
docs/0.4.1-alpha/interface/ozonefs.html | 155 ++++----
docs/0.4.1-alpha/interface/s3.html | 137 +++----
docs/0.4.1-alpha/recipe.html | 50 +--
docs/0.4.1-alpha/recipe/index.xml | 19 +-
docs/0.4.1-alpha/recipe/prometheus.html | 128 ++++---
docs/0.4.1-alpha/recipe/sparkozonefsk8s.html | 179 ++++-----
docs/0.4.1-alpha/security.html | 71 ++--
docs/0.4.1-alpha/security/index.xml | 33 +-
docs/0.4.1-alpha/security/secuitywithranger.html | 46 +--
docs/0.4.1-alpha/security/secureozone.html | 163 ++------
docs/0.4.1-alpha/security/securingdatanodes.html | 75 ++--
docs/0.4.1-alpha/security/securings3.html | 70 ++--
docs/0.4.1-alpha/security/securingtde.html | 68 ++--
docs/0.4.1-alpha/security/securityacls.html | 60 ++-
docs/0.4.1-alpha/shell.html | 63 ++--
docs/0.4.1-alpha/shell/bucketcommands.html | 91 ++---
docs/0.4.1-alpha/shell/format.html | 76 ++--
docs/0.4.1-alpha/shell/index.xml | 32 +-
docs/0.4.1-alpha/shell/keycommands.html | 115 ++----
docs/0.4.1-alpha/shell/volumecommands.html | 98 ++---
docs/0.4.1-alpha/sitemap.xml | 223 +++++------
docs/0.4.1-alpha/start.html | 99 +++--
docs/0.4.1-alpha/start/fromsource.html | 73 ++--
docs/0.4.1-alpha/start/index.xml | 32 +-
docs/0.4.1-alpha/start/kubernetes.html | 62 ++--
docs/0.4.1-alpha/start/minikube.html | 88 ++---
docs/0.4.1-alpha/start/onprem.html | 169 ++++-----
docs/0.4.1-alpha/start/runningviadocker.html | 76 ++--
docs/0.4.1-alpha/start/startfromdockerhub.html | 101 +++--
docs/0.4.1-alpha/tools.html | 67 ++--
docs/0.4.1-alpha/tools/auditparser.html | 63 ++--
docs/0.4.1-alpha/tools/genconf.html | 42 ++-
docs/0.4.1-alpha/tools/index.xml | 26 +-
docs/0.4.1-alpha/tools/scmcli.html | 42 ++-
docs/0.4.1-alpha/tools/testtools.html | 99 ++---
131 files changed, 4153 insertions(+), 6912 deletions(-)
diff --git a/docs/0.2.1-alpha/OzoneOverview.svg b/docs/0.2.1-alpha/OzoneOverview.svg
index c7fda5d..9d4660d 100644
--- a/docs/0.2.1-alpha/OzoneOverview.svg
+++ b/docs/0.2.1-alpha/OzoneOverview.svg
@@ -235,4 +235,4 @@
</text>
</g>
</g>
-</svg>
+</svg>
\ No newline at end of file
diff --git a/docs/0.2.1-alpha/bucketcommands.html b/docs/0.2.1-alpha/bucketcommands.html
index febbf0e..f465e02 100644
--- a/docs/0.2.1-alpha/bucketcommands.html
+++ b/docs/0.2.1-alpha/bucketcommands.html
@@ -188,27 +188,9 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Bucket Commands</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone shell supports the following bucket commands.</p>
-
<ul>
<li><a href="#create">create</a></li>
<li><a href="#delete">delete</a></li>
@@ -216,13 +198,9 @@
<li><a href="#list">list</a></li>
<li><a href="#update">update</a></li>
</ul>
-
<h3 id="create">Create</h3>
-
<p>The bucket create command allows a user to create a bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -230,7 +208,6 @@
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -238,19 +215,13 @@
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket create /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket create /hive/jan</span></span></code></pre></div>
<p>The above command will create a bucket called <em>jan</em> in the <em>hive</em> volume.
Since no scheme was specified this command defaults to O3 (RPC) protocol.</p>
-
<h3 id="delete">Delete</h3>
-
<p>The bucket delete commands allows an user to delete a volume. If the
bucket is not empty then this command will fail.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -258,7 +229,6 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -266,16 +236,11 @@ bucket is not empty then this command will fail.</p>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume delete /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume delete /hive/jan</span></span></code></pre></div>
<p>The above command will delete <em>jan</em> bucket if it is empty.</p>
-
<h3 id="info">Info</h3>
-
<p>The bucket info commands returns the information about the bucket.
-<strong><em>Params:</em></strong></p>
-
+<em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -283,7 +248,6 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -291,17 +255,11 @@ bucket is not empty then this command will fail.</p>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket info /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket info /hive/jan</span></span></code></pre></div>
<p>The above command will print out the information about <em>jan</em> bucket.</p>
-
<h3 id="list">List</h3>
-
<p>The bucket list commands allows uset to list the buckets in a volume.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -309,40 +267,30 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>-l, –length</td>
<td>Maximum number of results to return. Default: 100</td>
</tr>
-
<tr>
<td>-p, –prefix</td>
<td>Optional, Only buckets that match this prefix will be returned.</td>
</tr>
-
<tr>
<td>-s, –start</td>
<td>The listing will start from key after the start key.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the <em>volume</em>.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket list /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket list /hive</span></span></code></pre></div>
<p>This command will list all buckets on the volume <em>hive</em>.</p>
-
<h3 id="update">Update</h3>
-
<p>The bucket update command allows changing access permissions on bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -350,29 +298,23 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>–addAcl</td>
<td>Optional, Comma separated ACLs that will added to bucket.</td>
</tr>
-
<tr>
<td>–removeAcl</td>
<td>Optional, Comma separated list of acl to remove.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the bucket.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket update --addAcl<span style="color:#f92672">=</span>user:bilbo:rw /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket update --addAcl<span style="color:#f92672">=</span>user:bilbo:rw /hive/jan</span></span></code></pre></div>
<p>The above command gives user bilbo read/write permission to the bucket.</p>
-
<p>You can try out these commands from the docker instance of the <a href="runningviadocker.html">Alpha
Cluster</a>.</p>
diff --git a/docs/0.2.1-alpha/buildingsources.html b/docs/0.2.1-alpha/buildingsources.html
index 817787f..d457176 100644
--- a/docs/0.2.1-alpha/buildingsources.html
+++ b/docs/0.2.1-alpha/buildingsources.html
@@ -188,52 +188,27 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Building from Sources</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-<p><strong><em>This is a guide on how to build the ozone sources. If you are <font
-color="red">not</font>
-planning to build sources yourself, you can safely skip this page.</em></strong></p>
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
+<p><em><strong>This is a guide on how to build the ozone sources. If you are <!-- raw HTML omitted -->not<!-- raw HTML omitted -->
+planning to build sources yourself, you can safely skip this page.</strong></em></p>
<p>If you are a Hadoop ninja, and wise in the ways of Apache, you already know
that a real Apache release is a source release.</p>
-
<p>If you want to build from sources, Please untar the source tarball and run
the ozone build command. This instruction assumes that you have all the
dependencies to build Hadoop on your build machine. If you need instructions
on how to build Hadoop, please look at the Apache Hadoop Website.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">mvn clean package -DskipTests<span style="color:#f92672">=</span>true -Dmaven.javadoc.skip<span style="color:#f92672">=</span>true -Phdds -Pdist -Dtar -DskipShade</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>mvn clean package -DskipTests<span style="color:#f92672">=</span>true -Dmaven.javadoc.skip<span style="color:#f92672">=</span>true -Phdds -Pdist -Dtar -DskipShade</span></span></code></pre></div>
<p>This will build an ozone-<version>.tar.gz in your target directory.</p>
-
<p>You can copy this tarball and use this instead of binary artifacts that are
provided along with the official release.</p>
-
<h2 id="how-to-test-the-build">How to test the build</h2>
-
<p>You can run the acceptance tests in the hadoop-ozone directory to make sure
that your build is functional. To launch the acceptance tests, please follow
- the instructions in the <strong>README.md</strong> in the
- <code>$hadoop_src/hadoop-ozone/acceptance-test</code> directory. Acceptance tests
- will start a small ozone cluster and verify that ozone shell and ozone file
- system is fully functional.</p>
+the instructions in the <strong>README.md</strong> in the
+<code>$hadoop_src/hadoop-ozone/acceptance-test</code> directory. Acceptance tests
+will start a small ozone cluster and verify that ozone shell and ozone file
+system is fully functional.</p>
</div>
</div>
diff --git a/docs/0.2.1-alpha/categories/index.xml b/docs/0.2.1-alpha/categories/index.xml
index 5c6fe28..eca954b 100644
--- a/docs/0.2.1-alpha/categories/index.xml
+++ b/docs/0.2.1-alpha/categories/index.xml
@@ -1,14 +1,10 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Categories on Ozone</title>
<link>/categories.html</link>
<description>Recent content in Categories on Ozone</description>
<generator>Hugo -- gohugo.io</generator>
- <language>en-us</language>
-
- <atom:link href="/categories/index.xml" rel="self" type="application/rss+xml" />
-
-
+ <language>en-us</language><atom:link href="/categories/index.xml" rel="self" type="application/rss+xml" />
</channel>
-</rss>
\ No newline at end of file
+</rss>
diff --git a/docs/0.2.1-alpha/commandshell.html b/docs/0.2.1-alpha/commandshell.html
index a5a4f3c..6d55853 100644
--- a/docs/0.2.1-alpha/commandshell.html
+++ b/docs/0.2.1-alpha/commandshell.html
@@ -188,31 +188,11 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Ozone CLI</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone has a set of command line tools that can be used to manage ozone.</p>
-
<p>All these commands are invoked via the <code>ozone</code> script.</p>
-
<p>The commands supported by ozone are:</p>
-
<ul>
<li><strong>classpath</strong> - Prints the class path needed to get the hadoop jar and the
required libraries.</li>
@@ -234,34 +214,24 @@ Container Manager.</li>
<li><strong>genconf</strong> - Generate minimally required ozone configs and output to
ozone-site.xml.</li>
</ul>
-
<h2 id="understanding-ozone-command-shell">Understanding Ozone command shell</h2>
-
<p>The most used command when working with Ozone is the Ozone command shell.
Ozone command shell gives a command shell interface to work against
Ozone.</p>
-
<p>The Ozone shell commands take the following format.</p>
-
<blockquote>
<p><em>ozone sh object action url</em></p>
</blockquote>
-
<p><strong>ozone</strong> script is used to invoke all Ozone sub-commands. The ozone shell is
invoked via <code>sh</code> command.</p>
-
<p>The object can be a volume, bucket or a key. The action is various verbs like
- create, list, delete etc.</p>
-
+create, list, delete etc.</p>
<p>Ozone URL can point to a volume, bucket or keys in the following format:</p>
-
<p><em>[scheme][server:port]/volume/bucket/key</em></p>
-
<p>Where,</p>
-
<ol>
-<li><p>Scheme - Can be one of the following</p>
-
+<li>
+<p>Scheme - Can be one of the following</p>
<ul>
<li>o3 - Ozone’s native RPC protocol. If you specify this scheme, the
native RPC protocol is used while communicating with Ozone Manager and
@@ -271,29 +241,25 @@ that you are interested in using the Ozone Rest protocol and falls back
to using the REST protocol instead of RPC.
If no protocol is specified, the Ozone shell defaults to the native RPC
protocol.</li>
-</ul></li>
-
-<li><p>Server:Port - This is the address of the Ozone Manager. This can be server
+</ul>
+</li>
+<li>
+<p>Server:Port - This is the address of the Ozone Manager. This can be server
only, in that case, the default port is used. If this value is omitted
then the defaults specified in the ozone-site.xml will be used for Ozone
-Manager address.</p></li>
+Manager address.</p>
+</li>
</ol>
-
<p>Depending on the call, the volume/bucket/key names will be part of the URL.
Please see volume commands, bucket commands, and key commands section for more
detail.</p>
-
<h2 id="invoking-help">Invoking help</h2>
-
<p>Ozone shell help can be invoked at <em>object</em> level or at <em>action</em> level.
For example:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume --help</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume --help</span></span></code></pre></div>
<p>This will show all possible actions for volumes.</p>
-
<p>or it can be invoked to explain a specific action like
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume create --help</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume create --help</span></span></code></pre></div>
This command will give you command line options of the create command.</p>
</div>
diff --git a/docs/0.2.1-alpha/concepts.html b/docs/0.2.1-alpha/concepts.html
index bccaa5d..eb33fca 100644
--- a/docs/0.2.1-alpha/concepts.html
+++ b/docs/0.2.1-alpha/concepts.html
@@ -188,116 +188,61 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Architecture</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone is a redundant, distributed object store build by
leveraging primitives present in HDFS. The primary design point of ozone is scalability, and it aims to scale to billions of objects.</p>
-
<p>Ozone consists of volumes, buckets, and keys. A volume is similar to a home directory in the ozone world. Only an administrator can create it. Volumes are used to store buckets. Once a volume is created users can create as many buckets as needed. Ozone stores data as keys which live inside these buckets.</p>
-
<p>Ozone namespace is composed of many storage volumes. Storage volumes are also used as the basis for storage accounting.</p>
-
<p>To access a key, an Ozone URL has the following format:</p>
-
-<pre><code>http://servername:port/volume/bucket/key
-</code></pre>
-
-<p>Where the server name is the name of a data node, the port is the data node HTTP port. The volume represents the name of the ozone volume; bucket is an ozone bucket created by the user and key represents the file.</p>
-
+<pre tabindex="0"><code>http://servername:port/volume/bucket/key
+</code></pre><p>Where the server name is the name of a data node, the port is the data node HTTP port. The volume represents the name of the ozone volume; bucket is an ozone bucket created by the user and key represents the file.</p>
<p>Please look at the <a href="./commandshell.html#shell">command line interface</a> for more info.</p>
-
<p>Ozone supports both REST and RPC protocols. Clients can choose either of these protocols to communicate with Ozone. Please see the <a href="./javaapi.html">client documentation</a> for more details.</p>
-
<p>Ozone separates namespace management and block space management; this helps
ozone to scale much better. The namespace is managed by a daemon called
<a href="./ozonemanager.html">Ozone Manager </a> (OM), and block space is
-managed by <a href="./hdds.html">Storage Container Manager</a> (SCM).</p>
-
+managed by [Storage Container Manager] (/hdds.html) (SCM).</p>
<p>The data nodes provide replication and ability to store blocks; these blocks are stored in groups to reduce the metadata pressure on SCM. This groups of blocks are called storage containers. Hence the block manager is called storage container
manager.</p>
-
<h2 id="ozone-overview">Ozone Overview</h2>
-
<p>
The following diagram is a high-level overview of the core components of Ozone.
</p>
-
-<p><img src="../../OzoneOverview.svg" alt="Architecture diagram" /></p>
-
+<p><img src="../../OzoneOverview.svg" alt="Architecture diagram"></p>
<p>The main elements of Ozone are
:</p>
-
<h3 id="ozone-manager">Ozone Manager
</h3>
-
<p><a href="./ozonemanager.html">Ozone Manager</a> (OM) takes care of the Ozone’s namespace.
All ozone objects like volumes, buckets, and keys are managed by OM. In Short, OM is the metadata manager for Ozone.
OM talks to blockManager(SCM) to get blocks and passes it on to the Ozone
client. Ozone client writes data to these blocks.
OM will eventually be replicated via Apache Ratis for High Availability.
</p>
-
<h3 id="storage-container-manager">Storage Container Manager</h3>
-
<p><a href="./hdds.html">Storage Container Manager</a> (SCM) is the block and cluster manager for Ozone.
SCM along with data nodes offer a service called ‘storage containers’.
A storage container is a group unrelated of blocks that are managed together as a single entity.</p>
-
<p>SCM offers the following abstractions.
</p>
-
-<p><img src="../../SCMBlockDiagram.png" alt="SCM Abstractions" /></p>
-
+<p><img src="../../SCMBlockDiagram.png" alt="SCM Abstractions"></p>
<h3 id="blocks">Blocks</h3>
-
<p>Blocks are similar to blocks in HDFS. They are replicated store of data. Client writes data to blocks.</p>
-
<h3 id="containers">Containers</h3>
-
<p>A collection of blocks replicated and managed together.</p>
-
<h3 id="pipelines">Pipelines</h3>
-
<p>SCM allows each storage container to choose its method of replication.
For example, a storage container might decide that it needs only one copy of a block
and might choose a stand-alone pipeline. Another storage container might want to have a very high level of reliability and pick a RATIS based pipeline. In other words, SCM allows different kinds of replication strategies to co-exist. The client while writing data, chooses a storage container with required properties.</p>
-
<h3 id="pools">Pools</h3>
-
<p>A group of data nodes is called a pool. For scaling purposes,
we define a pool as a set of machines. This makes management of data nodes easier.</p>
-
<h3 id="nodes">Nodes</h3>
-
<p>The data node where data is stored. SCM monitors these nodes via heartbeat.</p>
-
<h3 id="clients">Clients</h3>
-
-<p>Ozone ships with a set of clients. Ozone <a href="./commandshell.html#shell">CLI</a> is the command line interface like ‘hdfs’ command.
<a href="./freon.html">Freon</a> is a load generation tool for Ozone.
</p>
-
+<p>Ozone ships with a set of clients. Ozone <a href="./commandshell.html#shell">CLI</a> is the command line interface like ‘hdfs’ command.
[Freon] (/freon.html) is a load generation tool for Ozone.
</p>
<h3 id="rest-handler">REST Handler</h3>
-
<p>Ozone provides an RPC (Remote Procedure Call) as well as a REST (Representational State Transfer) interface. This allows clients to be written in many languages quickly. Ozone strives to maintain an API compatibility between REST and RPC.
For most purposes, a client can make one line change to switch from REST to RPC or vice versa.
</p>
-
<h3 id="ozone-file-system">Ozone File System</h3>
-
<p>Ozone file system (TODO: Add documentation) is a Hadoop compatible file system. This allows Hadoop services and applications like Hive and Spark to run against
Ozone without any change.</p>
-
<h3 id="ozone-client">Ozone Client</h3>
-
<p>This is similar to DFSClient in HDFS. This is the standard client to talk to Ozone. All other components that we have discussed so far rely on Ozone client. Ozone client supports both RPC and REST protocols.</p>
</div>
diff --git a/docs/0.2.1-alpha/dozone.html b/docs/0.2.1-alpha/dozone.html
index a457a89..ea327b2 100644
--- a/docs/0.2.1-alpha/dozone.html
+++ b/docs/0.2.1-alpha/dozone.html
@@ -188,91 +188,45 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Dozone & Dev Tools</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Dozone stands for docker for ozone. Ozone supports docker to make it easy to develop and test ozone. Starting a docker based ozone container is simple.</p>
-
<p>In the <code>compose/ozone</code> directory there are two files that define the docker and ozone settings.</p>
-
<p>Developers can</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">cd compose/ozone</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>cd compose/ozone</span></span></code></pre></div>
<p>and simply run</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose up -d</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose up -d</span></span></code></pre></div>
<p>to run a ozone cluster on docker.</p>
-
<p>This command will launch a Namenode, OM, SCM and a data node.</p>
-
-<p>To access the OM UI, one can run ‘<a href="http://localhost:9874'">http://localhost:9874'</a>.</p>
-
+<p>To access the OM UI, one can run ‘http://localhost:9874’.</p>
<p><em>Please note</em>: dozone does not map the data node ports to the 9864. Instead, it maps to the ephemeral port range. So many examples in the command shell will not work if you run those commands from the host machine. To find out where the data node port is listening, you can run the <code>docker ps</code> command or always ssh into a container before running ozone commands.</p>
-
<p>To shutdown a running docker based ozone cluster, please run</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose down</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose down</span></span></code></pre></div>
<h2 id="adding-more-config-settings">Adding more config settings</h2>
-
<p>The file called <code>docker-config</code> contains all ozone specific config settings. This file is processed to create the ozone-site.xml.</p>
-
-<h2 id="useful-docker-ozone-commands">Useful Docker & Ozone Commands</h2>
-
+<h2 id="useful-docker--ozone-commands">Useful Docker & Ozone Commands</h2>
<p>If you make any modifications to ozone, the simplest way to test it is to run freon and unit tests.</p>
-
<p>Here are the instructions to run corona in a docker based cluster.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose exec datanode bash</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose exec datanode bash</span></span></code></pre></div>
<p>This will open a bash shell on the data node container.
Now we can execute corona for load generation.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span></span></span></code></pre></div>
<p>Here is a set helpful commands while working with docker for ozone.
To check the status of the components:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose ps</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose ps</span></span></code></pre></div>
<p>To get logs from a specific node/service:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose logs scm</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose logs scm</span></span></code></pre></div>
<p>As the WebUI ports are forwarded to the external machine, you can check the web UI:</p>
-
<ul>
-<li>For the Storage Container Manager: <a href="http://localhost:9876">http://localhost:9876</a></li>
-<li>For the Ozone Managerr: <a href="http://localhost:9874">http://localhost:9874</a></li>
+<li>For the Storage Container Manager: http://localhost:9876</li>
+<li>For the Ozone Managerr: http://localhost:9874</li>
<li>For the Datanode: check the port with docker ps (as there could be multiple data node ports are mapped to the ephemeral port range)</li>
-<li>For the Namenode: <a href="http://localhost:9870">http://localhost:9870</a></li>
+<li>For the Namenode: http://localhost:9870</li>
</ul>
-
<p>You can start multiple data nodes with:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose scale datanode<span style="color:#f92672">=</span><span style="color:#ae81ff">3</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose scale datanode<span style="color:#f92672">=</span><span style="color:#ae81ff">3</span></span></span></code></pre></div>
<p>You can test the commands from the <a href="./commandshell.html#shell">Ozone CLI</a> after opening a new bash shell in one of the containers:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose exec datanode bash</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose exec datanode bash</span></span></code></pre></div>
</div>
</div>
diff --git a/docs/0.2.1-alpha/freon.html b/docs/0.2.1-alpha/freon.html
index 984d89e..51e1d8f 100644
--- a/docs/0.2.1-alpha/freon.html
+++ b/docs/0.2.1-alpha/freon.html
@@ -188,60 +188,34 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Freon</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="overview">Overview</h2>
-
<p>Freon is a load-generator for Ozone. This tool is used for testing the functionality of ozone.</p>
-
<h3 id="random-keys">Random keys</h3>
-
<p>In randomkeys mode, the data written into ozone cluster is randomly generated.
Each key will be of size 10 KB.</p>
-
<p>The number of volumes/buckets/keys can be configured. The replication type and
factor (eg. replicate with ratis to 3 nodes) Also can be configured.</p>
-
<p>For more information use</p>
-
<p><code>bin/ozone freon --help</code></p>
-
<h3 id="example">Example</h3>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span> --replicationType<span style="color:#f92672">=</span>RATIS --factor<span style="color:#f92672">=</span>THREE</code>< [...]
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">***************************************************
-Status: Success
-Git Base Revision: 48aae081e5afacbb3240657556b26c29e61830c3
-Number of Volumes created: <span style="color:#ae81ff">10</span>
-Number of Buckets created: <span style="color:#ae81ff">100</span>
-Number of Keys added: <span style="color:#ae81ff">1000</span>
-Ratis replication factor: THREE
-Ratis replication type: RATIS
-Average Time spent in volume creation: <span style="color:#ae81ff">00</span>:00:00,035
-Average Time spent in bucket creation: <span style="color:#ae81ff">00</span>:00:00,319
-Average Time spent in key creation: <span style="color:#ae81ff">00</span>:00:03,659
-Average Time spent in key write: <span style="color:#ae81ff">00</span>:00:10,894
-Total bytes written: <span style="color:#ae81ff">10240000</span>
-Total Execution time: <span style="color:#ae81ff">00</span>:00:16,898
-***********************</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span> --replicationType<span style="color:#f92672">=</span>RATIS --factor< [...]
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>***************************************************
+</span></span><span style="display:flex;"><span>Status: Success
+</span></span><span style="display:flex;"><span>Git Base Revision: 48aae081e5afacbb3240657556b26c29e61830c3
+</span></span><span style="display:flex;"><span>Number of Volumes created: <span style="color:#ae81ff">10</span>
+</span></span><span style="display:flex;"><span>Number of Buckets created: <span style="color:#ae81ff">100</span>
+</span></span><span style="display:flex;"><span>Number of Keys added: <span style="color:#ae81ff">1000</span>
+</span></span><span style="display:flex;"><span>Ratis replication factor: THREE
+</span></span><span style="display:flex;"><span>Ratis replication type: RATIS
+</span></span><span style="display:flex;"><span>Average Time spent in volume creation: 00:00:00,035
+</span></span><span style="display:flex;"><span>Average Time spent in bucket creation: 00:00:00,319
+</span></span><span style="display:flex;"><span>Average Time spent in key creation: 00:00:03,659
+</span></span><span style="display:flex;"><span>Average Time spent in key write: 00:00:10,894
+</span></span><span style="display:flex;"><span>Total bytes written: <span style="color:#ae81ff">10240000</span>
+</span></span><span style="display:flex;"><span>Total Execution time: 00:00:16,898
+</span></span><span style="display:flex;"><span>***********************</span></span></code></pre></div>
</div>
</div>
diff --git a/docs/0.2.1-alpha/hdds.html b/docs/0.2.1-alpha/hdds.html
index 3344f80..b12020d 100644
--- a/docs/0.2.1-alpha/hdds.html
+++ b/docs/0.2.1-alpha/hdds.html
@@ -188,38 +188,16 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Hadoop Distributed Data Store</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="scm-overview">SCM Overview</h2>
-
<p>Storage Container Manager or SCM is a very important component of ozone. SCM
offers block and container-based services to Ozone Manager. A container is a
collection of unrelated blocks under ozone. SCM and data nodes work together
to maintain the replication levels needed by the cluster.</p>
-
<p>It is easier to look at a putKey operation to understand the role that SCM plays.</p>
-
<p>To put a key, a client makes a call to KSM with the following arguments.</p>
-
<p>– putKey(keyName, data, pipeline type, replication count)</p>
-
<ol>
<li>keyName - refers to the file name.</li>
<li>data - The data that the client wants to write.</li>
@@ -228,21 +206,16 @@ refers to the replication strategy used for replicating a block. Ozone
currently supports Stand Alone and Ratis as two different pipeline types.</li>
<li>replication count - This specifies how many copies of the block replica should be maintained.</li>
</ol>
-
<p>In most cases, the client does not specify the pipeline type and replication
- count. The default pipeline type and replication count are used.</p>
-
+count. The default pipeline type and replication count are used.</p>
<p>Ozone Manager when it receives the putKey call, makes a call to SCM asking
for a pipeline instance with the specified property. So if the client asked
for RATIS replication strategy and a replication count of three, then OM
requests SCM to return a set of data nodes that meet this capability.</p>
-
<p>If SCM can find this a pipeline ( that is a set of data nodes) that can meet
the requirement from the client, then those nodes are returned to OM. OM will
persist this info and return a tuple consisting of {BlockID, ContainerName, and Pipeline}.</p>
-
<p>If SCM is not able to find a pipeline, then SCM creates a logical pipeline and then returns it.</p>
-
<p>SCM manages blocks, containers, and pipelines. To return healthy pipelines,
SCM also needs to understand the node health. So SCM listens to heartbeats
from data nodes and acts as the node manager too.</p>
diff --git a/docs/0.2.1-alpha/index.html b/docs/0.2.1-alpha/index.html
index a29b817..ca01e70 100644
--- a/docs/0.2.1-alpha/index.html
+++ b/docs/0.2.1-alpha/index.html
@@ -3,7 +3,7 @@
<!DOCTYPE html>
<html lang="en">
<head>
- <meta name="generator" content="Hugo 0.48" />
+ <meta name="generator" content="Hugo 0.108.0">
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
@@ -187,40 +187,19 @@
</div>
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h1 id="apache-hadoop-ozone">Apache Hadoop Ozone</h1>
-
<p>Ozone is a scalable, distributed object store for Hadoop. Applications like
Apache Spark, Hive and YARN, can run against Ozone without any
-modifications. Ozone comes with a <a href="./javaapi.html">Java client library</a> and a <a href="./commandshell.html#shell">command line interface</a> which makes it easy to use Ozone. This client library supports both RPC and REST protocols.</p>
-
+modifications. Ozone comes with a <a href="./javaapi.html">Java client library</a> and a [command line interface] (/commandshell.html#shell) which makes it easy to use Ozone. This client library supports both RPC and REST protocols.</p>
<p>Ozone consists of volumes, buckets, and Keys.</p>
-
<ul>
<li>Volumes are similar to user accounts. Only administrators can create or delete volumes.</li>
<li>Buckets are similar to directories. A bucket can contain any number of keys, but buckets cannot contain other buckets.</li>
<li>Keys are similar to files. A bucket can contain any number of keys.</li>
</ul>
-
-<p><a href="./runningviadocker.html"><button class="btn btn-danger btn-lg">Getting started</button></a></p>
+<p><!-- raw HTML omitted --><!-- raw HTML omitted -->Getting started<!-- raw HTML omitted --><!-- raw HTML omitted --></p>
diff --git a/docs/0.2.1-alpha/index.xml b/docs/0.2.1-alpha/index.xml
index 774ebbc..a4acc5e 100644
--- a/docs/0.2.1-alpha/index.xml
+++ b/docs/0.2.1-alpha/index.xml
@@ -1,16 +1,11 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Ozone Overview on Ozone</title>
<link>/</link>
<description>Recent content in Ozone Overview on Ozone</description>
<generator>Hugo -- gohugo.io</generator>
- <language>en-us</language>
- <lastBuildDate>Thu, 14 Sep 2017 00:00:00 +0000</lastBuildDate>
-
- <atom:link href="/index.xml" rel="self" type="application/rss+xml" />
-
-
+ <language>en-us</language><atom:link href="/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>Alpha Cluster</title>
<link>/runningviadocker.html</link>
@@ -40,7 +35,7 @@ If you want to build from sources, Please untar the source tarball and run the o
<guid>/settings.html</guid>
<description>If you are feeling adventurous, you can setup ozone in a real cluster. Setting up a real cluster requires us to understand the components of Ozone. Ozone is designed to work concurrently with HDFS. However, Ozone is also capable of running independently. The components of ozone are the same in both approaches.
-Ozone Components Ozone Manager - Is the server that is in charge of the namespace of Ozone.</description>
+Ozone Components Ozone Manager - Is the server that is in charge of the namespace of Ozone. Ozone Manager is responsible for all volume, bucket and key operations.</description>
</item>
<item>
@@ -162,9 +157,10 @@ SCMCLI allows the developer to access SCM directly. Please note: Improper usage
<guid>/bucketcommands.html</guid>
<description>Ozone shell supports the following bucket commands.
- create delete info list update Create The bucket create command allows a user to create a bucket.
+create delete info list update Create The bucket create command allows a user to create a bucket.
Params:
- Arguments Comment Uri The name of the bucket in /volume/bucket format. ozone sh bucket create /hive/jan The above command will create a bucket called jan in the hive volume. Since no scheme was specified this command defaults to O3 (RPC) protocol.</description>
+Arguments Comment Uri The name of the bucket in /volume/bucket format. ozone sh bucket create /hive/jan The above command will create a bucket called jan in the hive volume. Since no scheme was specified this command defaults to O3 (RPC) protocol.
+Delete The bucket delete commands allows an user to delete a volume.</description>
</item>
<item>
@@ -174,9 +170,10 @@ Params:
<guid>/keycommands.html</guid>
<description>Ozone shell supports the following key commands.
- get put delete info list Get The key get command downloads a key from Ozone cluster to local file system.
+get put delete info list Get The key get command downloads a key from Ozone cluster to local file system.
Params:
- Arguments Comment Uri The name of the key in /volume/bucket/key format. FileName Local file to download the key to. ozone sh key get /hive/jan/sales.orc sales.orc Downloads the file sales.</description>
+Arguments Comment Uri The name of the key in /volume/bucket/key format. FileName Local file to download the key to. ozone sh key get /hive/jan/sales.orc sales.orc Downloads the file sales.orc from the /hive/jan bucket and writes to the local file sales.orc.
+Put Uploads a file from the local file system to the specified bucket.</description>
</item>
<item>
@@ -188,7 +185,7 @@ Params:
<description>Ozone has a set of command line tools that can be used to manage ozone.
All these commands are invoked via the ozone script.
The commands supported by ozone are:
- classpath - Prints the class path needed to get the hadoop jar and the required libraries. fs - Runs a command on ozone file system. datanode - Via daemon command, the HDDS data nodes can be started or stopped.</description>
+classpath - Prints the class path needed to get the hadoop jar and the required libraries. fs - Runs a command on ozone file system. datanode - Via daemon command, the HDDS data nodes can be started or stopped. envvars - Display computed Hadoop environment variables.</description>
</item>
<item>
@@ -200,7 +197,7 @@ The commands supported by ozone are:
<description>The Ozone REST API&rsquo;s allows user to access ozone via REST protocol.
Authentication and Authorization For time being, The default authentication mode of REST API is insecure access mode, which is Simple mode. Under this mode, ozone server trusts the user name specified by client and it does not perform any authentication.
User name can be specified in HTTP header by
- x-ozone-user: {USER_NAME} for example if add following header x-ozone-user: bilbo in the HTTP request, then operation will be executed as bilbo user.</description>
+x-ozone-user: {USER_NAME} for example if add following header x-ozone-user: bilbo in the HTTP request, then operation will be executed as bilbo user.</description>
</item>
<item>
@@ -210,10 +207,10 @@ User name can be specified in HTTP header by
<guid>/volumecommands.html</guid>
<description>Volume commands generally need administrator privileges. The ozone shell supports the following volume commands.
- create delete info list update Create The volume create command allows an administrator to create a volume and assign it to a user.
+create delete info list update Create The volume create command allows an administrator to create a volume and assign it to a user.
Params:
- Arguments Comment -q, &ndash;quota Optional, This argument that specifies the maximum size this volume can use in the Ozone cluster. -u, &ndash;user Required, The name of the user who owns this volume.</description>
+Arguments Comment -q, &ndash;quota Optional, This argument that specifies the maximum size this volume can use in the Ozone cluster. -u, &ndash;user Required, The name of the user who owns this volume.</description>
</item>
</channel>
-</rss>
\ No newline at end of file
+</rss>
diff --git a/docs/0.2.1-alpha/javaapi.html b/docs/0.2.1-alpha/javaapi.html
index 242c6e0..a27d699 100644
--- a/docs/0.2.1-alpha/javaapi.html
+++ b/docs/0.2.1-alpha/javaapi.html
@@ -188,155 +188,110 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Java API</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="introduction">Introduction</h2>
-
<p>Ozone ships with it own client library, that supports both RPC(Remote
Procedure call) and REST(Representational State Transfer). This library is
the primary user interface to ozone.</p>
-
<p>It is trivial to switch from RPC to REST or vice versa, by setting the
property <em>ozone.client.protocol</em> in the configuration or by calling the
appropriate factory method.</p>
-
<h2 id="creating-an-ozone-client">Creating an Ozone client</h2>
-
<p>The Ozone client factory creates the ozone client. It allows the user to
specify the protocol of communication. For example, to get an REST client, we
can use</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getRestClient</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getRestClient</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>And to get a a RPC client we can call</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getRpcClient</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getRpcClient</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>If the user want to create a client based on the configuration, then they can
call</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>and an appropriate client based on configuration will be returned.</p>
-
<h2 id="writing-data-using-ozone-client">Writing data using Ozone Client</h2>
-
<p>The hierarchy of data inside ozone is a volume, bucket and a key. A volume
is a collection of buckets. A bucket is a collection of keys. To write data
to the ozone, you need a volume, bucket and a key.</p>
-
<h3 id="creating-a-volume">Creating a Volume</h3>
-
<p>Once we have a client, we need to get a reference to the ObjectStore. This
is done via</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>An object store represents an active cluster against which the client is working.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java"><span style="color:#75715e">// Let us create a volume to store our game assets.
-</span><span style="color:#75715e">// This uses default arguments for creating that volume.
-</span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Let us verify that the volume got created.
-</span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#75715e">// Let us create a volume to store our game assets.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e">// This uses default arguments for creating that volume.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us verify that the volume got created.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span></span></span></code></pre></div>
<p>It is possible to pass an array of arguments to the createVolume by creating volume arguments.</p>
-
<h3 id="creating-a-bucket">Creating a Bucket</h3>
-
<p>Once you have a volume, you can create buckets inside the volume.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">// Let us create a bucket called videos.
-assets.createBucket<span style="color:#f92672">(</span>“videos”<span style="color:#f92672">)</span>;
-Ozonebucket video <span style="color:#f92672">=</span> assets.getBucket<span style="color:#f92672">(</span>“videos”<span style="color:#f92672">)</span>;</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>// Let us create a bucket called videos.
+</span></span><span style="display:flex;"><span>assets.createBucket<span style="color:#f92672">(</span>“videos”<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>Ozonebucket video <span style="color:#f92672">=</span> assets.getBucket<span style="color:#f92672">(</span>“videos”<span style="color:#f92672">)</span>;</span></span></code></pre></div>
<p>At this point we have a usable volume and a bucket. Our volume is called assets and bucket is called videos.</p>
-
<p>Now we can create a Key.</p>
-
<h3 id="reading-and-writing-a-key">Reading and Writing a Key</h3>
-
<p>With a bucket object the users can now read and write keys. The following code reads a video called intro.mp4 from the local disk and stores in the video bucket that we just created.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">// read data from the file, this is a user provided <span style="color:#66d9ef">function</span>.
-byte <span style="color:#f92672">[]</span> vidoeData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span>“into.mp4”<span style="color:#f92672">)</span>;
-
-// Create an output stream and write data.
-OzoneOutputStream videoStream <span style="color:#f92672">=</span> video.createKey<span style="color:#f92672">(</span>“intro.mp4”, <span style="color:#ae81ff">1048576</span><span style="color:#f92672">)</span>;
-videoStream.write<span style="color:#f92672">(</span>videoData<span style="color:#f92672">)</span>;
-
-// Close the stream when it is <span style="color:#66d9ef">done</span>.
- videoStream.close<span style="color:#f92672">()</span>;
-
-
-// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
-// Let us allocate a byte array to hold the video first.
-byte<span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> new byte<span style="color:#f92672">[(</span>int<span style="color:#f92672">)</span><span style="color:#ae81ff">1048576</span><span style="color:#f92672">]</span>;
-OzoneInputStream introStream <span style="color:#f92672">=</span> video.readKey<span style="color:#f92672">(</span>“intro.mp4”<span style="color:#f92672">)</span>;
-// read intro.mp4 into the data buffer
-introStream.read<span style="color:#f92672">(</span>data<span style="color:#f92672">)</span>;
-introStream.close<span style="color:#f92672">()</span>;</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>// read data from the file, this is a user provided <span style="color:#66d9ef">function</span>.
+</span></span><span style="display:flex;"><span>byte <span style="color:#f92672">[]</span> vidoeData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span>“into.mp4”<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>// Create an output stream and write data.
+</span></span><span style="display:flex;"><span>OzoneOutputStream videoStream <span style="color:#f92672">=</span> video.createKey<span style="color:#f92672">(</span>“intro.mp4”, 1048576<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>videoStream.write<span style="color:#f92672">(</span>videoData<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>// Close the stream when it is <span style="color:#66d9ef">done</span>.
+</span></span><span style="display:flex;"><span> videoStream.close<span style="color:#f92672">()</span>;
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
+</span></span><span style="display:flex;"><span>// Let us allocate a byte array to hold the video first.
+</span></span><span style="display:flex;"><span>byte<span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> new byte<span style="color:#f92672">[(</span>int<span style="color:#f92672">)</span>1048576<span style="color:#f92672">]</span>;
+</span></span><span style="display:flex;"><span>OzoneInputStream introStream <span style="color:#f92672">=</span> video.readKey<span style="color:#f92672">(</span>“intro.mp4”<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>// read intro.mp4 into the data buffer
+</span></span><span style="display:flex;"><span>introStream.read<span style="color:#f92672">(</span>data<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>introStream.close<span style="color:#f92672">()</span>;</span></span></code></pre></div>
<p>Here is a complete example of the code that we just wrote. Please note the close functions being called in this program.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java"><span style="color:#75715e">// Let us create a client
-</span><span style="color:#75715e"></span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span>
-
-<span style="color:#75715e">// Get a reference to the ObjectStore using the client
-</span><span style="color:#75715e"></span>ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span>
-
-<span style="color:#75715e">// Let us create a volume to store our game assets.
-</span><span style="color:#75715e">// This default arguments for creating that volume.
-</span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Let us verify that the volume got created.
-</span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Let us create a bucket called videos.
-</span><span style="color:#75715e"></span>assets<span style="color:#f92672">.</span><span style="color:#a6e22e">createBucket</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>videos<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-Ozonebucket video <span style="color:#f92672">=</span> assets<span style="color:#f92672">.</span><span style="color:#a6e22e">getBucket</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>videos<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// read data from the file, this is assumed to be a user provided function.
-</span><span style="color:#75715e"></span><span style="color:#66d9ef">byte</span> <span style="color:#f92672">[]</span> vidoeData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>into<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Create an output stream and write data.
-</span><span style="color:#75715e"></span>OzoneOutputStream videoStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">createKey</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>intro<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">,</span> 1048576<span style="colo [...]
-videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">write</span><span style="color:#f92672">(</span>videoData<span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Close the stream when it is done.
-</span><span style="color:#75715e"></span> videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
-
-
-<span style="color:#75715e">// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
-</span><span style="color:#75715e">// Let us allocate a byte array to hold the video first.
-</span><span style="color:#75715e"></span>
-<span style="color:#66d9ef">byte</span><span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> <span style="color:#66d9ef">new</span> <span style="color:#66d9ef">byte</span><span style="color:#f92672">[(</span><span style="color:#66d9ef">int</span><span style="color:#f92672">)</span>1048576<span style="color:#f92672">];</span>
-OzoneInputStream introStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">readKey</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>into<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">read</span><span style="color:#f92672">(</span>data<span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Close the stream when it is done.
-</span><span style="color:#75715e"></span>introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
-
-<span style="color:#75715e">// Close the client.
-</span><span style="color:#75715e"></span>ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span></code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#75715e">// Let us create a client
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Get a reference to the ObjectStore using the client
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us create a volume to store our game assets.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e">// This default arguments for creating that volume.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us verify that the volume got created.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us create a bucket called videos.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>assets<span style="color:#f92672">.</span><span style="color:#a6e22e">createBucket</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>videos<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>Ozonebucket video <span style="color:#f92672">=</span> assets<span style="color:#f92672">.</span><span style="color:#a6e22e">getBucket</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>videos<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// read data from the file, this is assumed to be a user provided function.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">byte</span> <span style="color:#f92672">[]</span> vidoeData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>into<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Create an output stream and write data.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneOutputStream videoStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">createKey</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>intro<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style=" [...]
+</span></span><span style="display:flex;"><span>videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">write</span><span style="color:#f92672">(</span>videoData<span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Close the stream when it is done.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us allocate a byte array to hold the video first.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>
+</span></span><span style="display:flex;"><span><span style="color:#66d9ef">byte</span><span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> <span style="color:#66d9ef">new</span> <span style="color:#66d9ef">byte</span><span style="color:#f92672">[(</span><span style="color:#66d9ef">int</span><span style="color:#f92672">)</span><span style="color:#ae81ff">1048576</span><span style="color:#f92672">];</span>
+</span></span><span style="display:flex;"><span>OzoneInputStream introStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">readKey</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>into<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">read</span><span style="color:#f92672">(</span>data<span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Close the stream when it is done.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Close the client.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span></span></span></code></pre></div>
</div>
</div>
diff --git a/docs/0.2.1-alpha/keycommands.html b/docs/0.2.1-alpha/keycommands.html
index 59a1224..11537f5 100644
--- a/docs/0.2.1-alpha/keycommands.html
+++ b/docs/0.2.1-alpha/keycommands.html
@@ -188,27 +188,9 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Key Commands</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone shell supports the following key commands.</p>
-
<ul>
<li><a href="#get">get</a></li>
<li><a href="#put">put</a></li>
@@ -216,13 +198,9 @@
<li><a href="#info">info</a></li>
<li><a href="#list">list</a></li>
</ul>
-
<h3 id="get">Get</h3>
-
<p>The key get command downloads a key from Ozone cluster to local file system.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -230,30 +208,23 @@
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
<td>The name of the key in <strong>/volume/bucket/key</strong> format.</td>
</tr>
-
<tr>
<td>FileName</td>
<td>Local file to download the key to.</td>
</tr>
</tbody>
</table>
-
-<p><div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key get /hive/jan/sales.orc sales.orc</code></pre></div>
+<p><div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key get /hive/jan/sales.orc sales.orc</span></span></code></pre></div>
Downloads the file sales.orc from the <em>/hive/jan</em> bucket and writes to the
local file sales.orc.</p>
-
<h3 id="put">Put</h3>
-
<p>Uploads a file from the local file system to the specified bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -261,34 +232,26 @@ local file sales.orc.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
<td>The name of the key in <strong>/volume/bucket/key</strong> format.</td>
</tr>
-
<tr>
<td>FileName</td>
<td>Local file to upload.</td>
</tr>
-
<tr>
<td>-r, –replication</td>
<td>Optional, Number of copies, ONE or THREE are the options. Picks up the default from cluster configuration.</td>
</tr>
</tbody>
</table>
-
-<p><div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key put /hive/jan/corrected-sales.orc sales.orc</code></pre></div>
+<p><div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key put /hive/jan/corrected-sales.orc sales.orc</span></span></code></pre></div>
The above command will put the sales.orc as a new key into <em>/hive/jan/corrected-sales.orc</em>.</p>
-
<h3 id="delete">Delete</h3>
-
<p>The delete key command removes the key from the bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -296,7 +259,6 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -304,16 +266,11 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key delete /hive/jan/corrected-sales.orc</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key delete /hive/jan/corrected-sales.orc</span></span></code></pre></div>
<p>The above command deletes the key <em>/hive/jan/corrected-sales.orc</em>.</p>
-
<h3 id="info">Info</h3>
-
<p>The key info commands returns the information about the key.
-<strong><em>Params:</em></strong></p>
-
+<em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -321,7 +278,6 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -329,18 +285,12 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key info /hive/jan/sales.orc</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key info /hive/jan/sales.orc</span></span></code></pre></div>
<p>The above command will print out the information about <em>/hive/jan/sales.orc</em>
key.</p>
-
<h3 id="list">List</h3>
-
<p>The key list commands allows user to list all keys in a bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -348,34 +298,27 @@ key.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>-l, –length</td>
<td>Maximum number of results to return. Default: 1000</td>
</tr>
-
<tr>
<td>-p, –prefix</td>
<td>Optional, Only buckets that match this prefix will be returned.</td>
</tr>
-
<tr>
<td>-s, –start</td>
<td>The listing will start from key after the start key.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the <em>volume</em>.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key list /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key list /hive/jan</span></span></code></pre></div>
<p>This command will list all key in the bucket <em>/hive/jan</em>.</p>
-
<p>You can try out these commands from the docker instance of the <a href="runningviadocker.html">Alpha
Cluster</a>.</p>
diff --git a/docs/0.2.1-alpha/ozonefs.html b/docs/0.2.1-alpha/ozonefs.html
index 740065f..8c5b052 100644
--- a/docs/0.2.1-alpha/ozonefs.html
+++ b/docs/0.2.1-alpha/ozonefs.html
@@ -188,64 +188,32 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Ozone File System</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>There are many Hadoop compatible files systems under Hadoop. Hadoop compatible file systems ensures that storage backends like Ozone can easily be integrated into Hadoop eco-system.</p>
-
<h2 id="setting-up-the-ozone-file-system">Setting up the Ozone file system</h2>
-
<p>To create an ozone file system, we have to choose a bucket where the file system would live. This bucket will be used as the backend store for OzoneFileSystem. All the files and directories will be stored as keys in this bucket.</p>
-
<p>Please run the following commands to create a volume and bucket, if you don’t have them already.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume create /volume
-ozone sh bucket create /volume/bucket</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume create /volume
+</span></span><span style="display:flex;"><span>ozone sh bucket create /volume/bucket</span></span></code></pre></div>
<p>Once this is created, please make sure that bucket exists via the listVolume or listBucket commands.</p>
-
<p>Please add the following entry to the core-site.xml.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>fs.o3.impl<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>org.apache.hadoop.fs.ozone.OzoneFileSystem<span style="color:#f92672"></value></span>
-<span style="color:#f92672"></property></span>
-<span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>fs.default.name<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>o3://localhost:9864/volume/bucket<span style="color:#f92672"></value></span>
-<span style="color:#f92672"></property></span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span><span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>fs.o3.impl<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>org.apache.hadoop.fs.ozone.OzoneFileSystem<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"></property></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>fs.default.name<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>o3://localhost:9864/volume/bucket<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"></property></span></span></span></code></pre></div>
<p>This will make this bucket to be the default file system for HDFS dfs commands and register the o3 file system type..</p>
-
<p>You also need to add the ozone-filesystem.jar file to the classpath:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">export HADOOP_CLASSPATH<span style="color:#f92672">=</span>/opt/ozone/share/hadoop/ozonefs/hadoop-ozone-filesystem.jar:$HADOOP_CLASSPATH</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>export HADOOP_CLASSPATH<span style="color:#f92672">=</span>/opt/ozone/share/hadoop/ozonefs/hadoop-ozone-filesystem.jar:$HADOOP_CLASSPATH</span></span></code></pre></div>
<p>Once the default Filesystem has been setup, users can run commands like ls, put, mkdir, etc.
For example,</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">hdfs dfs -ls /</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>hdfs dfs -ls /</span></span></code></pre></div>
<p>or</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">hdfs dfs -mkdir /users</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>hdfs dfs -mkdir /users</span></span></code></pre></div>
<p>Or put command etc. In other words, all programs like Hive, Spark, and Distcp will work against this file system.
Please note that any keys created/deleted in the bucket using methods apart from OzoneFileSystem will show up as diectories and files in the Ozone File System.</p>
diff --git a/docs/0.2.1-alpha/ozonemanager.html b/docs/0.2.1-alpha/ozonemanager.html
index c04f7ea..8d64c93 100644
--- a/docs/0.2.1-alpha/ozonemanager.html
+++ b/docs/0.2.1-alpha/ozonemanager.html
@@ -188,72 +188,33 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Ozone Manager</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="om-overview">OM Overview</h2>
-
<p>Ozone Manager or OM is the namespace manager for Ozone. The clients (RPC clients, Rest proxy, Ozone file system, etc.) communicate with OM to create and delete various ozone objects.</p>
-
<p>Each ozone volume is the root of a namespace under OM. This is very different from HDFS which provides a single rooted file system.</p>
-
<p>Ozone’s namespace is a collection of volumes or is a forest instead of a
single rooted tree as in HDFS. This property makes it easy to deploy multiple
- OMs for scaling, this feature is under development.</p>
-
+OMs for scaling, this feature is under development.</p>
<h2 id="om-metadata">OM Metadata</h2>
-
<p>Conceptually, OM maintains a list of volumes, buckets, and keys. For each user, it maintains a list of volumes. For each volume, the list of buckets and for each bucket the list of keys.</p>
-
<p>Right now, OM is a single instance service. Ozone already relies on Apache Ratis (A Replicated State Machine based on Raft protocol). OM will be extended to replicate all its metadata via Ratis. With that, OM will be highly available.</p>
-
<h2 id="om-ui">OM UI</h2>
-
-<p>OM supports a simple UI for the time being. The default port of OM is 9874. To access the OM UI, the user can connect to <a href="http://OM:port">http://OM:port</a> or for a concrete example,</p>
-
-<pre><code>http://omserver:9874/
-</code></pre>
-
-<p>OM UI primarily tries to measure load and latency of OM. The first section of OM UI relates to the number of operations seen by the cluster broken down by the object, operation and whether the operation was successful.</p>
-
+<p>OM supports a simple UI for the time being. The default port of OM is 9874. To access the OM UI, the user can connect to http://OM:port or for a concrete example,</p>
+<pre tabindex="0"><code>http://omserver:9874/
+</code></pre><p>OM UI primarily tries to measure load and latency of OM. The first section of OM UI relates to the number of operations seen by the cluster broken down by the object, operation and whether the operation was successful.</p>
<p>The latter part of the UI is focused on latency and number of operations that OM is performing.</p>
-
<p>One of the hardest problems in HDFS world is discovering the numerous settings offered to tune HDFS. Ozone solves that problem by tagging the configs. To discover settings, click on “Common Tools”->Config. This will take you to the ozone config UI.</p>
-
<h2 id="config-ui">Config UI</h2>
-
<p>The ozone config UI is a matrix with row representing the tags, and columns representing All, OM and SCM.</p>
-
<p>Suppose a user wanted to discover the required settings for ozone. Then the user can tick the checkbox that says “Required.”
This will filter out all “Required” settings along with the description of what each setting does.</p>
-
<p>The user can combine different checkboxes and UI will combine the results. That is, If you have more than one row selected, then all keys for those chosen tags are displayed together.</p>
-
<p>We are hopeful that this leads to a more straightforward way of discovering settings that manage ozone.</p>
-
<h2 id="om-and-scm">OM and SCM</h2>
-
<p><a href="./hdds.html">Storage container manager</a> or (SCM) is the block manager
- for ozone. When a client requests OM for a set of data nodes to write data, OM talk to SCM and gets a block.</p>
-
+for ozone. When a client requests OM for a set of data nodes to write data, OM talk to SCM and gets a block.</p>
<p>A block returned by SCM contains a pipeline, which is a set of nodes that we participate in that block replication.</p>
-
<p>So OM is dependent on SCM for reading and writing of Keys. However, OM is independent of SCM while doing metadata operations like ozone volume or bucket operations.</p>
</div>
diff --git a/docs/0.2.1-alpha/realcluster.html b/docs/0.2.1-alpha/realcluster.html
index a1dbcfd..77d274b 100644
--- a/docs/0.2.1-alpha/realcluster.html
+++ b/docs/0.2.1-alpha/realcluster.html
@@ -188,56 +188,30 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Starting an Ozone Cluster</h1>
<div class="col-md-9">
- <!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Before we boot up the Ozone cluster, we need to initialize both SCM and Ozone Manager.</p>
-
-<p><div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone scm -init</code></pre></div>
+<p><div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone scm -init</span></span></code></pre></div>
This allows SCM to create the cluster Identity and initialize its state.
The <code>init</code> command is similar to Namenode format. Init command is executed only once, that allows SCM to create all the required on-disk structures to work correctly.
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start scm</code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone --daemon start scm</span></span></code></pre></div></p>
<p>Once we know SCM is up and running, we can create an Object Store for our use. This is done by running the following command.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone om -createObjectStore</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone om -createObjectStore</span></span></code></pre></div>
<p>Once Ozone manager has created the Object Store, we are ready to run the name
services.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start om</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone --daemon start om</span></span></code></pre></div>
<p>At this point Ozone’s name services, the Ozone manager, and the block service SCM is both running.
<strong>Please note</strong>: If SCM is not running
<code>createObjectStore</code> command will fail. SCM start will fail if on-disk data structures are missing. So please make sure you have done both <code>init</code> and <code>createObjectStore</code> commands.</p>
-
<p>Now we need to start the data nodes. Please run the following command on each datanode.
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start datanode</code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone --daemon start datanode</span></span></code></pre></div></p>
<p>At this point SCM, Ozone Manager and data nodes are up and running.</p>
-
-<p><strong><em>Congratulations!, You have set up a functional ozone cluster.</em></strong></p>
-
-<hr />
-
+<p><em><strong>Congratulations!, You have set up a functional ozone cluster.</strong></em></p>
+<hr>
<p>If you want to make your life simpler, you can just run
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone scm -init
-ozone om -createObjectStore
-start-ozone.sh</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone scm -init
+</span></span><span style="display:flex;"><span>ozone om -createObjectStore
+</span></span><span style="display:flex;"><span>start-ozone.sh</span></span></code></pre></div>
This assumes that you have set up the slaves file correctly and ssh
configuration that allows ssh-ing to all data nodes. This is the same as the
HDFS configuration, so please refer to HDFS documentation on how to set this
diff --git a/docs/0.2.1-alpha/rest.html b/docs/0.2.1-alpha/rest.html
index 601d206..3290ecb 100644
--- a/docs/0.2.1-alpha/rest.html
+++ b/docs/0.2.1-alpha/rest.html
@@ -188,150 +188,102 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>REST API</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>The Ozone REST API’s allows user to access ozone via REST protocol.</p>
-
<h2 id="authentication-and-authorization">Authentication and Authorization</h2>
-
<p>For time being, The default authentication mode of REST API is insecure access
mode, which is <em>Simple</em> mode. Under this mode, ozone server trusts the user
name specified by client and it does not perform any authentication.</p>
-
<p>User name can be specified in HTTP header by</p>
-
<ul>
<li><code>x-ozone-user: {USER_NAME}</code></li>
</ul>
-
<p>for example if add following header <em>x-ozone-user: bilbo</em> in the HTTP request,
then operation will be executed as <em>bilbo</em> user.
In <em>Simple</em> mode, there is no real authorization either. Client can be
authorized to obtain administrator privilege by using HTTP header</p>
-
<ul>
<li><code>Authorization: {AUTH_METHOD} {SIGNATURE}</code></li>
</ul>
-
<p>for example set following header <em>Authorization: OZONE root</em> in the HTTP request,
then ozone will authorize the client with administrator privilege.</p>
-
<h2 id="common-rest-headers">Common REST Headers</h2>
-
<p>The following HTTP headers must be set for each REST call.</p>
-
<table>
<thead>
<tr>
-<th align="left">Property</th>
-<th align="left">Description</th>
+<th style="text-align:left">Property</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">Authorization</td>
-<td align="left">The authorization field determines which authentication method is used by ozone. Currently only <em>simple</em> mode is supported, the corresponding value is <em>OZONE</em>. Optionally an user name can be set as <em>OZONE {USER_NAME}</em> to authorize as a particular user.</td>
+<td style="text-align:left">Authorization</td>
+<td style="text-align:left">The authorization field determines which authentication method is used by ozone. Currently only <em>simple</em> mode is supported, the corresponding value is <em>OZONE</em>. Optionally an user name can be set as <em>OZONE {USER_NAME}</em> to authorize as a particular user.</td>
</tr>
-
<tr>
-<td align="left">Date</td>
-<td align="left">Standard HTTP header that represents dates. The format is - day of the week, month, day, year and time (military time format) in GMT. Any other time zone will be rejected by ozone server. Eg. <em>Date : Mon, Apr 4, 2016 06:22:00 GMT</em>. This field is required.</td>
+<td style="text-align:left">Date</td>
+<td style="text-align:left">Standard HTTP header that represents dates. The format is - day of the week, month, day, year and time (military time format) in GMT. Any other time zone will be rejected by ozone server. Eg. <em>Date : Mon, Apr 4, 2016 06:22:00 GMT</em>. This field is required.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-version</td>
-<td align="left">A required HTTP header to indicate which version of API this call will be communicating to. E.g <em>x-ozone-version: v1</em>. Currently ozone only publishes v1 version API.</td>
+<td style="text-align:left">x-ozone-version</td>
+<td style="text-align:left">A required HTTP header to indicate which version of API this call will be communicating to. E.g <em>x-ozone-version: v1</em>. Currently ozone only publishes v1 version API.</td>
</tr>
</tbody>
</table>
-
<h2 id="common-reply-headers">Common Reply Headers</h2>
-
<p>The common reply headers are part of all Ozone server replies.</p>
-
<table>
<thead>
<tr>
-<th align="left">Property</th>
-<th align="left">Description</th>
+<th style="text-align:left">Property</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">Date</td>
-<td align="left">This is the HTTP date header and it is set to server’s local time expressed in GMT.</td>
+<td style="text-align:left">Date</td>
+<td style="text-align:left">This is the HTTP date header and it is set to server’s local time expressed in GMT.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-request-id</td>
-<td align="left">This is a UUID string that represents an unique request ID. This ID is used to track the request through the ozone system and is useful for debugging purposes.</td>
+<td style="text-align:left">x-ozone-request-id</td>
+<td style="text-align:left">This is a UUID string that represents an unique request ID. This ID is used to track the request through the ozone system and is useful for debugging purposes.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-server-name</td>
-<td align="left">Fully qualified domain name of the sever which handled the request.</td>
+<td style="text-align:left">x-ozone-server-name</td>
+<td style="text-align:left">Fully qualified domain name of the sever which handled the request.</td>
</tr>
</tbody>
</table>
-
<h2 id="volume-apis">Volume APIs</h2>
-
<h3 id="create-a-volume">Create a Volume</h3>
-
<p>This API allows admins to create a new storage volume.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>POST /{volume}?quota=<VOLUME_QUOTA></code></li>
</ul>
-
<p>Query Parameter:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">quota</td>
-<td align="left">long<BYTES \| MB \| GB \| TB></td>
-<td align="left">Optional. Quota size in BYTEs, MBs, GBs or TBs</td>
+<td style="text-align:left">quota</td>
+<td style="text-align:left">long<BYTES | MB | GB | TB></td>
+<td style="text-align:left">Optional. Quota size in BYTEs, MBs, GBs or TBs</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP POST request:</p>
-
<pre><code>curl -i -X POST -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE root" "http://localhost:9880/volume-to-create"
</code></pre>
-
<p>this request creates a volume as user <em>bilbo</em>, the authorization field is set to <em>OZONE root</em> because this call requires administration privilege. The client receives a response with zero content length.</p>
-
<pre><code>HTTP/1.1 201 Created
x-ozone-server-name: localhost
x-ozone-request-id: 2173deb5-bbb7-4f0a-8236-f354784e3bae
@@ -340,61 +292,43 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="update-volume">Update Volume</h3>
-
<p>This API allows administrators to update volume info such as ownership and quota. This API requires administration privilege.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>PUT /{volume}?quota=<VOLUME_QUOTA></code></li>
</ul>
-
<p>Query Parameter:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">quota</td>
-<td align="left">long<BYTES \| MB \| GB \| TB> | remove</td>
-<td align="left">Optional. Quota size in BYTEs, MBs, GBs or TBs. Or use string value <em>remove</em> to remove an existing quota for a volume.</td>
+<td style="text-align:left">quota</td>
+<td style="text-align:left">long<BYTES | MB | GB | TB> | remove</td>
+<td style="text-align:left">Optional. Quota size in BYTEs, MBs, GBs or TBs. Or use string value <em>remove</em> to remove an existing quota for a volume.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP PUT request:</p>
-
<pre><code>curl -X PUT -H "Authorization:OZONE root" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "x-ozone-version: v1" -H "x-ozone-user: john" http://localhost:9880/volume-to-update
</code></pre>
-
<p>this request modifies the owner of <em>/volume-to-update</em> to <em>john</em>.</p>
-
<h3 id="delete-volume">Delete Volume</h3>
-
<p>This API allows user to delete a volume owned by themselves if the volume is not empty. Administrators can delete volumes owned by any user.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>DELETE /{volume}</code></li>
</ul>
-
<p>Sample HTTP DELETE request:</p>
-
<pre><code>curl -i -X DELETE -H "Authorization:OZONE root" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "x-ozone-version: v1" -H "x-ozone-user: bilbo" http://localhost:9880/volume-to-delete
</code></pre>
-
<p>this request deletes an empty volume <em>/volume-to-delete</em>. The client receives a zero length content.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: 6af14c64-e3a9-40fe-9634-df60b7cbbc6a
@@ -403,44 +337,33 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="info-volume">Info Volume</h3>
-
<p>This API allows user to read the info of a volume owned by themselves. Administrators can read volume info owned by any user.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}?info=volume</code></li>
</ul>
-
<p>Query Parameter:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">info</td>
-<td align="left">“volume”</td>
-<td align="left">Required and enforced with this value.</td>
+<td style="text-align:left">info</td>
+<td style="text-align:left">“volume”</td>
+<td style="text-align:left">Required and enforced with this value.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo?info=volume"
</code></pre>
-
<p>this request gets the info of volume <em>/volume-of-bilbo</em>, the client receives a response with a JSON object of volume info.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: a2224806-beaf-42dd-a68e-533cd7508f74
@@ -457,62 +380,48 @@ Connection: keep-alive
"createdBy" : "root"
}
</code></pre>
-
<h3 id="list-volumes">List Volumes</h3>
-
<p>This API allows user to list all volumes owned by themselves. Administrators can list all volumes owned by any user.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /?prefix=<PREFIX>&max-keys=<MAX_RESULT_SIZE>&prev-key=<PREVIOUS_VOLUME_KEY></code></li>
</ul>
-
<p>Query Parameter:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">prefix</td>
-<td align="left">string</td>
-<td align="left">Optional. Only volumes with this prefix are included in the result.</td>
+<td style="text-align:left">prefix</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Only volumes with this prefix are included in the result.</td>
</tr>
-
<tr>
-<td align="left">max-keys</td>
-<td align="left">int</td>
-<td align="left">Optional. Maximum number of volumes included in the result. Default is 1024 if not specified.</td>
+<td style="text-align:left">max-keys</td>
+<td style="text-align:left">int</td>
+<td style="text-align:left">Optional. Maximum number of volumes included in the result. Default is 1024 if not specified.</td>
</tr>
-
<tr>
-<td align="left">prev-key</td>
-<td align="left">string</td>
-<td align="left">Optional. Volume name from where listing should start, this key is excluded in the result. It must be a valid volume name.</td>
+<td style="text-align:left">prev-key</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Volume name from where listing should start, this key is excluded in the result. It must be a valid volume name.</td>
</tr>
-
<tr>
-<td align="left">root-scan</td>
-<td align="left">bool</td>
-<td align="left">Optional. List all volumes in the cluster if this is set to true. Default false.</td>
+<td style="text-align:left">root-scan</td>
+<td style="text-align:left">bool</td>
+<td style="text-align:left">Optional. List all volumes in the cluster if this is set to true. Default false.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/?max-keys=100&prefix=Jan"
</code></pre>
-
<p>this request gets all volumes owned by <em>bilbo</em> and each volume’s name contains prefix <em>Jan</em>, the result at most contains <em>100</em> entries. The client receives a list of SON objects, each of them describes the info of a volume.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: 7fa0dce1-a8bd-4387-bc3c-1dac4b710bb1
@@ -534,58 +443,44 @@ Connection: keep-alive
]
}
</code></pre>
-
<h2 id="bucket-apis">Bucket APIs</h2>
-
<h3 id="create-bucket">Create Bucket</h3>
-
<p>This API allows an user to create a bucket in a volume.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>POST /{volume}/{bucket}</code></li>
</ul>
-
<p>Additional HTTP Headers:</p>
-
<table>
<thead>
<tr>
-<th align="left">HTTP Header</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">HTTP Header</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">x-ozone-acl</td>
-<td align="left">ozone ACLs</td>
-<td align="left">Optional. Ozone acls.</td>
+<td style="text-align:left">x-ozone-acl</td>
+<td style="text-align:left">ozone ACLs</td>
+<td style="text-align:left">Optional. Ozone acls.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-storage-class</td>
-<td align="left"><DEFAULT \| ARCHIVE \| DISK \| RAM_DISK \| SSD ></td>
-<td align="left">Optional. Storage type for a volume.</td>
+<td style="text-align:left">x-ozone-storage-class</td>
+<td style="text-align:left"><DEFAULT | ARCHIVE | DISK | RAM_DISK | SSD ></td>
+<td style="text-align:left">Optional. Storage type for a volume.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-bucket-versioning</td>
-<td align="left">enabled/disabled</td>
-<td align="left">Optional. Do enable bucket versioning or not.</td>
+<td style="text-align:left">x-ozone-bucket-versioning</td>
+<td style="text-align:left">enabled/disabled</td>
+<td style="text-align:left">Optional. Do enable bucket versioning or not.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP POST request:</p>
-
<pre><code>curl -i -X POST -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" http://localhost:9880/volume-of-bilbo/bucket-0
</code></pre>
-
<p>this request creates a bucket <em>bucket-0</em> under volume <em>volume-of-bilbo</em>.</p>
-
<pre><code>HTTP/1.1 201 Created
x-ozone-server-name: localhost
x-ozone-request-id: 49acfeec-4c85-470a-872b-2eaebd8d751e
@@ -594,50 +489,38 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="update-bucket">Update Bucket</h3>
-
<p>Updates bucket meta-data, like ACLs.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>PUT /{volume}/{bucket}</code></li>
</ul>
-
<p>Additional HTTP Headers:</p>
-
<table>
<thead>
<tr>
-<th align="left">HTTP Header</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">HTTP Header</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">x-ozone-acl</td>
-<td align="left">ozone ACLs</td>
-<td align="left">Optional. Ozone acls.</td>
+<td style="text-align:left">x-ozone-acl</td>
+<td style="text-align:left">ozone ACLs</td>
+<td style="text-align:left">Optional. Ozone acls.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-bucket-versioning</td>
-<td align="left">enabled/disabled</td>
-<td align="left">Optional. Do enable bucket versioning or not.</td>
+<td style="text-align:left">x-ozone-bucket-versioning</td>
+<td style="text-align:left">enabled/disabled</td>
+<td style="text-align:left">Optional. Do enable bucket versioning or not.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP PUT request:</p>
-
<pre><code>curl -i -X PUT -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" -H "x-ozone-acl: ADD user:peregrin:rw" http://localhost:9880/volume-of-bilbo/bucket-to-update
</code></pre>
-
<p>this request adds an ACL policy specified by HTTP header <em>x-ozone-acl</em> to bucket <em>/volume-of-bilbo/bucket-to-update</em>, the ACL field <em>ADD user:peregrin:rw</em> gives add additional read/write permission to user <em>peregrin</em> to this bucket.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: b061a295-5faf-4b98-94b9-8b3e87c8eb5e
@@ -646,24 +529,16 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="delete-bucket">Delete Bucket</h3>
-
<p>Deletes a bucket if it is empty. An user can only delete bucket owned by themselves, and administrators can delete buckets owned by any user, as long as it is empty.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>DELETE /{volume}/{bucket}</code></li>
</ul>
-
<p>Sample HTTP DELETE request:</p>
-
<pre><code>curl -i -X DELETE -H "Authorization:OZONE root" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "x-ozone-version: v1" -H "x-ozone-user:bilbo" "http://localhost:9880/volume-of-bilbo/bucket-0"
</code></pre>
-
<p>this request deletes bucket <em>/volume-of-bilbo/bucket-0</em>. The client receives a zero length content response.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: f57acd7a-2116-4c2f-aa2f-5a483db81c9c
@@ -672,44 +547,33 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="info-bucket">Info Bucket</h3>
-
<p>This API returns information about a given bucket.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}/{bucket}?info=bucket</code></li>
</ul>
-
<p>Query Parameters:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">info</td>
-<td align="left">“bucket”</td>
-<td align="left">Required and enforced with this value.</td>
+<td style="text-align:left">info</td>
+<td style="text-align:left">“bucket”</td>
+<td style="text-align:left">Required and enforced with this value.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo/bucket-0?info=bucket"
</code></pre>
-
<p>this request gets the info of bucket <em>/volume-of-bilbo/bucket-0</em>. The client receives a response of JSON object contains bucket info.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: f125485b-8cae-4c7f-a2d6-5b1fefd6f193
@@ -727,56 +591,43 @@ Connection: keep-alive
"storageType" : "DISK"
}
</code></pre>
-
<h3 id="list-buckets">List Buckets</h3>
-
<p>List buckets in a given volume.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}?prefix=<PREFIX>&max-keys=<MAX_RESULT_SIZE>&prev-key=<PREVIOUS_BUCKET_KEY></code></li>
</ul>
-
<p>Query Parameters:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">prefix</td>
-<td align="left">string</td>
-<td align="left">Optional. Only buckets with this prefix are included in the result.</td>
+<td style="text-align:left">prefix</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Only buckets with this prefix are included in the result.</td>
</tr>
-
<tr>
-<td align="left">max-keys</td>
-<td align="left">int</td>
-<td align="left">Optional. Maximum number of buckets included in the result. Default is 1024 if not specified.</td>
+<td style="text-align:left">max-keys</td>
+<td style="text-align:left">int</td>
+<td style="text-align:left">Optional. Maximum number of buckets included in the result. Default is 1024 if not specified.</td>
</tr>
-
<tr>
-<td align="left">prev-key</td>
-<td align="left">string</td>
-<td align="left">Optional. Bucket name from where listing should start, this key is excluded in the result. It must be a valid bucket name.</td>
+<td style="text-align:left">prev-key</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Bucket name from where listing should start, this key is excluded in the result. It must be a valid bucket name.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo?max-keys=10"
</code></pre>
-
<p>this request lists all the buckets under volume <em>volume-of-bilbo</em>, and the result at most contains 10 entries. The client receives response of a array of JSON objects, each of them represents for a bucket info.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: e048c3d5-169c-470f-9903-632d9f9e32d5
@@ -800,63 +651,44 @@ Connection: keep-alive
]
}
</code></pre>
-
<h2 id="key-apis">Key APIs</h2>
-
<h3 id="put-key">Put Key</h3>
-
<p>This API allows user to create or overwrite keys inside of a bucket.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>PUT /{volume}/{bucket}/{key}</code></li>
</ul>
-
<p>Additional HTTP headers:</p>
-
<table>
<thead>
<tr>
-<th align="left">HTTP Header</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">HTTP Header</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">Content-MD5</td>
-<td align="left">MD5 digest</td>
-<td align="left">Standard HTTP header, file hash.</td>
+<td style="text-align:left">Content-MD5</td>
+<td style="text-align:left">MD5 digest</td>
+<td style="text-align:left">Standard HTTP header, file hash.</td>
</tr>
</tbody>
</table>
-
<p>Sample PUT HTTP request:</p>
-
<pre><code>curl -X PUT -T /path/to/localfile -H "Authorization:OZONE" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "x-ozone-version: v1" -H "x-ozone-user:bilbo" "http://localhost:9880/volume-of-bilbo/bucket-0/file-0"
</code></pre>
-
<p>this request uploads a local file <em>/path/to/localfile</em> specified by option <em>-T</em> to ozone as user <em>bilbo</em>, mapped to ozone key <em>/volume-of-bilbo/bucket-0/file-0</em>. The client receives a zero length content response.</p>
-
<h3 id="get-key">Get Key</h3>
-
<p>This API allows user to get or download a key from an ozone bucket.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}/{bucket}/{key}</code></li>
</ul>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo/bucket-0/file-0"
</code></pre>
-
<p>this request reads the content of key <em>/volume-of-bilbo/bucket-0/file-0</em>. If the content of the file is plain text, it can be directly dumped onto stdout.</p>
-
<pre><code>HTTP/1.1 200 OK
Content-Type: application/octet-stream
x-ozone-server-name: localhost
@@ -867,36 +699,24 @@ Connection: keep-alive
Hello Ozone!
</code></pre>
-
<p>if the file is not plain text, specify <em>-O</em> option in curl command and the file <em>file-0</em> will be downloaded into current working directory, file name will be same as the key. A sample request like following:</p>
-
<pre><code>curl -O -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo/bucket-0/file-1"
</code></pre>
-
<p>response looks like following:</p>
-
<pre><code>% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6148k 100 6148k 0 0 24.0M 0 --:--:-- --:--:-- --:--:-- 24.1M
</code></pre>
-
<h3 id="delete-key">Delete Key</h3>
-
<p>This API allows user to delete a key from a bucket.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>DELETE /{volume}/{bucket}/{key}</code></li>
</ul>
-
<p>Sample HTTP DELETE request:</p>
-
<pre><code>curl -i -X DELETE -H "Authorization:OZONE root" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "x-ozone-version: v1" -H "x-ozone-user:bilbo" "http://localhost:9880/volume-of-bilbo/bucket-0/file-0"
</code></pre>
-
<p>this request deletes key <em>/volume-of-bilbo/bucket-0/file-0</em>. The client receives a zero length content result:</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: f8c4a373-dd5f-4e3a-b6c4-ddf7e191fe91
@@ -905,44 +725,33 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="info-key">Info Key</h3>
-
<p>This API returns information about a given key.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}/{bucket}/{key}?info=key</code></li>
</ul>
-
<p>Query Parameter:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">info</td>
-<td align="left">String, “key”</td>
-<td align="left">Required and enforced with this value.</td>
+<td style="text-align:left">info</td>
+<td style="text-align:left">String, “key”</td>
+<td style="text-align:left">Required and enforced with this value.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP DELETE request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo/buket-0/file-0?info=key"
</code></pre>
-
<p>this request returns information of the key <em>/volume-of-bilbo/bucket-0/file-0</em>. The client receives a JSON object listed attributes of the key.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: c674343c-a0f2-49e4-bbd6-daa73e7dc131
@@ -960,56 +769,43 @@ Connection: keep-alive
"keyName" : "file-0"
}
</code></pre>
-
<h3 id="list-keys">List Keys</h3>
-
<p>This API allows user to list keys in a bucket.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}/{bucket}?prefix=<PREFIX>&max-keys=<MAX_RESULT_SIZE>&prev-key=<PREVIOUS_KEY></code></li>
</ul>
-
<p>Query Parameters:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">prefix</td>
-<td align="left">string</td>
-<td align="left">Optional. Only keys with this prefix are included in the result.</td>
+<td style="text-align:left">prefix</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Only keys with this prefix are included in the result.</td>
</tr>
-
<tr>
-<td align="left">max-keys</td>
-<td align="left">int</td>
-<td align="left">Optional. Maximum number of keys included in the result. Default is 1024 if not specified.</td>
+<td style="text-align:left">max-keys</td>
+<td style="text-align:left">int</td>
+<td style="text-align:left">Optional. Maximum number of keys included in the result. Default is 1024 if not specified.</td>
</tr>
-
<tr>
-<td align="left">prev-key</td>
-<td align="left">string</td>
-<td align="left">Optional. Key name from where listing should start, this key is excluded in the result. It must be a valid key name.</td>
+<td style="text-align:left">prev-key</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Key name from where listing should start, this key is excluded in the result. It must be a valid key name.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http:/localhost:9880/volume-of-bilbo/bucket-0/?max-keys=100&prefix=file"
</code></pre>
-
<p>this request list keys under bucket <em>/volume-of-bilbo/bucket-0</em>, the listing result is filtered by prefix <em>file</em>. The client receives an array of JSON objects, each of them represents the info of a matched key.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: 7f9fc970-9904-4c56-b671-83a086c6f555
diff --git a/docs/0.2.1-alpha/runningviadocker.html b/docs/0.2.1-alpha/runningviadocker.html
index bd9818b..0f09134 100644
--- a/docs/0.2.1-alpha/runningviadocker.html
+++ b/docs/0.2.1-alpha/runningviadocker.html
@@ -188,66 +188,42 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Alpha Cluster</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-<p><strong><em>This is an alpha release of Ozone. Please don’t use this release in
-production.</em></strong> Please check the road map page for features under
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
+<p><em><strong>This is an alpha release of Ozone. Please don’t use this release in
+production.</strong></em> Please check the road map page for features under
development.</p>
-
<p>The easiest way to run ozone is to download the release tarball and launch
ozone via Docker. Docker will create a small ozone cluster on your machine,
including the data nodes and ozone services.</p>
-
<h2 id="running-ozone-via-docker">Running Ozone via Docker</h2>
-
<p><strong>This assumes that you have Docker installed on the machine.</strong></p>
-
<ul>
-<li><p>Download the Ozone tarball and untar it.</p></li>
-
-<li><p>Go to the directory where the docker compose files exist and tell
+<li>
+<p>Download the Ozone tarball and untar it.</p>
+</li>
+<li>
+<p>Go to the directory where the docker compose files exist and tell
<code>docker-compose</code> to start Ozone in the background. This will start a small
-ozone instance on your machine.</p></li>
+ozone instance on your machine.</p>
+</li>
</ul>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">cd ozone-0.2.1-SNAPSHOT/compose/ozone/
-
-docker-compose up -d</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>cd ozone-0.2.1-SNAPSHOT/compose/ozone/
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>docker-compose up -d</span></span></code></pre></div>
<p>To verify that ozone is working as expected, let us log into a data node and
run <em>freon</em>, the load generator for Ozone. The <code>exec datanode bash</code> command
will open a bash shell on the datanode. The ozone freon command is executed
within the datanode container. You can quit freon via CTRL-C any time. The
<code>rk</code> profile instructs freon to generate random keys.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose exec datanode bash
-ozone freon rk</code></pre></div>
-
-<p>You can check out the <strong>OzoneManager UI</strong> at <a href="http://localhost:9874/">http://localhost:9874/</a> to see the
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose exec datanode bash
+</span></span><span style="display:flex;"><span>ozone freon rk</span></span></code></pre></div>
+<p>You can check out the <strong>OzoneManager UI</strong> at http://localhost:9874/ to see the
activity generated by freon.
While you are there, please don’t forget to check out the ozone configuration explorer.</p>
-
-<p><strong><em>Congratulations, You have just run your first ozone cluster.</em></strong></p>
-
+<p><em><strong>Congratulations, You have just run your first ozone cluster.</strong></em></p>
<p>To shutdown the cluster, please run
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose down</code></pre></div></p>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose down</span></span></code></pre></div></p>
</div>
</div>
diff --git a/docs/0.2.1-alpha/runningwithhdfs.html b/docs/0.2.1-alpha/runningwithhdfs.html
index df7e017..859675a 100644
--- a/docs/0.2.1-alpha/runningwithhdfs.html
+++ b/docs/0.2.1-alpha/runningwithhdfs.html
@@ -188,53 +188,26 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Running concurrently with HDFS</h1>
<div class="col-md-9">
- <!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone is designed to work with HDFS. So it is easy to deploy ozone in an
existing HDFS cluster.</p>
-
<p>Ozone does <em>not</em> support security today. It is a work in progress and tracked
- in
+in
<a href="https://issues.apache.org/jira/browse/HDDS-4">HDDS-4</a>. If you enable ozone
in a secure HDFS cluster, for your own protection Ozone will refuse to work.</p>
-
<p>In other words, till Ozone security work is done, Ozone will not work in any
secure clusters.</p>
-
<p>The container manager part of Ozone runs inside DataNodes as a pluggable module.
To activate ozone you should define the service plugin implementation class.</p>
-
-<div class="alert alert-warning" role="alert">
-<b>Important</b>: It should be added to the <b>hdfs-site.xml</b> as the plugin should
-be activated as part of the normal HDFS Datanode bootstrap.
-</div>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>dfs.datanode.plugins<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>org.apache.hadoop.ozone.HddsDatanodeService<span style="color:#f92672"></value></span>
-<span style="color:#f92672"></property></span></code></pre></div>
-
+<!-- raw HTML omitted -->
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span><span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>dfs.datanode.plugins<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>org.apache.hadoop.ozone.HddsDatanodeService<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"></property></span></span></span></code></pre></div>
<p>You also need to add the ozone-datanode-plugin jar file to the classpath:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">export HADOOP_CLASSPATH<span style="color:#f92672">=</span>/opt/ozone/share/hadoop/ozoneplugin/hadoop-ozone-datanode-plugin.jar</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>export HADOOP_CLASSPATH<span style="color:#f92672">=</span>/opt/ozone/share/hadoop/ozoneplugin/hadoop-ozone-datanode-plugin.jar</span></span></code></pre></div>
<p>To start ozone with HDFS you should start the the following components:</p>
-
<ol>
<li>HDFS Namenode (from Hadoop distribution)</li>
<li>HDFS Datanode (from the Hadoop distribution with the plugin on the
@@ -242,16 +215,10 @@ classpath from the Ozone distribution)</li>
<li>Ozone Manager (from the Ozone distribution)</li>
<li>Storage Container manager (from the Ozone distribution)</li>
</ol>
-
<p>Please check the log of the datanode whether the HDDS/Ozone plugin is started or
not. Log of datanode should contain something like this:</p>
-
-<pre><code>2018-09-17 16:19:24 INFO HddsDatanodeService:158 - Started plug-in org.apache.hadoop.ozone.web.OzoneHddsDatanodeService@6f94fb9d
-</code></pre>
-
-<div class="alert alert-warning" role="alert">
-<b>Note:</b> The current version of Ozone is tested with Hadoop 3.1.
-</div>
+<pre tabindex="0"><code>2018-09-17 16:19:24 INFO HddsDatanodeService:158 - Started plug-in org.apache.hadoop.ozone.web.OzoneHddsDatanodeService@6f94fb9d
+</code></pre><!-- raw HTML omitted -->
</div>
</div>
diff --git a/docs/0.2.1-alpha/scmcli.html b/docs/0.2.1-alpha/scmcli.html
index 2cca6f5..f884bd0 100644
--- a/docs/0.2.1-alpha/scmcli.html
+++ b/docs/0.2.1-alpha/scmcli.html
@@ -188,25 +188,9 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>SCMCLI</h1>
<div class="col-md-9">
- <!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>SCM is the block service for Ozone. It is also the workhorse for ozone. But user process never talks to SCM. However, being able to read the state of SCM is useful.</p>
-
<p>SCMCLI allows the developer to access SCM directly. Please note: Improper usage of this tool can destroy your cluster. Unless you know exactly what you are doing, Please do <em>not</em> use this tool. In other words, this is a developer only tool. We might even remove this command in future to prevent improper use.</p>
</div>
diff --git a/docs/0.2.1-alpha/settings.html b/docs/0.2.1-alpha/settings.html
index 3db0677..a7e65de 100644
--- a/docs/0.2.1-alpha/settings.html
+++ b/docs/0.2.1-alpha/settings.html
@@ -188,87 +188,62 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Configuration</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>If you are feeling adventurous, you can setup ozone in a real cluster.
Setting up a real cluster requires us to understand the components of Ozone.
Ozone is designed to work concurrently with HDFS. However, Ozone is also
capable of running independently. The components of ozone are the same in both approaches.</p>
-
<h2 id="ozone-components">Ozone Components</h2>
-
<ol>
<li>Ozone Manager - Is the server that is in charge of the namespace of Ozone. Ozone Manager is responsible for all volume, bucket and key operations.</li>
<li>Storage Container Manager - Acts as the block manager. Ozone Manager
requests blocks from SCM, to which clients can write data.</li>
<li>Datanodes - Ozone data node code runs inside the HDFS datanode or in the independent deployment case runs an ozone datanode daemon.</li>
</ol>
-
<h2 id="setting-up-an-ozone-only-cluster">Setting up an Ozone only cluster</h2>
-
<ul>
-<li><p>Please untar the ozone-0.2.1-SNAPSHOT to the directory where you are going
+<li>
+<p>Please untar the ozone-0.2.1-SNAPSHOT to the directory where you are going
to run Ozone from. We need Ozone jars on all machines in the cluster. So you
-need to do this on all machines in the cluster.</p></li>
-
-<li><p>Ozone relies on a configuration file called <code>ozone-site.xml</code>. To
+need to do this on all machines in the cluster.</p>
+</li>
+<li>
+<p>Ozone relies on a configuration file called <code>ozone-site.xml</code>. To
generate a template that you can replace with proper values, please run the
following command. This will generate a template called <code>ozone-site.xml</code> at
-the specified path (directory).</p></li>
+the specified path (directory).</p>
+</li>
</ul>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone genconf -output <path></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone genconf -output <path></span></span></code></pre></div>
<p>Let us look at the settings inside the generated file (ozone-site.xml) and
how they control ozone. Once the right values are defined, this file
needs to be copied to <code>ozone directory/etc/Hadoop</code>.</p>
-
<ul>
<li><strong>ozone.enabled</strong> This is the most critical setting for ozone.
Ozone is a work in progress and users have to enable this service explicitly.
By default, Ozone is disabled. Setting this flag to <code>true</code> enables ozone in the
HDFS or Ozone cluster.</li>
</ul>
-
<p>Here is an example,</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.enabled<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>True<span style="color:#f92672"></value></span>
- <span style="color:#f92672"></property></span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span> <span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.enabled<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>True<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"></property></span></span></span></code></pre></div>
<ul>
-<li><strong>ozone.metadata.dirs</strong> Allows Administrators to specify where the
+<li>
+<p><strong>ozone.metadata.dirs</strong> Allows Administrators to specify where the
metadata must reside. Usually you pick your fastest disk (SSD if
you have them on your nodes). OzoneManager, SCM and datanode will write the
metadata to this path. This is a required setting, if this is missing Ozone
-will fail to come up.</li>
-</ul>
-
+will fail to come up.</p>
<p>Here is an example,</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.metadata.dirs<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>/data/disk1/meta<span style="color:#f92672"></value></span>
- <span style="color:#f92672"></property></span></code></pre></div>
-
+</li>
+</ul>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span> <span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.metadata.dirs<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>/data/disk1/meta<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"></property></span></span></span></code></pre></div>
<ul>
<li><strong>ozone.scm.names</strong> Storage container manager(SCM) is a distributed block
service which is used by ozone. This property allows data nodes to discover
@@ -276,40 +251,31 @@ SCM’s address. Data nodes send heartbeat to SCM.
Until HA feature is complete, we configure ozone.scm.names to be a
single machine.</li>
</ul>
-
<p>Here is an example,</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.scm.names<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>scm.hadoop.apache.org<span style="color:#f92672"></value></span>
- <span style="color:#f92672"></property></span>
- </code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span> <span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.scm.names<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>scm.hadoop.apache.org<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"></property></span></span></span></code></pre></div>
<ul>
<li><strong>ozone.scm.datanode.id</strong> Data nodes generate a Unique ID called Datanode
ID. This identity is written to the file specified by this path. <em>Data nodes
will create this path if it doesn’t exist already.</em></li>
</ul>
-
<p>Here is an example,
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.scm.datanode.id<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>/data/disk1/meta/node/datanode.id<span style="color:#f92672"></value></span>
- <span style="color:#f92672"></property></span></code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span> <span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.scm.datanode.id<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>/data/disk1/meta/node/datanode.id<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"></property></span></span></span></code></pre></div></p>
<ul>
<li><strong>ozone.om.address</strong> OM server address. This is used by OzoneClient and
Ozone File System.</li>
</ul>
-
<p>Here is an example,
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.om.address<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>ozonemanager.hadoop.apache.org<span style="color:#f92672"></value></span>
- <span style="color:#f92672"></property></span></code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span> <span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.om.address<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>ozonemanager.hadoop.apache.org<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"></property></span></span></span></code></pre></div></p>
<h3 id="ozone-settings-summary">Ozone Settings Summary</h3>
-
<table>
<thead>
<tr>
@@ -318,44 +284,37 @@ Ozone File System.</li>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>ozone.enabled</td>
<td>true</td>
<td>This enables SCM and containers in HDFS cluster.</td>
</tr>
-
<tr>
<td>ozone.metadata.dirs</td>
<td>file path</td>
<td>The metadata will be stored here.</td>
</tr>
-
<tr>
<td>ozone.scm.names</td>
<td>SCM server name</td>
<td>Hostname:port or IP:port address of SCM.</td>
</tr>
-
<tr>
<td>ozone.scm.block.client.address</td>
<td>SCM server name and port</td>
<td>Used by services like OM</td>
</tr>
-
<tr>
<td>ozone.scm.client.address</td>
<td>SCM server name and port</td>
<td>Used by client-side</td>
</tr>
-
<tr>
<td>ozone.scm.datanode.address</td>
<td>SCM server name and port</td>
<td>Used by datanode to talk to SCM</td>
</tr>
-
<tr>
<td>ozone.om.address</td>
<td>OM server name</td>
diff --git a/docs/0.2.1-alpha/sitemap.xml b/docs/0.2.1-alpha/sitemap.xml
index ddacddc..1bdb297 100644
--- a/docs/0.2.1-alpha/sitemap.xml
+++ b/docs/0.2.1-alpha/sitemap.xml
@@ -1,100 +1,66 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:xhtml="http://www.w3.org/1999/xhtml">
-
<url>
<loc>/</loc>
- <lastmod>2017-09-14T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/runningviadocker.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/buildingsources.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/settings.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/runningwithhdfs.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/realcluster.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/hdds.html</loc>
- <lastmod>2017-09-14T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/ozonemanager.html</loc>
- <lastmod>2017-09-14T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/concepts.html</loc>
- <lastmod>2017-10-10T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/javaapi.html</loc>
- <lastmod>2017-09-14T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-09-18T10:23:36-07:00</lastmod>
+ </url><url>
<loc>/ozonefs.html</loc>
- <lastmod>2017-09-14T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-09-18T10:23:36-07:00</lastmod>
+ </url><url>
<loc>/freon.html</loc>
- <lastmod>2017-09-02T23:58:17-07:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/dozone.html</loc>
- <lastmod>2017-08-10T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/scmcli.html</loc>
- <lastmod>2017-08-10T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/bucketcommands.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/categories.html</loc>
- <priority>0</priority>
- </url>
-
- <url>
+ </url><url>
<loc>/keycommands.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/commandshell.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/rest.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
+ </url><url>
<loc>/tags.html</loc>
- <priority>0</priority>
- </url>
-
- <url>
+ </url><url>
<loc>/volumecommands.html</loc>
+ <lastmod>2018-09-17T16:21:10-07:00</lastmod>
</url>
-
-</urlset>
\ No newline at end of file
+</urlset>
diff --git a/docs/0.2.1-alpha/tags/index.xml b/docs/0.2.1-alpha/tags/index.xml
index 8a5c1ee..22e514b 100644
--- a/docs/0.2.1-alpha/tags/index.xml
+++ b/docs/0.2.1-alpha/tags/index.xml
@@ -1,14 +1,10 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Tags on Ozone</title>
<link>/tags.html</link>
<description>Recent content in Tags on Ozone</description>
<generator>Hugo -- gohugo.io</generator>
- <language>en-us</language>
-
- <atom:link href="/tags/index.xml" rel="self" type="application/rss+xml" />
-
-
+ <language>en-us</language><atom:link href="/tags/index.xml" rel="self" type="application/rss+xml" />
</channel>
-</rss>
\ No newline at end of file
+</rss>
diff --git a/docs/0.2.1-alpha/volumecommands.html b/docs/0.2.1-alpha/volumecommands.html
index 1941f8b..ac79a00 100644
--- a/docs/0.2.1-alpha/volumecommands.html
+++ b/docs/0.2.1-alpha/volumecommands.html
@@ -188,27 +188,9 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Volume Commands</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Volume commands generally need administrator privileges. The ozone shell supports the following volume commands.</p>
-
<ul>
<li><a href="#create">create</a></li>
<li><a href="#delete">delete</a></li>
@@ -216,14 +198,10 @@
<li><a href="#list">list</a></li>
<li><a href="#update">update</a></li>
</ul>
-
<h3 id="create">Create</h3>
-
<p>The volume create command allows an administrator to create a volume and
assign it to a user.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -231,37 +209,28 @@ assign it to a user.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>-q, –quota</td>
<td>Optional, This argument that specifies the maximum size this volume can use in the Ozone cluster.</td>
</tr>
-
<tr>
<td>-u, –user</td>
<td>Required, The name of the user who owns this volume. This user can create, buckets and keys on this volume.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the volume.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume create --quota<span style="color:#f92672">=</span>1TB --user<span style="color:#f92672">=</span>bilbo /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume create --quota<span style="color:#f92672">=</span>1TB --user<span style="color:#f92672">=</span>bilbo /hive</span></span></code></pre></div>
<p>The above command will create a volume called <em>hive</em> on the ozone cluster. This
volume has a quota of 1TB, and the owner is <em>bilbo</em>.</p>
-
<h3 id="delete">Delete</h3>
-
<p>The volume delete commands allows an administrator to delete a volume. If the
volume is not empty then this command will fail.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -269,7 +238,6 @@ volume is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -277,18 +245,13 @@ volume is not empty then this command will fail.</p>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume delete /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume delete /hive</span></span></code></pre></div>
<p>The above command will delete the volume hive, if the volume has no buckets
inside it.</p>
-
<h3 id="info">Info</h3>
-
<p>The volume info commands returns the information about the volume including
quota and owner information.
-<strong><em>Params:</em></strong></p>
-
+<em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -296,7 +259,6 @@ quota and owner information.
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -304,25 +266,15 @@ quota and owner information.
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume info /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume info /hive</span></span></code></pre></div>
<p>The above command will print out the information about hive volume.</p>
-
<h3 id="list">List</h3>
-
<p>The volume list command will list the volumes owned by a user.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume list --user hadoop</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume list --user hadoop</span></span></code></pre></div>
<p>The above command will print out all the volumes owned by the user hadoop.</p>
-
<h3 id="update">Update</h3>
-
<p>The volume update command allows changing of owner and quota on a given volume.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -330,29 +282,23 @@ quota and owner information.
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>-q, –quota</td>
<td>Optional, This argument that specifies the maximum size this volume can use in the Ozone cluster.</td>
</tr>
-
<tr>
<td>-u, –user</td>
<td>Optional, The name of the user who owns this volume. This user can create, buckets and keys on this volume.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the volume.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume update --quota<span style="color:#f92672">=</span>10TB /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume update --quota<span style="color:#f92672">=</span>10TB /hive</span></span></code></pre></div>
<p>The above command updates the volume quota to 10TB.</p>
-
<p>You can try out these commands from the docker instance of the <a href="runningviadocker.html">Alpha
Cluster</a>.</p>
diff --git a/docs/0.3.0-alpha/OzoneOverview.svg b/docs/0.3.0-alpha/OzoneOverview.svg
index c7fda5d..9d4660d 100644
--- a/docs/0.3.0-alpha/OzoneOverview.svg
+++ b/docs/0.3.0-alpha/OzoneOverview.svg
@@ -235,4 +235,4 @@
</text>
</g>
</g>
-</svg>
+</svg>
\ No newline at end of file
diff --git a/docs/0.3.0-alpha/bucketcommands.html b/docs/0.3.0-alpha/bucketcommands.html
index 89cbff1..f367735 100644
--- a/docs/0.3.0-alpha/bucketcommands.html
+++ b/docs/0.3.0-alpha/bucketcommands.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,27 +221,9 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Bucket Commands</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone shell supports the following bucket commands.</p>
-
<ul>
<li><a href="#create">create</a></li>
<li><a href="#delete">delete</a></li>
@@ -227,13 +231,9 @@
<li><a href="#list">list</a></li>
<li><a href="#update">update</a></li>
</ul>
-
<h3 id="create">Create</h3>
-
<p>The bucket create command allows a user to create a bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -241,7 +241,6 @@
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -249,19 +248,13 @@
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket create /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket create /hive/jan</span></span></code></pre></div>
<p>The above command will create a bucket called <em>jan</em> in the <em>hive</em> volume.
Since no scheme was specified this command defaults to O3 (RPC) protocol.</p>
-
<h3 id="delete">Delete</h3>
-
<p>The bucket delete commands allows an user to delete a volume. If the
bucket is not empty then this command will fail.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -269,7 +262,6 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -277,16 +269,11 @@ bucket is not empty then this command will fail.</p>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume delete /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume delete /hive/jan</span></span></code></pre></div>
<p>The above command will delete <em>jan</em> bucket if it is empty.</p>
-
<h3 id="info">Info</h3>
-
<p>The bucket info commands returns the information about the bucket.
-<strong><em>Params:</em></strong></p>
-
+<em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -294,7 +281,6 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -302,17 +288,11 @@ bucket is not empty then this command will fail.</p>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket info /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket info /hive/jan</span></span></code></pre></div>
<p>The above command will print out the information about <em>jan</em> bucket.</p>
-
<h3 id="list">List</h3>
-
<p>The bucket list commands allows uset to list the buckets in a volume.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -320,40 +300,30 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>-l, –length</td>
<td>Maximum number of results to return. Default: 100</td>
</tr>
-
<tr>
<td>-p, –prefix</td>
<td>Optional, Only buckets that match this prefix will be returned.</td>
</tr>
-
<tr>
<td>-s, –start</td>
<td>The listing will start from key after the start key.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the <em>volume</em>.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket list /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket list /hive</span></span></code></pre></div>
<p>This command will list all buckets on the volume <em>hive</em>.</p>
-
<h3 id="update">Update</h3>
-
<p>The bucket update command allows changing access permissions on bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -361,29 +331,23 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>–addAcl</td>
<td>Optional, Comma separated ACLs that will added to bucket.</td>
</tr>
-
<tr>
<td>–removeAcl</td>
<td>Optional, Comma separated list of acl to remove.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the bucket.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket update --addAcl<span style="color:#f92672">=</span>user:bilbo:rw /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket update --addAcl<span style="color:#f92672">=</span>user:bilbo:rw /hive/jan</span></span></code></pre></div>
<p>The above command gives user bilbo read/write permission to the bucket.</p>
-
<p>You can try out these commands from the docker instance of the <a href="runningviadocker.html">Alpha
Cluster</a>.</p>
diff --git a/docs/0.3.0-alpha/buildingsources.html b/docs/0.3.0-alpha/buildingsources.html
index a710366..5ec142b 100644
--- a/docs/0.3.0-alpha/buildingsources.html
+++ b/docs/0.3.0-alpha/buildingsources.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,52 +221,27 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Building from Sources</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-<p><strong><em>This is a guide on how to build the ozone sources. If you are <font
-color="red">not</font>
-planning to build sources yourself, you can safely skip this page.</em></strong></p>
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
+<p><em><strong>This is a guide on how to build the ozone sources. If you are <!-- raw HTML omitted -->not<!-- raw HTML omitted -->
+planning to build sources yourself, you can safely skip this page.</strong></em></p>
<p>If you are a Hadoop ninja, and wise in the ways of Apache, you already know
that a real Apache release is a source release.</p>
-
<p>If you want to build from sources, Please untar the source tarball and run
the ozone build command. This instruction assumes that you have all the
dependencies to build Hadoop on your build machine. If you need instructions
on how to build Hadoop, please look at the Apache Hadoop Website.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">mvn clean package -DskipTests<span style="color:#f92672">=</span>true -Dmaven.javadoc.skip<span style="color:#f92672">=</span>true -Phdds -Pdist -Dtar -DskipShade</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>mvn clean package -DskipTests<span style="color:#f92672">=</span>true -Dmaven.javadoc.skip<span style="color:#f92672">=</span>true -Phdds -Pdist -Dtar -DskipShade</span></span></code></pre></div>
<p>This will build an ozone-<version>.tar.gz in your target directory.</p>
-
<p>You can copy this tarball and use this instead of binary artifacts that are
provided along with the official release.</p>
-
<h2 id="how-to-test-the-build">How to test the build</h2>
-
<p>You can run the acceptance tests in the hadoop-ozone directory to make sure
that your build is functional. To launch the acceptance tests, please follow
- the instructions in the <strong>README.md</strong> in the
- <code>$hadoop_src/hadoop-ozone/acceptance-test</code> directory. Acceptance tests
- will start a small ozone cluster and verify that ozone shell and ozone file
- system is fully functional.</p>
+the instructions in the <strong>README.md</strong> in the
+<code>$hadoop_src/hadoop-ozone/acceptance-test</code> directory. Acceptance tests
+will start a small ozone cluster and verify that ozone shell and ozone file
+system is fully functional.</p>
</div>
</div>
diff --git a/docs/0.3.0-alpha/categories/index.xml b/docs/0.3.0-alpha/categories/index.xml
index 5c6fe28..eca954b 100644
--- a/docs/0.3.0-alpha/categories/index.xml
+++ b/docs/0.3.0-alpha/categories/index.xml
@@ -1,14 +1,10 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Categories on Ozone</title>
<link>/categories.html</link>
<description>Recent content in Categories on Ozone</description>
<generator>Hugo -- gohugo.io</generator>
- <language>en-us</language>
-
- <atom:link href="/categories/index.xml" rel="self" type="application/rss+xml" />
-
-
+ <language>en-us</language><atom:link href="/categories/index.xml" rel="self" type="application/rss+xml" />
</channel>
-</rss>
\ No newline at end of file
+</rss>
diff --git a/docs/0.3.0-alpha/commandshell.html b/docs/0.3.0-alpha/commandshell.html
index 628d039..05e4027 100644
--- a/docs/0.3.0-alpha/commandshell.html
+++ b/docs/0.3.0-alpha/commandshell.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,31 +221,11 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Ozone CLI</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone has a set of command line tools that can be used to manage ozone.</p>
-
<p>All these commands are invoked via the <code>ozone</code> script.</p>
-
<p>The commands supported by ozone are:</p>
-
<ul>
<li><strong>classpath</strong> - Prints the class path needed to get the hadoop jar and the
required libraries.</li>
@@ -245,34 +247,24 @@ Container Manager.</li>
<li><strong>genconf</strong> - Generate minimally required ozone configs and output to
ozone-site.xml.</li>
</ul>
-
<h2 id="understanding-ozone-command-shell">Understanding Ozone command shell</h2>
-
<p>The most used command when working with Ozone is the Ozone command shell.
Ozone command shell gives a command shell interface to work against
Ozone.</p>
-
<p>The Ozone shell commands take the following format.</p>
-
<blockquote>
<p><em>ozone sh object action url</em></p>
</blockquote>
-
<p><strong>ozone</strong> script is used to invoke all Ozone sub-commands. The ozone shell is
invoked via <code>sh</code> command.</p>
-
<p>The object can be a volume, bucket or a key. The action is various verbs like
- create, list, delete etc.</p>
-
+create, list, delete etc.</p>
<p>Ozone URL can point to a volume, bucket or keys in the following format:</p>
-
<p><em>[scheme][server:port]/volume/bucket/key</em></p>
-
<p>Where,</p>
-
<ol>
-<li><p>Scheme - Can be one of the following</p>
-
+<li>
+<p>Scheme - Can be one of the following</p>
<ul>
<li>o3 - Ozone’s native RPC protocol. If you specify this scheme, the
native RPC protocol is used while communicating with Ozone Manager and
@@ -282,29 +274,25 @@ that you are interested in using the Ozone Rest protocol and falls back
to using the REST protocol instead of RPC.
If no protocol is specified, the Ozone shell defaults to the native RPC
protocol.</li>
-</ul></li>
-
-<li><p>Server:Port - This is the address of the Ozone Manager. This can be server
+</ul>
+</li>
+<li>
+<p>Server:Port - This is the address of the Ozone Manager. This can be server
only, in that case, the default port is used. If this value is omitted
then the defaults specified in the ozone-site.xml will be used for Ozone
-Manager address.</p></li>
+Manager address.</p>
+</li>
</ol>
-
<p>Depending on the call, the volume/bucket/key names will be part of the URL.
Please see volume commands, bucket commands, and key commands section for more
detail.</p>
-
<h2 id="invoking-help">Invoking help</h2>
-
<p>Ozone shell help can be invoked at <em>object</em> level or at <em>action</em> level.
For example:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume --help</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume --help</span></span></code></pre></div>
<p>This will show all possible actions for volumes.</p>
-
<p>or it can be invoked to explain a specific action like
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume create --help</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume create --help</span></span></code></pre></div>
This command will give you command line options of the create command.</p>
</div>
diff --git a/docs/0.3.0-alpha/concepts.html b/docs/0.3.0-alpha/concepts.html
index ebb6703..1adccee 100644
--- a/docs/0.3.0-alpha/concepts.html
+++ b/docs/0.3.0-alpha/concepts.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,116 +221,61 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Architecture</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone is a redundant, distributed object store build by
leveraging primitives present in HDFS. The primary design point of ozone is scalability, and it aims to scale to billions of objects.</p>
-
<p>Ozone consists of volumes, buckets, and keys. A volume is similar to a home directory in the ozone world. Only an administrator can create it. Volumes are used to store buckets. Once a volume is created users can create as many buckets as needed. Ozone stores data as keys which live inside these buckets.</p>
-
<p>Ozone namespace is composed of many storage volumes. Storage volumes are also used as the basis for storage accounting.</p>
-
<p>To access a key, an Ozone URL has the following format:</p>
-
-<pre><code>http://servername:port/volume/bucket/key
-</code></pre>
-
-<p>Where the server name is the name of a data node, the port is the data node HTTP port. The volume represents the name of the ozone volume; bucket is an ozone bucket created by the user and key represents the file.</p>
-
+<pre tabindex="0"><code>http://servername:port/volume/bucket/key
+</code></pre><p>Where the server name is the name of a data node, the port is the data node HTTP port. The volume represents the name of the ozone volume; bucket is an ozone bucket created by the user and key represents the file.</p>
<p>Please look at the <a href="./commandshell.html#shell">command line interface</a> for more info.</p>
-
<p>Ozone supports both REST and RPC protocols. Clients can choose either of these protocols to communicate with Ozone. Please see the <a href="./javaapi.html">client documentation</a> for more details.</p>
-
<p>Ozone separates namespace management and block space management; this helps
ozone to scale much better. The namespace is managed by a daemon called
<a href="./ozonemanager.html">Ozone Manager </a> (OM), and block space is
-managed by <a href="./hdds.html">Storage Container Manager</a> (SCM).</p>
-
+managed by [Storage Container Manager] (/hdds.html) (SCM).</p>
<p>The data nodes provide replication and ability to store blocks; these blocks are stored in groups to reduce the metadata pressure on SCM. This groups of blocks are called storage containers. Hence the block manager is called storage container
manager.</p>
-
<h2 id="ozone-overview">Ozone Overview</h2>
-
<p>
The following diagram is a high-level overview of the core components of Ozone.
</p>
-
-<p><img src="../../OzoneOverview.svg" alt="Architecture diagram" /></p>
-
+<p><img src="../../OzoneOverview.svg" alt="Architecture diagram"></p>
<p>The main elements of Ozone are
:</p>
-
<h3 id="ozone-manager">Ozone Manager
</h3>
-
<p><a href="./ozonemanager.html">Ozone Manager</a> (OM) takes care of the Ozone’s namespace.
All ozone objects like volumes, buckets, and keys are managed by OM. In Short, OM is the metadata manager for Ozone.
OM talks to blockManager(SCM) to get blocks and passes it on to the Ozone
client. Ozone client writes data to these blocks.
OM will eventually be replicated via Apache Ratis for High Availability.
</p>
-
<h3 id="storage-container-manager">Storage Container Manager</h3>
-
<p><a href="./hdds.html">Storage Container Manager</a> (SCM) is the block and cluster manager for Ozone.
SCM along with data nodes offer a service called ‘storage containers’.
A storage container is a group unrelated of blocks that are managed together as a single entity.</p>
-
<p>SCM offers the following abstractions.
</p>
-
-<p><img src="../../SCMBlockDiagram.png" alt="SCM Abstractions" /></p>
-
+<p><img src="../../SCMBlockDiagram.png" alt="SCM Abstractions"></p>
<h3 id="blocks">Blocks</h3>
-
<p>Blocks are similar to blocks in HDFS. They are replicated store of data. Client writes data to blocks.</p>
-
<h3 id="containers">Containers</h3>
-
<p>A collection of blocks replicated and managed together.</p>
-
<h3 id="pipelines">Pipelines</h3>
-
<p>SCM allows each storage container to choose its method of replication.
For example, a storage container might decide that it needs only one copy of a block
and might choose a stand-alone pipeline. Another storage container might want to have a very high level of reliability and pick a RATIS based pipeline. In other words, SCM allows different kinds of replication strategies to co-exist. The client while writing data, chooses a storage container with required properties.</p>
-
<h3 id="pools">Pools</h3>
-
<p>A group of data nodes is called a pool. For scaling purposes,
we define a pool as a set of machines. This makes management of data nodes easier.</p>
-
<h3 id="nodes">Nodes</h3>
-
<p>The data node where data is stored. SCM monitors these nodes via heartbeat.</p>
-
<h3 id="clients">Clients</h3>
-
-<p>Ozone ships with a set of clients. Ozone <a href="./commandshell.html#shell">CLI</a> is the command line interface like ‘hdfs’ command.
<a href="./freon.html">Freon</a> is a load generation tool for Ozone.
</p>
-
+<p>Ozone ships with a set of clients. Ozone <a href="./commandshell.html#shell">CLI</a> is the command line interface like ‘hdfs’ command.
[Freon] (/freon.html) is a load generation tool for Ozone.
</p>
<h3 id="rest-handler">REST Handler</h3>
-
<p>Ozone provides an RPC (Remote Procedure Call) as well as a REST (Representational State Transfer) interface. This allows clients to be written in many languages quickly. Ozone strives to maintain an API compatibility between REST and RPC.
For most purposes, a client can make one line change to switch from REST to RPC or vice versa.
</p>
-
<h3 id="ozone-file-system">Ozone File System</h3>
-
<p>Ozone file system (TODO: Add documentation) is a Hadoop compatible file system. This allows Hadoop services and applications like Hive and Spark to run against
Ozone without any change.</p>
-
<h3 id="ozone-client">Ozone Client</h3>
-
<p>This is similar to DFSClient in HDFS. This is the standard client to talk to Ozone. All other components that we have discussed so far rely on Ozone client. Ozone client supports both RPC and REST protocols.</p>
</div>
diff --git a/docs/0.3.0-alpha/dozone.html b/docs/0.3.0-alpha/dozone.html
index c369303..2d21d29 100644
--- a/docs/0.3.0-alpha/dozone.html
+++ b/docs/0.3.0-alpha/dozone.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,91 +221,45 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Dozone & Dev Tools</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Dozone stands for docker for ozone. Ozone supports docker to make it easy to develop and test ozone. Starting a docker based ozone container is simple.</p>
-
<p>In the <code>compose/ozone</code> directory there are two files that define the docker and ozone settings.</p>
-
<p>Developers can</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">cd compose/ozone</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>cd compose/ozone</span></span></code></pre></div>
<p>and simply run</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose up -d</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose up -d</span></span></code></pre></div>
<p>to run a ozone cluster on docker.</p>
-
<p>This command will launch a Namenode, OM, SCM and a data node.</p>
-
-<p>To access the OM UI, one can run ‘<a href="http://localhost:9874'">http://localhost:9874'</a>.</p>
-
+<p>To access the OM UI, one can run ‘http://localhost:9874’.</p>
<p><em>Please note</em>: dozone does not map the data node ports to the 9864. Instead, it maps to the ephemeral port range. So many examples in the command shell will not work if you run those commands from the host machine. To find out where the data node port is listening, you can run the <code>docker ps</code> command or always ssh into a container before running ozone commands.</p>
-
<p>To shutdown a running docker based ozone cluster, please run</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose down</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose down</span></span></code></pre></div>
<h2 id="adding-more-config-settings">Adding more config settings</h2>
-
<p>The file called <code>docker-config</code> contains all ozone specific config settings. This file is processed to create the ozone-site.xml.</p>
-
-<h2 id="useful-docker-ozone-commands">Useful Docker & Ozone Commands</h2>
-
+<h2 id="useful-docker--ozone-commands">Useful Docker & Ozone Commands</h2>
<p>If you make any modifications to ozone, the simplest way to test it is to run freon and unit tests.</p>
-
<p>Here are the instructions to run freon in a docker based cluster.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose exec datanode bash</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose exec datanode bash</span></span></code></pre></div>
<p>This will open a bash shell on the data node container.
Now we can execute freon for load generation.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span></span></span></code></pre></div>
<p>Here is a set helpful commands while working with docker for ozone.
To check the status of the components:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose ps</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose ps</span></span></code></pre></div>
<p>To get logs from a specific node/service:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose logs scm</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose logs scm</span></span></code></pre></div>
<p>As the WebUI ports are forwarded to the external machine, you can check the web UI:</p>
-
<ul>
-<li>For the Storage Container Manager: <a href="http://localhost:9876">http://localhost:9876</a></li>
-<li>For the Ozone Managerr: <a href="http://localhost:9874">http://localhost:9874</a></li>
+<li>For the Storage Container Manager: http://localhost:9876</li>
+<li>For the Ozone Managerr: http://localhost:9874</li>
<li>For the Datanode: check the port with docker ps (as there could be multiple data node ports are mapped to the ephemeral port range)</li>
-<li>For the Namenode: <a href="http://localhost:9870">http://localhost:9870</a></li>
+<li>For the Namenode: http://localhost:9870</li>
</ul>
-
<p>You can start multiple data nodes with:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose scale datanode<span style="color:#f92672">=</span><span style="color:#ae81ff">3</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose scale datanode<span style="color:#f92672">=</span><span style="color:#ae81ff">3</span></span></span></code></pre></div>
<p>You can test the commands from the <a href="./commandshell.html#shell">Ozone CLI</a> after opening a new bash shell in one of the containers:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose exec datanode bash</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose exec datanode bash</span></span></code></pre></div>
</div>
</div>
diff --git a/docs/0.3.0-alpha/freon.html b/docs/0.3.0-alpha/freon.html
index 0f7ca73..2721c76 100644
--- a/docs/0.3.0-alpha/freon.html
+++ b/docs/0.3.0-alpha/freon.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,60 +221,34 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Freon</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="overview">Overview</h2>
-
<p>Freon is a load-generator for Ozone. This tool is used for testing the functionality of ozone.</p>
-
<h3 id="random-keys">Random keys</h3>
-
<p>In randomkeys mode, the data written into ozone cluster is randomly generated.
Each key will be of size 10 KB.</p>
-
<p>The number of volumes/buckets/keys can be configured. The replication type and
factor (eg. replicate with ratis to 3 nodes) Also can be configured.</p>
-
<p>For more information use</p>
-
<p><code>bin/ozone freon --help</code></p>
-
<h3 id="example">Example</h3>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span> --replicationType<span style="color:#f92672">=</span>RATIS --factor<span style="color:#f92672">=</span>THREE</code>< [...]
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">***************************************************
-Status: Success
-Git Base Revision: 48aae081e5afacbb3240657556b26c29e61830c3
-Number of Volumes created: <span style="color:#ae81ff">10</span>
-Number of Buckets created: <span style="color:#ae81ff">100</span>
-Number of Keys added: <span style="color:#ae81ff">1000</span>
-Ratis replication factor: THREE
-Ratis replication type: RATIS
-Average Time spent in volume creation: <span style="color:#ae81ff">00</span>:00:00,035
-Average Time spent in bucket creation: <span style="color:#ae81ff">00</span>:00:00,319
-Average Time spent in key creation: <span style="color:#ae81ff">00</span>:00:03,659
-Average Time spent in key write: <span style="color:#ae81ff">00</span>:00:10,894
-Total bytes written: <span style="color:#ae81ff">10240000</span>
-Total Execution time: <span style="color:#ae81ff">00</span>:00:16,898
-***********************</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span> --replicationType<span style="color:#f92672">=</span>RATIS --factor< [...]
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>***************************************************
+</span></span><span style="display:flex;"><span>Status: Success
+</span></span><span style="display:flex;"><span>Git Base Revision: 48aae081e5afacbb3240657556b26c29e61830c3
+</span></span><span style="display:flex;"><span>Number of Volumes created: <span style="color:#ae81ff">10</span>
+</span></span><span style="display:flex;"><span>Number of Buckets created: <span style="color:#ae81ff">100</span>
+</span></span><span style="display:flex;"><span>Number of Keys added: <span style="color:#ae81ff">1000</span>
+</span></span><span style="display:flex;"><span>Ratis replication factor: THREE
+</span></span><span style="display:flex;"><span>Ratis replication type: RATIS
+</span></span><span style="display:flex;"><span>Average Time spent in volume creation: 00:00:00,035
+</span></span><span style="display:flex;"><span>Average Time spent in bucket creation: 00:00:00,319
+</span></span><span style="display:flex;"><span>Average Time spent in key creation: 00:00:03,659
+</span></span><span style="display:flex;"><span>Average Time spent in key write: 00:00:10,894
+</span></span><span style="display:flex;"><span>Total bytes written: <span style="color:#ae81ff">10240000</span>
+</span></span><span style="display:flex;"><span>Total Execution time: 00:00:16,898
+</span></span><span style="display:flex;"><span>***********************</span></span></code></pre></div>
</div>
</div>
diff --git a/docs/0.3.0-alpha/hdds.html b/docs/0.3.0-alpha/hdds.html
index df660a5..adcb729 100644
--- a/docs/0.3.0-alpha/hdds.html
+++ b/docs/0.3.0-alpha/hdds.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,38 +221,16 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Hadoop Distributed Data Store</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="scm-overview">SCM Overview</h2>
-
<p>Storage Container Manager or SCM is a very important component of ozone. SCM
offers block and container-based services to Ozone Manager. A container is a
collection of unrelated blocks under ozone. SCM and data nodes work together
to maintain the replication levels needed by the cluster.</p>
-
<p>It is easier to look at a putKey operation to understand the role that SCM plays.</p>
-
<p>To put a key, a client makes a call to OM with the following arguments.</p>
-
<p>– putKey(keyName, data, pipeline type, replication count)</p>
-
<ol>
<li>keyName - refers to the file name.</li>
<li>data - The data that the client wants to write.</li>
@@ -239,21 +239,16 @@ refers to the replication strategy used for replicating a block. Ozone
currently supports Stand Alone and Ratis as two different pipeline types.</li>
<li>replication count - This specifies how many copies of the block replica should be maintained.</li>
</ol>
-
<p>In most cases, the client does not specify the pipeline type and replication
- count. The default pipeline type and replication count are used.</p>
-
+count. The default pipeline type and replication count are used.</p>
<p>Ozone Manager when it receives the putKey call, makes a call to SCM asking
for a pipeline instance with the specified property. So if the client asked
for RATIS replication strategy and a replication count of three, then OM
requests SCM to return a set of data nodes that meet this capability.</p>
-
<p>If SCM can find this a pipeline ( that is a set of data nodes) that can meet
the requirement from the client, then those nodes are returned to OM. OM will
persist this info and return a tuple consisting of {BlockID, ContainerName, and Pipeline}.</p>
-
<p>If SCM is not able to find a pipeline, then SCM creates a logical pipeline and then returns it.</p>
-
<p>SCM manages blocks, containers, and pipelines. To return healthy pipelines,
SCM also needs to understand the node health. So SCM listens to heartbeats
from data nodes and acts as the node manager too.</p>
diff --git a/docs/0.3.0-alpha/index.html b/docs/0.3.0-alpha/index.html
index ddea892..06f79d9 100644
--- a/docs/0.3.0-alpha/index.html
+++ b/docs/0.3.0-alpha/index.html
@@ -3,7 +3,7 @@
<!DOCTYPE html>
<html lang="en">
<head>
- <meta name="generator" content="Hugo 0.50" />
+ <meta name="generator" content="Hugo 0.108.0">
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
@@ -18,6 +18,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -198,40 +220,19 @@
</div>
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h1 id="apache-hadoop-ozone">Apache Hadoop Ozone</h1>
-
<p>Ozone is a scalable, distributed object store for Hadoop. Applications like
Apache Spark, Hive and YARN, can run against Ozone without any
-modifications. Ozone comes with a <a href="./javaapi.html">Java client library</a> and a <a href="./commandshell.html#shell">command line interface</a> which makes it easy to use Ozone. This client library supports both RPC and REST protocols.</p>
-
+modifications. Ozone comes with a <a href="./javaapi.html">Java client library</a> and a [command line interface] (/commandshell.html#shell) which makes it easy to use Ozone. This client library supports both RPC and REST protocols.</p>
<p>Ozone consists of volumes, buckets, and Keys.</p>
-
<ul>
<li>Volumes are similar to user accounts. Only administrators can create or delete volumes.</li>
<li>Buckets are similar to directories. A bucket can contain any number of keys, but buckets cannot contain other buckets.</li>
<li>Keys are similar to files. A bucket can contain any number of keys.</li>
</ul>
-
-<p><a href="./runningviadocker.html"><button class="btn btn-danger btn-lg">Getting started</button></a></p>
+<p><!-- raw HTML omitted --><!-- raw HTML omitted -->Getting started<!-- raw HTML omitted --><!-- raw HTML omitted --></p>
diff --git a/docs/0.3.0-alpha/index.xml b/docs/0.3.0-alpha/index.xml
index 59a80e4..7697856 100644
--- a/docs/0.3.0-alpha/index.xml
+++ b/docs/0.3.0-alpha/index.xml
@@ -1,16 +1,11 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Ozone Overview on Ozone</title>
<link>/</link>
<description>Recent content in Ozone Overview on Ozone</description>
<generator>Hugo -- gohugo.io</generator>
- <language>en-us</language>
- <lastBuildDate>Thu, 14 Sep 2017 00:00:00 +0000</lastBuildDate>
-
- <atom:link href="/index.xml" rel="self" type="application/rss+xml" />
-
-
+ <language>en-us</language><atom:link href="/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>Alpha Cluster</title>
<link>/runningviadocker.html</link>
@@ -40,7 +35,7 @@ If you want to build from sources, Please untar the source tarball and run the o
<guid>/settings.html</guid>
<description>If you are feeling adventurous, you can setup ozone in a real cluster. Setting up a real cluster requires us to understand the components of Ozone. Ozone is designed to work concurrently with HDFS. However, Ozone is also capable of running independently. The components of ozone are the same in both approaches.
-Ozone Components Ozone Manager - Is the server that is in charge of the namespace of Ozone.</description>
+Ozone Components Ozone Manager - Is the server that is in charge of the namespace of Ozone. Ozone Manager is responsible for all volume, bucket and key operations.</description>
</item>
<item>
@@ -162,9 +157,10 @@ SCMCLI allows the developer to access SCM directly. Please note: Improper usage
<guid>/bucketcommands.html</guid>
<description>Ozone shell supports the following bucket commands.
- create delete info list update Create The bucket create command allows a user to create a bucket.
+create delete info list update Create The bucket create command allows a user to create a bucket.
Params:
- Arguments Comment Uri The name of the bucket in /volume/bucket format. ozone sh bucket create /hive/jan The above command will create a bucket called jan in the hive volume. Since no scheme was specified this command defaults to O3 (RPC) protocol.</description>
+Arguments Comment Uri The name of the bucket in /volume/bucket format. ozone sh bucket create /hive/jan The above command will create a bucket called jan in the hive volume. Since no scheme was specified this command defaults to O3 (RPC) protocol.
+Delete The bucket delete commands allows an user to delete a volume.</description>
</item>
<item>
@@ -174,9 +170,10 @@ Params:
<guid>/keycommands.html</guid>
<description>Ozone shell supports the following key commands.
- get put delete info list Get The key get command downloads a key from Ozone cluster to local file system.
+get put delete info list Get The key get command downloads a key from Ozone cluster to local file system.
Params:
- Arguments Comment Uri The name of the key in /volume/bucket/key format. FileName Local file to download the key to. ozone sh key get /hive/jan/sales.orc sales.orc Downloads the file sales.</description>
+Arguments Comment Uri The name of the key in /volume/bucket/key format. FileName Local file to download the key to. ozone sh key get /hive/jan/sales.orc sales.orc Downloads the file sales.orc from the /hive/jan bucket and writes to the local file sales.orc.
+Put Uploads a file from the local file system to the specified bucket.</description>
</item>
<item>
@@ -188,7 +185,7 @@ Params:
<description>Ozone has a set of command line tools that can be used to manage ozone.
All these commands are invoked via the ozone script.
The commands supported by ozone are:
- classpath - Prints the class path needed to get the hadoop jar and the required libraries. fs - Runs a command on ozone file system. datanode - Via daemon command, the HDDS data nodes can be started or stopped.</description>
+classpath - Prints the class path needed to get the hadoop jar and the required libraries. fs - Runs a command on ozone file system. datanode - Via daemon command, the HDDS data nodes can be started or stopped. envvars - Display computed Hadoop environment variables.</description>
</item>
<item>
@@ -200,7 +197,7 @@ The commands supported by ozone are:
<description>The Ozone REST API&rsquo;s allows user to access ozone via REST protocol.
Authentication and Authorization For time being, The default authentication mode of REST API is insecure access mode, which is Simple mode. Under this mode, ozone server trusts the user name specified by client and it does not perform any authentication.
User name can be specified in HTTP header by
- x-ozone-user: {USER_NAME} for example if add following header x-ozone-user: bilbo in the HTTP request, then operation will be executed as bilbo user.</description>
+x-ozone-user: {USER_NAME} for example if add following header x-ozone-user: bilbo in the HTTP request, then operation will be executed as bilbo user.</description>
</item>
<item>
@@ -223,10 +220,10 @@ docker-compose up -d You can access the S3 gateway at http://localhost:9878</des
<guid>/volumecommands.html</guid>
<description>Volume commands generally need administrator privileges. The ozone shell supports the following volume commands.
- create delete info list update Create The volume create command allows an administrator to create a volume and assign it to a user.
+create delete info list update Create The volume create command allows an administrator to create a volume and assign it to a user.
Params:
- Arguments Comment -q, &ndash;quota Optional, This argument that specifies the maximum size this volume can use in the Ozone cluster. -u, &ndash;user Required, The name of the user who owns this volume.</description>
+Arguments Comment -q, &ndash;quota Optional, This argument that specifies the maximum size this volume can use in the Ozone cluster. -u, &ndash;user Required, The name of the user who owns this volume.</description>
</item>
</channel>
-</rss>
\ No newline at end of file
+</rss>
diff --git a/docs/0.3.0-alpha/javaapi.html b/docs/0.3.0-alpha/javaapi.html
index f00b003..9e9b7f7 100644
--- a/docs/0.3.0-alpha/javaapi.html
+++ b/docs/0.3.0-alpha/javaapi.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,155 +221,110 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Java API</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="introduction">Introduction</h2>
-
<p>Ozone ships with it own client library, that supports both RPC(Remote
Procedure call) and REST(Representational State Transfer). This library is
the primary user interface to ozone.</p>
-
<p>It is trivial to switch from RPC to REST or vice versa, by setting the
property <em>ozone.client.protocol</em> in the configuration or by calling the
appropriate factory method.</p>
-
<h2 id="creating-an-ozone-client">Creating an Ozone client</h2>
-
<p>The Ozone client factory creates the ozone client. It allows the user to
specify the protocol of communication. For example, to get an REST client, we
can use</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getRestClient</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getRestClient</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>And to get a RPC client we can call</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getRpcClient</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getRpcClient</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>If the user want to create a client based on the configuration, then they can
call</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>and an appropriate client based on configuration will be returned.</p>
-
<h2 id="writing-data-using-ozone-client">Writing data using Ozone Client</h2>
-
<p>The hierarchy of data inside ozone is a volume, bucket and a key. A volume
is a collection of buckets. A bucket is a collection of keys. To write data
to the ozone, you need a volume, bucket and a key.</p>
-
<h3 id="creating-a-volume">Creating a Volume</h3>
-
<p>Once we have a client, we need to get a reference to the ObjectStore. This
is done via</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>An object store represents an active cluster against which the client is working.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java"><span style="color:#75715e">// Let us create a volume to store our game assets.
-</span><span style="color:#75715e">// This uses default arguments for creating that volume.
-</span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Let us verify that the volume got created.
-</span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#75715e">// Let us create a volume to store our game assets.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e">// This uses default arguments for creating that volume.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us verify that the volume got created.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span></span></span></code></pre></div>
<p>It is possible to pass an array of arguments to the createVolume by creating volume arguments.</p>
-
<h3 id="creating-a-bucket">Creating a Bucket</h3>
-
<p>Once you have a volume, you can create buckets inside the volume.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">// Let us create a bucket called videos.
-assets.createBucket<span style="color:#f92672">(</span>“videos”<span style="color:#f92672">)</span>;
-Ozonebucket video <span style="color:#f92672">=</span> assets.getBucket<span style="color:#f92672">(</span>“videos”<span style="color:#f92672">)</span>;</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>// Let us create a bucket called videos.
+</span></span><span style="display:flex;"><span>assets.createBucket<span style="color:#f92672">(</span>“videos”<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>Ozonebucket video <span style="color:#f92672">=</span> assets.getBucket<span style="color:#f92672">(</span>“videos”<span style="color:#f92672">)</span>;</span></span></code></pre></div>
<p>At this point we have a usable volume and a bucket. Our volume is called assets and bucket is called videos.</p>
-
<p>Now we can create a Key.</p>
-
<h3 id="reading-and-writing-a-key">Reading and Writing a Key</h3>
-
<p>With a bucket object the users can now read and write keys. The following code reads a video called intro.mp4 from the local disk and stores in the video bucket that we just created.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">// read data from the file, this is a user provided <span style="color:#66d9ef">function</span>.
-byte <span style="color:#f92672">[]</span> vidoeData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span>“into.mp4”<span style="color:#f92672">)</span>;
-
-// Create an output stream and write data.
-OzoneOutputStream videoStream <span style="color:#f92672">=</span> video.createKey<span style="color:#f92672">(</span>“intro.mp4”, <span style="color:#ae81ff">1048576</span><span style="color:#f92672">)</span>;
-videoStream.write<span style="color:#f92672">(</span>videoData<span style="color:#f92672">)</span>;
-
-// Close the stream when it is <span style="color:#66d9ef">done</span>.
- videoStream.close<span style="color:#f92672">()</span>;
-
-
-// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
-// Let us allocate a byte array to hold the video first.
-byte<span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> new byte<span style="color:#f92672">[(</span>int<span style="color:#f92672">)</span><span style="color:#ae81ff">1048576</span><span style="color:#f92672">]</span>;
-OzoneInputStream introStream <span style="color:#f92672">=</span> video.readKey<span style="color:#f92672">(</span>“intro.mp4”<span style="color:#f92672">)</span>;
-// read intro.mp4 into the data buffer
-introStream.read<span style="color:#f92672">(</span>data<span style="color:#f92672">)</span>;
-introStream.close<span style="color:#f92672">()</span>;</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>// read data from the file, this is a user provided <span style="color:#66d9ef">function</span>.
+</span></span><span style="display:flex;"><span>byte <span style="color:#f92672">[]</span> vidoeData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span>“into.mp4”<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>// Create an output stream and write data.
+</span></span><span style="display:flex;"><span>OzoneOutputStream videoStream <span style="color:#f92672">=</span> video.createKey<span style="color:#f92672">(</span>“intro.mp4”, 1048576<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>videoStream.write<span style="color:#f92672">(</span>videoData<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>// Close the stream when it is <span style="color:#66d9ef">done</span>.
+</span></span><span style="display:flex;"><span> videoStream.close<span style="color:#f92672">()</span>;
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
+</span></span><span style="display:flex;"><span>// Let us allocate a byte array to hold the video first.
+</span></span><span style="display:flex;"><span>byte<span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> new byte<span style="color:#f92672">[(</span>int<span style="color:#f92672">)</span>1048576<span style="color:#f92672">]</span>;
+</span></span><span style="display:flex;"><span>OzoneInputStream introStream <span style="color:#f92672">=</span> video.readKey<span style="color:#f92672">(</span>“intro.mp4”<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>// read intro.mp4 into the data buffer
+</span></span><span style="display:flex;"><span>introStream.read<span style="color:#f92672">(</span>data<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>introStream.close<span style="color:#f92672">()</span>;</span></span></code></pre></div>
<p>Here is a complete example of the code that we just wrote. Please note the close functions being called in this program.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java"><span style="color:#75715e">// Let us create a client
-</span><span style="color:#75715e"></span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span>
-
-<span style="color:#75715e">// Get a reference to the ObjectStore using the client
-</span><span style="color:#75715e"></span>ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span>
-
-<span style="color:#75715e">// Let us create a volume to store our game assets.
-</span><span style="color:#75715e">// This default arguments for creating that volume.
-</span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Let us verify that the volume got created.
-</span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Let us create a bucket called videos.
-</span><span style="color:#75715e"></span>assets<span style="color:#f92672">.</span><span style="color:#a6e22e">createBucket</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>videos<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-Ozonebucket video <span style="color:#f92672">=</span> assets<span style="color:#f92672">.</span><span style="color:#a6e22e">getBucket</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>videos<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// read data from the file, this is assumed to be a user provided function.
-</span><span style="color:#75715e"></span><span style="color:#66d9ef">byte</span> <span style="color:#f92672">[]</span> vidoeData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>into<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Create an output stream and write data.
-</span><span style="color:#75715e"></span>OzoneOutputStream videoStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">createKey</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>intro<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">,</span> 1048576<span style="colo [...]
-videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">write</span><span style="color:#f92672">(</span>videoData<span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Close the stream when it is done.
-</span><span style="color:#75715e"></span> videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
-
-
-<span style="color:#75715e">// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
-</span><span style="color:#75715e">// Let us allocate a byte array to hold the video first.
-</span><span style="color:#75715e"></span>
-<span style="color:#66d9ef">byte</span><span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> <span style="color:#66d9ef">new</span> <span style="color:#66d9ef">byte</span><span style="color:#f92672">[(</span><span style="color:#66d9ef">int</span><span style="color:#f92672">)</span>1048576<span style="color:#f92672">];</span>
-OzoneInputStream introStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">readKey</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>into<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
-introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">read</span><span style="color:#f92672">(</span>data<span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Close the stream when it is done.
-</span><span style="color:#75715e"></span>introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
-
-<span style="color:#75715e">// Close the client.
-</span><span style="color:#75715e"></span>ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span></code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#75715e">// Let us create a client
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Get a reference to the ObjectStore using the client
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us create a volume to store our game assets.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e">// This default arguments for creating that volume.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us verify that the volume got created.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>assets<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us create a bucket called videos.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>assets<span style="color:#f92672">.</span><span style="color:#a6e22e">createBucket</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>videos<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>Ozonebucket video <span style="color:#f92672">=</span> assets<span style="color:#f92672">.</span><span style="color:#a6e22e">getBucket</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>videos<span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// read data from the file, this is assumed to be a user provided function.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">byte</span> <span style="color:#f92672">[]</span> vidoeData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>into<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Create an output stream and write data.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneOutputStream videoStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">createKey</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>intro<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style=" [...]
+</span></span><span style="display:flex;"><span>videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">write</span><span style="color:#f92672">(</span>videoData<span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Close the stream when it is done.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us allocate a byte array to hold the video first.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>
+</span></span><span style="display:flex;"><span><span style="color:#66d9ef">byte</span><span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> <span style="color:#66d9ef">new</span> <span style="color:#66d9ef">byte</span><span style="color:#f92672">[(</span><span style="color:#66d9ef">int</span><span style="color:#f92672">)</span><span style="color:#ae81ff">1048576</span><span style="color:#f92672">];</span>
+</span></span><span style="display:flex;"><span>OzoneInputStream introStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">readKey</span><span style="color:#f92672">(</span><span style="color:#960050;background-color:#1e0010">“</span>into<span style="color:#f92672">.</span><span style="color:#a6e22e">mp4</span><span style="color:#960050;background-color:#1e0010">”</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">read</span><span style="color:#f92672">(</span>data<span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Close the stream when it is done.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Close the client.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span></span></span></code></pre></div>
</div>
</div>
diff --git a/docs/0.3.0-alpha/keycommands.html b/docs/0.3.0-alpha/keycommands.html
index eb75b5f..149dbd3 100644
--- a/docs/0.3.0-alpha/keycommands.html
+++ b/docs/0.3.0-alpha/keycommands.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,27 +221,9 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Key Commands</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone shell supports the following key commands.</p>
-
<ul>
<li><a href="#get">get</a></li>
<li><a href="#put">put</a></li>
@@ -227,13 +231,9 @@
<li><a href="#info">info</a></li>
<li><a href="#list">list</a></li>
</ul>
-
<h3 id="get">Get</h3>
-
<p>The key get command downloads a key from Ozone cluster to local file system.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -241,30 +241,23 @@
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
<td>The name of the key in <strong>/volume/bucket/key</strong> format.</td>
</tr>
-
<tr>
<td>FileName</td>
<td>Local file to download the key to.</td>
</tr>
</tbody>
</table>
-
-<p><div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key get /hive/jan/sales.orc sales.orc</code></pre></div>
+<p><div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key get /hive/jan/sales.orc sales.orc</span></span></code></pre></div>
Downloads the file sales.orc from the <em>/hive/jan</em> bucket and writes to the
local file sales.orc.</p>
-
<h3 id="put">Put</h3>
-
<p>Uploads a file from the local file system to the specified bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -272,34 +265,26 @@ local file sales.orc.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
<td>The name of the key in <strong>/volume/bucket/key</strong> format.</td>
</tr>
-
<tr>
<td>FileName</td>
<td>Local file to upload.</td>
</tr>
-
<tr>
<td>-r, –replication</td>
<td>Optional, Number of copies, ONE or THREE are the options. Picks up the default from cluster configuration.</td>
</tr>
</tbody>
</table>
-
-<p><div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key put /hive/jan/corrected-sales.orc sales.orc</code></pre></div>
+<p><div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key put /hive/jan/corrected-sales.orc sales.orc</span></span></code></pre></div>
The above command will put the sales.orc as a new key into <em>/hive/jan/corrected-sales.orc</em>.</p>
-
<h3 id="delete">Delete</h3>
-
<p>The delete key command removes the key from the bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -307,7 +292,6 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -315,16 +299,11 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key delete /hive/jan/corrected-sales.orc</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key delete /hive/jan/corrected-sales.orc</span></span></code></pre></div>
<p>The above command deletes the key <em>/hive/jan/corrected-sales.orc</em>.</p>
-
<h3 id="info">Info</h3>
-
<p>The key info commands returns the information about the key.
-<strong><em>Params:</em></strong></p>
-
+<em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -332,7 +311,6 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -340,18 +318,12 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key info /hive/jan/sales.orc</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key info /hive/jan/sales.orc</span></span></code></pre></div>
<p>The above command will print out the information about <em>/hive/jan/sales.orc</em>
key.</p>
-
<h3 id="list">List</h3>
-
<p>The key list commands allows user to list all keys in a bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -359,34 +331,27 @@ key.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>-l, –length</td>
<td>Maximum number of results to return. Default: 1000</td>
</tr>
-
<tr>
<td>-p, –prefix</td>
<td>Optional, Only buckets that match this prefix will be returned.</td>
</tr>
-
<tr>
<td>-s, –start</td>
<td>The listing will start from key after the start key.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the <em>volume</em>.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key list /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key list /hive/jan</span></span></code></pre></div>
<p>This command will list all key in the bucket <em>/hive/jan</em>.</p>
-
<p>You can try out these commands from the docker instance of the <a href="runningviadocker.html">Alpha
Cluster</a>.</p>
diff --git a/docs/0.3.0-alpha/ozonefs.html b/docs/0.3.0-alpha/ozonefs.html
index 36df0be..cceeab0 100644
--- a/docs/0.3.0-alpha/ozonefs.html
+++ b/docs/0.3.0-alpha/ozonefs.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,64 +221,32 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Ozone File System</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>There are many Hadoop compatible files systems under Hadoop. Hadoop compatible file systems ensures that storage backends like Ozone can easily be integrated into Hadoop eco-system.</p>
-
<h2 id="setting-up-the-ozone-file-system">Setting up the Ozone file system</h2>
-
<p>To create an ozone file system, we have to choose a bucket where the file system would live. This bucket will be used as the backend store for OzoneFileSystem. All the files and directories will be stored as keys in this bucket.</p>
-
<p>Please run the following commands to create a volume and bucket, if you don’t have them already.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume create /volume
-ozone sh bucket create /volume/bucket</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume create /volume
+</span></span><span style="display:flex;"><span>ozone sh bucket create /volume/bucket</span></span></code></pre></div>
<p>Once this is created, please make sure that bucket exists via the listVolume or listBucket commands.</p>
-
<p>Please add the following entry to the core-site.xml.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>fs.o3fs.impl<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>org.apache.hadoop.fs.ozone.OzoneFileSystem<span style="color:#f92672"></value></span>
-<span style="color:#f92672"></property></span>
-<span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>fs.defaultFS<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>o3fs://localhost:9864/volume/bucket<span style="color:#f92672"></value></span>
-<span style="color:#f92672"></property></span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span><span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>fs.o3fs.impl<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>org.apache.hadoop.fs.ozone.OzoneFileSystem<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"></property></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>fs.defaultFS<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>o3fs://localhost:9864/volume/bucket<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"></property></span></span></span></code></pre></div>
<p>This will make this bucket to be the default file system for HDFS dfs commands and register the o3fs file system type..</p>
-
<p>You also need to add the ozone-filesystem.jar file to the classpath:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">export HADOOP_CLASSPATH<span style="color:#f92672">=</span>/opt/ozone/share/hadoop/ozonefs/hadoop-ozone-filesystem.jar:$HADOOP_CLASSPATH</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>export HADOOP_CLASSPATH<span style="color:#f92672">=</span>/opt/ozone/share/hadoop/ozonefs/hadoop-ozone-filesystem.jar:$HADOOP_CLASSPATH</span></span></code></pre></div>
<p>Once the default Filesystem has been setup, users can run commands like ls, put, mkdir, etc.
For example,</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">hdfs dfs -ls /</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>hdfs dfs -ls /</span></span></code></pre></div>
<p>or</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">hdfs dfs -mkdir /users</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>hdfs dfs -mkdir /users</span></span></code></pre></div>
<p>Or put command etc. In other words, all programs like Hive, Spark, and Distcp will work against this file system.
Please note that any keys created/deleted in the bucket using methods apart from OzoneFileSystem will show up as diectories and files in the Ozone File System.</p>
diff --git a/docs/0.3.0-alpha/ozonemanager.html b/docs/0.3.0-alpha/ozonemanager.html
index 170f9da..3979167 100644
--- a/docs/0.3.0-alpha/ozonemanager.html
+++ b/docs/0.3.0-alpha/ozonemanager.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,72 +221,33 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Ozone Manager</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="om-overview">OM Overview</h2>
-
<p>Ozone Manager or OM is the namespace manager for Ozone. The clients (RPC clients, Rest proxy, Ozone file system, etc.) communicate with OM to create and delete various ozone objects.</p>
-
<p>Each ozone volume is the root of a namespace under OM. This is very different from HDFS which provides a single rooted file system.</p>
-
<p>Ozone’s namespace is a collection of volumes or is a forest instead of a
single rooted tree as in HDFS. This property makes it easy to deploy multiple
- OMs for scaling, this feature is under development.</p>
-
+OMs for scaling, this feature is under development.</p>
<h2 id="om-metadata">OM Metadata</h2>
-
<p>Conceptually, OM maintains a list of volumes, buckets, and keys. For each user, it maintains a list of volumes. For each volume, the list of buckets and for each bucket the list of keys.</p>
-
<p>Right now, OM is a single instance service. Ozone already relies on Apache Ratis (A Replicated State Machine based on Raft protocol). OM will be extended to replicate all its metadata via Ratis. With that, OM will be highly available.</p>
-
<h2 id="om-ui">OM UI</h2>
-
-<p>OM supports a simple UI for the time being. The default port of OM is 9874. To access the OM UI, the user can connect to <a href="http://OM:port">http://OM:port</a> or for a concrete example,</p>
-
-<pre><code>http://omserver:9874/
-</code></pre>
-
-<p>OM UI primarily tries to measure load and latency of OM. The first section of OM UI relates to the number of operations seen by the cluster broken down by the object, operation and whether the operation was successful.</p>
-
+<p>OM supports a simple UI for the time being. The default port of OM is 9874. To access the OM UI, the user can connect to http://OM:port or for a concrete example,</p>
+<pre tabindex="0"><code>http://omserver:9874/
+</code></pre><p>OM UI primarily tries to measure load and latency of OM. The first section of OM UI relates to the number of operations seen by the cluster broken down by the object, operation and whether the operation was successful.</p>
<p>The latter part of the UI is focused on latency and number of operations that OM is performing.</p>
-
<p>One of the hardest problems in HDFS world is discovering the numerous settings offered to tune HDFS. Ozone solves that problem by tagging the configs. To discover settings, click on “Common Tools”->Config. This will take you to the ozone config UI.</p>
-
<h2 id="config-ui">Config UI</h2>
-
<p>The ozone config UI is a matrix with row representing the tags, and columns representing All, OM and SCM.</p>
-
<p>Suppose a user wanted to discover the required settings for ozone. Then the user can tick the checkbox that says “Required.”
This will filter out all “Required” settings along with the description of what each setting does.</p>
-
<p>The user can combine different checkboxes and UI will combine the results. That is, If you have more than one row selected, then all keys for those chosen tags are displayed together.</p>
-
<p>We are hopeful that this leads to a more straightforward way of discovering settings that manage ozone.</p>
-
<h2 id="om-and-scm">OM and SCM</h2>
-
<p><a href="./hdds.html">Storage container manager</a> or (SCM) is the block manager
- for ozone. When a client requests OM for a set of data nodes to write data, OM talks to SCM and gets a block.</p>
-
+for ozone. When a client requests OM for a set of data nodes to write data, OM talks to SCM and gets a block.</p>
<p>A block returned by SCM contains a pipeline, which is a set of nodes that we participate in that block replication.</p>
-
<p>So OM is dependent on SCM for reading and writing of Keys. However, OM is independent of SCM while doing metadata operations like ozone volume or bucket operations.</p>
</div>
diff --git a/docs/0.3.0-alpha/realcluster.html b/docs/0.3.0-alpha/realcluster.html
index 17563c3..6827e5e 100644
--- a/docs/0.3.0-alpha/realcluster.html
+++ b/docs/0.3.0-alpha/realcluster.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,56 +221,30 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Starting an Ozone Cluster</h1>
<div class="col-md-9">
- <!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Before we boot up the Ozone cluster, we need to initialize both SCM and Ozone Manager.</p>
-
-<p><div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone scm --init</code></pre></div>
+<p><div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone scm --init</span></span></code></pre></div>
This allows SCM to create the cluster Identity and initialize its state.
The <code>init</code> command is similar to Namenode format. Init command is executed only once, that allows SCM to create all the required on-disk structures to work correctly.
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start scm</code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone --daemon start scm</span></span></code></pre></div></p>
<p>Once we know SCM is up and running, we can create an Object Store for our use. This is done by running the following command.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone om --init</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone om --init</span></span></code></pre></div>
<p>Once Ozone manager has created the Object Store, we are ready to run the name
services.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start om</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone --daemon start om</span></span></code></pre></div>
<p>At this point Ozone’s name services, the Ozone manager, and the block service SCM is both running.
<strong>Please note</strong>: If SCM is not running
<code>om --init</code> command will fail. SCM start will fail if on-disk data structures are missing. So please make sure you have done both <code>scm --init</code> and <code>om --init</code> commands.</p>
-
<p>Now we need to start the data nodes. Please run the following command on each datanode.
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start datanode</code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone --daemon start datanode</span></span></code></pre></div></p>
<p>At this point SCM, Ozone Manager and data nodes are up and running.</p>
-
-<p><strong><em>Congratulations!, You have set up a functional ozone cluster.</em></strong></p>
-
-<hr />
-
+<p><em><strong>Congratulations!, You have set up a functional ozone cluster.</strong></em></p>
+<hr>
<p>If you want to make your life simpler, you can just run
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone scm --init
-ozone om --init
-start-ozone.sh</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone scm --init
+</span></span><span style="display:flex;"><span>ozone om --init
+</span></span><span style="display:flex;"><span>start-ozone.sh</span></span></code></pre></div>
This assumes that you have set up the slaves file correctly and ssh
configuration that allows ssh-ing to all data nodes. This is the same as the
HDFS configuration, so please refer to HDFS documentation on how to set this
diff --git a/docs/0.3.0-alpha/rest.html b/docs/0.3.0-alpha/rest.html
index 525ce2d..f709f4b 100644
--- a/docs/0.3.0-alpha/rest.html
+++ b/docs/0.3.0-alpha/rest.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,150 +221,102 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>REST API</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>The Ozone REST API’s allows user to access ozone via REST protocol.</p>
-
<h2 id="authentication-and-authorization">Authentication and Authorization</h2>
-
<p>For time being, The default authentication mode of REST API is insecure access
mode, which is <em>Simple</em> mode. Under this mode, ozone server trusts the user
name specified by client and it does not perform any authentication.</p>
-
<p>User name can be specified in HTTP header by</p>
-
<ul>
<li><code>x-ozone-user: {USER_NAME}</code></li>
</ul>
-
<p>for example if add following header <em>x-ozone-user: bilbo</em> in the HTTP request,
then operation will be executed as <em>bilbo</em> user.
In <em>Simple</em> mode, there is no real authorization either. Client can be
authorized to obtain administrator privilege by using HTTP header</p>
-
<ul>
<li><code>Authorization: {AUTH_METHOD} {SIGNATURE}</code></li>
</ul>
-
<p>for example set following header <em>Authorization: OZONE root</em> in the HTTP request,
then ozone will authorize the client with administrator privilege.</p>
-
<h2 id="common-rest-headers">Common REST Headers</h2>
-
<p>The following HTTP headers must be set for each REST call.</p>
-
<table>
<thead>
<tr>
-<th align="left">Property</th>
-<th align="left">Description</th>
+<th style="text-align:left">Property</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">Authorization</td>
-<td align="left">The authorization field determines which authentication method is used by ozone. Currently only <em>simple</em> mode is supported, the corresponding value is <em>OZONE</em>. Optionally an user name can be set as <em>OZONE {USER_NAME}</em> to authorize as a particular user.</td>
+<td style="text-align:left">Authorization</td>
+<td style="text-align:left">The authorization field determines which authentication method is used by ozone. Currently only <em>simple</em> mode is supported, the corresponding value is <em>OZONE</em>. Optionally an user name can be set as <em>OZONE {USER_NAME}</em> to authorize as a particular user.</td>
</tr>
-
<tr>
-<td align="left">Date</td>
-<td align="left">Standard HTTP header that represents dates. The format is - day of the week, month, day, year and time (military time format) in GMT. Any other time zone will be rejected by ozone server. Eg. <em>Date : Mon, Apr 4, 2016 06:22:00 GMT</em>. This field is required.</td>
+<td style="text-align:left">Date</td>
+<td style="text-align:left">Standard HTTP header that represents dates. The format is - day of the week, month, day, year and time (military time format) in GMT. Any other time zone will be rejected by ozone server. Eg. <em>Date : Mon, Apr 4, 2016 06:22:00 GMT</em>. This field is required.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-version</td>
-<td align="left">A required HTTP header to indicate which version of API this call will be communicating to. E.g <em>x-ozone-version: v1</em>. Currently ozone only publishes v1 version API.</td>
+<td style="text-align:left">x-ozone-version</td>
+<td style="text-align:left">A required HTTP header to indicate which version of API this call will be communicating to. E.g <em>x-ozone-version: v1</em>. Currently ozone only publishes v1 version API.</td>
</tr>
</tbody>
</table>
-
<h2 id="common-reply-headers">Common Reply Headers</h2>
-
<p>The common reply headers are part of all Ozone server replies.</p>
-
<table>
<thead>
<tr>
-<th align="left">Property</th>
-<th align="left">Description</th>
+<th style="text-align:left">Property</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">Date</td>
-<td align="left">This is the HTTP date header and it is set to server’s local time expressed in GMT.</td>
+<td style="text-align:left">Date</td>
+<td style="text-align:left">This is the HTTP date header and it is set to server’s local time expressed in GMT.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-request-id</td>
-<td align="left">This is a UUID string that represents an unique request ID. This ID is used to track the request through the ozone system and is useful for debugging purposes.</td>
+<td style="text-align:left">x-ozone-request-id</td>
+<td style="text-align:left">This is a UUID string that represents an unique request ID. This ID is used to track the request through the ozone system and is useful for debugging purposes.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-server-name</td>
-<td align="left">Fully qualified domain name of the sever which handled the request.</td>
+<td style="text-align:left">x-ozone-server-name</td>
+<td style="text-align:left">Fully qualified domain name of the sever which handled the request.</td>
</tr>
</tbody>
</table>
-
<h2 id="volume-apis">Volume APIs</h2>
-
<h3 id="create-a-volume">Create a Volume</h3>
-
<p>This API allows admins to create a new storage volume.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>POST /{volume}?quota=<VOLUME_QUOTA></code></li>
</ul>
-
<p>Query Parameter:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">quota</td>
-<td align="left">long<BYTES \| MB \| GB \| TB></td>
-<td align="left">Optional. Quota size in BYTEs, MBs, GBs or TBs</td>
+<td style="text-align:left">quota</td>
+<td style="text-align:left">long<BYTES | MB | GB | TB></td>
+<td style="text-align:left">Optional. Quota size in BYTEs, MBs, GBs or TBs</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP POST request:</p>
-
<pre><code>curl -i -X POST -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE root" "http://localhost:9880/volume-to-create"
</code></pre>
-
<p>this request creates a volume as user <em>bilbo</em>, the authorization field is set to <em>OZONE root</em> because this call requires administration privilege. The client receives a response with zero content length.</p>
-
<pre><code>HTTP/1.1 201 Created
x-ozone-server-name: localhost
x-ozone-request-id: 2173deb5-bbb7-4f0a-8236-f354784e3bae
@@ -351,61 +325,43 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="update-volume">Update Volume</h3>
-
<p>This API allows administrators to update volume info such as ownership and quota. This API requires administration privilege.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>PUT /{volume}?quota=<VOLUME_QUOTA></code></li>
</ul>
-
<p>Query Parameter:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">quota</td>
-<td align="left">long<BYTES \| MB \| GB \| TB> | remove</td>
-<td align="left">Optional. Quota size in BYTEs, MBs, GBs or TBs. Or use string value <em>remove</em> to remove an existing quota for a volume.</td>
+<td style="text-align:left">quota</td>
+<td style="text-align:left">long<BYTES | MB | GB | TB> | remove</td>
+<td style="text-align:left">Optional. Quota size in BYTEs, MBs, GBs or TBs. Or use string value <em>remove</em> to remove an existing quota for a volume.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP PUT request:</p>
-
<pre><code>curl -X PUT -H "Authorization:OZONE root" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "x-ozone-version: v1" -H "x-ozone-user: john" http://localhost:9880/volume-to-update
</code></pre>
-
<p>this request modifies the owner of <em>/volume-to-update</em> to <em>john</em>.</p>
-
<h3 id="delete-volume">Delete Volume</h3>
-
<p>This API allows user to delete a volume owned by themselves if the volume is not empty. Administrators can delete volumes owned by any user.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>DELETE /{volume}</code></li>
</ul>
-
<p>Sample HTTP DELETE request:</p>
-
<pre><code>curl -i -X DELETE -H "Authorization:OZONE root" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "x-ozone-version: v1" -H "x-ozone-user: bilbo" http://localhost:9880/volume-to-delete
</code></pre>
-
<p>this request deletes an empty volume <em>/volume-to-delete</em>. The client receives a zero length content.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: 6af14c64-e3a9-40fe-9634-df60b7cbbc6a
@@ -414,44 +370,33 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="info-volume">Info Volume</h3>
-
<p>This API allows user to read the info of a volume owned by themselves. Administrators can read volume info owned by any user.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}?info=volume</code></li>
</ul>
-
<p>Query Parameter:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">info</td>
-<td align="left">“volume”</td>
-<td align="left">Required and enforced with this value.</td>
+<td style="text-align:left">info</td>
+<td style="text-align:left">“volume”</td>
+<td style="text-align:left">Required and enforced with this value.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo?info=volume"
</code></pre>
-
<p>this request gets the info of volume <em>/volume-of-bilbo</em>, the client receives a response with a JSON object of volume info.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: a2224806-beaf-42dd-a68e-533cd7508f74
@@ -468,62 +413,48 @@ Connection: keep-alive
"createdBy" : "root"
}
</code></pre>
-
<h3 id="list-volumes">List Volumes</h3>
-
<p>This API allows user to list all volumes owned by themselves. Administrators can list all volumes owned by any user.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /?prefix=<PREFIX>&max-keys=<MAX_RESULT_SIZE>&prev-key=<PREVIOUS_VOLUME_KEY></code></li>
</ul>
-
<p>Query Parameter:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">prefix</td>
-<td align="left">string</td>
-<td align="left">Optional. Only volumes with this prefix are included in the result.</td>
+<td style="text-align:left">prefix</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Only volumes with this prefix are included in the result.</td>
</tr>
-
<tr>
-<td align="left">max-keys</td>
-<td align="left">int</td>
-<td align="left">Optional. Maximum number of volumes included in the result. Default is 1024 if not specified.</td>
+<td style="text-align:left">max-keys</td>
+<td style="text-align:left">int</td>
+<td style="text-align:left">Optional. Maximum number of volumes included in the result. Default is 1024 if not specified.</td>
</tr>
-
<tr>
-<td align="left">prev-key</td>
-<td align="left">string</td>
-<td align="left">Optional. Volume name from where listing should start, this key is excluded in the result. It must be a valid volume name.</td>
+<td style="text-align:left">prev-key</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Volume name from where listing should start, this key is excluded in the result. It must be a valid volume name.</td>
</tr>
-
<tr>
-<td align="left">root-scan</td>
-<td align="left">bool</td>
-<td align="left">Optional. List all volumes in the cluster if this is set to true. Default false.</td>
+<td style="text-align:left">root-scan</td>
+<td style="text-align:left">bool</td>
+<td style="text-align:left">Optional. List all volumes in the cluster if this is set to true. Default false.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/?max-keys=100&prefix=Jan"
</code></pre>
-
<p>this request gets all volumes owned by <em>bilbo</em> and each volume’s name contains prefix <em>Jan</em>, the result at most contains <em>100</em> entries. The client receives a list of SON objects, each of them describes the info of a volume.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: 7fa0dce1-a8bd-4387-bc3c-1dac4b710bb1
@@ -545,58 +476,44 @@ Connection: keep-alive
]
}
</code></pre>
-
<h2 id="bucket-apis">Bucket APIs</h2>
-
<h3 id="create-bucket">Create Bucket</h3>
-
<p>This API allows an user to create a bucket in a volume.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>POST /{volume}/{bucket}</code></li>
</ul>
-
<p>Additional HTTP Headers:</p>
-
<table>
<thead>
<tr>
-<th align="left">HTTP Header</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">HTTP Header</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">x-ozone-acl</td>
-<td align="left">ozone ACLs</td>
-<td align="left">Optional. Ozone acls.</td>
+<td style="text-align:left">x-ozone-acl</td>
+<td style="text-align:left">ozone ACLs</td>
+<td style="text-align:left">Optional. Ozone acls.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-storage-class</td>
-<td align="left"><DEFAULT \| ARCHIVE \| DISK \| RAM_DISK \| SSD ></td>
-<td align="left">Optional. Storage type for a volume.</td>
+<td style="text-align:left">x-ozone-storage-class</td>
+<td style="text-align:left"><DEFAULT | ARCHIVE | DISK | RAM_DISK | SSD ></td>
+<td style="text-align:left">Optional. Storage type for a volume.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-bucket-versioning</td>
-<td align="left">enabled/disabled</td>
-<td align="left">Optional. Do enable bucket versioning or not.</td>
+<td style="text-align:left">x-ozone-bucket-versioning</td>
+<td style="text-align:left">enabled/disabled</td>
+<td style="text-align:left">Optional. Do enable bucket versioning or not.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP POST request:</p>
-
<pre><code>curl -i -X POST -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" http://localhost:9880/volume-of-bilbo/bucket-0
</code></pre>
-
<p>this request creates a bucket <em>bucket-0</em> under volume <em>volume-of-bilbo</em>.</p>
-
<pre><code>HTTP/1.1 201 Created
x-ozone-server-name: localhost
x-ozone-request-id: 49acfeec-4c85-470a-872b-2eaebd8d751e
@@ -605,50 +522,38 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="update-bucket">Update Bucket</h3>
-
<p>Updates bucket meta-data, like ACLs.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>PUT /{volume}/{bucket}</code></li>
</ul>
-
<p>Additional HTTP Headers:</p>
-
<table>
<thead>
<tr>
-<th align="left">HTTP Header</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">HTTP Header</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">x-ozone-acl</td>
-<td align="left">ozone ACLs</td>
-<td align="left">Optional. Ozone acls.</td>
+<td style="text-align:left">x-ozone-acl</td>
+<td style="text-align:left">ozone ACLs</td>
+<td style="text-align:left">Optional. Ozone acls.</td>
</tr>
-
<tr>
-<td align="left">x-ozone-bucket-versioning</td>
-<td align="left">enabled/disabled</td>
-<td align="left">Optional. Do enable bucket versioning or not.</td>
+<td style="text-align:left">x-ozone-bucket-versioning</td>
+<td style="text-align:left">enabled/disabled</td>
+<td style="text-align:left">Optional. Do enable bucket versioning or not.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP PUT request:</p>
-
<pre><code>curl -i -X PUT -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" -H "x-ozone-acl: ADD user:peregrin:rw" http://localhost:9880/volume-of-bilbo/bucket-to-update
</code></pre>
-
<p>this request adds an ACL policy specified by HTTP header <em>x-ozone-acl</em> to bucket <em>/volume-of-bilbo/bucket-to-update</em>, the ACL field <em>ADD user:peregrin:rw</em> gives add additional read/write permission to user <em>peregrin</em> to this bucket.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: b061a295-5faf-4b98-94b9-8b3e87c8eb5e
@@ -657,24 +562,16 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="delete-bucket">Delete Bucket</h3>
-
<p>Deletes a bucket if it is empty. An user can only delete bucket owned by themselves, and administrators can delete buckets owned by any user, as long as it is empty.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>DELETE /{volume}/{bucket}</code></li>
</ul>
-
<p>Sample HTTP DELETE request:</p>
-
<pre><code>curl -i -X DELETE -H "Authorization:OZONE root" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "x-ozone-version: v1" -H "x-ozone-user:bilbo" "http://localhost:9880/volume-of-bilbo/bucket-0"
</code></pre>
-
<p>this request deletes bucket <em>/volume-of-bilbo/bucket-0</em>. The client receives a zero length content response.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: f57acd7a-2116-4c2f-aa2f-5a483db81c9c
@@ -683,44 +580,33 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="info-bucket">Info Bucket</h3>
-
<p>This API returns information about a given bucket.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}/{bucket}?info=bucket</code></li>
</ul>
-
<p>Query Parameters:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">info</td>
-<td align="left">“bucket”</td>
-<td align="left">Required and enforced with this value.</td>
+<td style="text-align:left">info</td>
+<td style="text-align:left">“bucket”</td>
+<td style="text-align:left">Required and enforced with this value.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo/bucket-0?info=bucket"
</code></pre>
-
<p>this request gets the info of bucket <em>/volume-of-bilbo/bucket-0</em>. The client receives a response of JSON object contains bucket info.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: f125485b-8cae-4c7f-a2d6-5b1fefd6f193
@@ -738,56 +624,43 @@ Connection: keep-alive
"storageType" : "DISK"
}
</code></pre>
-
<h3 id="list-buckets">List Buckets</h3>
-
<p>List buckets in a given volume.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}?prefix=<PREFIX>&max-keys=<MAX_RESULT_SIZE>&prev-key=<PREVIOUS_BUCKET_KEY></code></li>
</ul>
-
<p>Query Parameters:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">prefix</td>
-<td align="left">string</td>
-<td align="left">Optional. Only buckets with this prefix are included in the result.</td>
+<td style="text-align:left">prefix</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Only buckets with this prefix are included in the result.</td>
</tr>
-
<tr>
-<td align="left">max-keys</td>
-<td align="left">int</td>
-<td align="left">Optional. Maximum number of buckets included in the result. Default is 1024 if not specified.</td>
+<td style="text-align:left">max-keys</td>
+<td style="text-align:left">int</td>
+<td style="text-align:left">Optional. Maximum number of buckets included in the result. Default is 1024 if not specified.</td>
</tr>
-
<tr>
-<td align="left">prev-key</td>
-<td align="left">string</td>
-<td align="left">Optional. Bucket name from where listing should start, this key is excluded in the result. It must be a valid bucket name.</td>
+<td style="text-align:left">prev-key</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Bucket name from where listing should start, this key is excluded in the result. It must be a valid bucket name.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo?max-keys=10"
</code></pre>
-
<p>this request lists all the buckets under volume <em>volume-of-bilbo</em>, and the result at most contains 10 entries. The client receives response of a array of JSON objects, each of them represents for a bucket info.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: e048c3d5-169c-470f-9903-632d9f9e32d5
@@ -811,63 +684,44 @@ Connection: keep-alive
]
}
</code></pre>
-
<h2 id="key-apis">Key APIs</h2>
-
<h3 id="put-key">Put Key</h3>
-
<p>This API allows user to create or overwrite keys inside of a bucket.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>PUT /{volume}/{bucket}/{key}</code></li>
</ul>
-
<p>Additional HTTP headers:</p>
-
<table>
<thead>
<tr>
-<th align="left">HTTP Header</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">HTTP Header</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">Content-MD5</td>
-<td align="left">MD5 digest</td>
-<td align="left">Standard HTTP header, file hash.</td>
+<td style="text-align:left">Content-MD5</td>
+<td style="text-align:left">MD5 digest</td>
+<td style="text-align:left">Standard HTTP header, file hash.</td>
</tr>
</tbody>
</table>
-
<p>Sample PUT HTTP request:</p>
-
<pre><code>curl -X PUT -T /path/to/localfile -H "Authorization:OZONE" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "x-ozone-version: v1" -H "x-ozone-user:bilbo" "http://localhost:9880/volume-of-bilbo/bucket-0/file-0"
</code></pre>
-
<p>this request uploads a local file <em>/path/to/localfile</em> specified by option <em>-T</em> to ozone as user <em>bilbo</em>, mapped to ozone key <em>/volume-of-bilbo/bucket-0/file-0</em>. The client receives a zero length content response.</p>
-
<h3 id="get-key">Get Key</h3>
-
<p>This API allows user to get or download a key from an ozone bucket.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}/{bucket}/{key}</code></li>
</ul>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo/bucket-0/file-0"
</code></pre>
-
<p>this request reads the content of key <em>/volume-of-bilbo/bucket-0/file-0</em>. If the content of the file is plain text, it can be directly dumped onto stdout.</p>
-
<pre><code>HTTP/1.1 200 OK
Content-Type: application/octet-stream
x-ozone-server-name: localhost
@@ -878,36 +732,24 @@ Connection: keep-alive
Hello Ozone!
</code></pre>
-
<p>if the file is not plain text, specify <em>-O</em> option in curl command and the file <em>file-0</em> will be downloaded into current working directory, file name will be same as the key. A sample request like following:</p>
-
<pre><code>curl -O -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo/bucket-0/file-1"
</code></pre>
-
<p>response looks like following:</p>
-
<pre><code>% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6148k 100 6148k 0 0 24.0M 0 --:--:-- --:--:-- --:--:-- 24.1M
</code></pre>
-
<h3 id="delete-key">Delete Key</h3>
-
<p>This API allows user to delete a key from a bucket.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>DELETE /{volume}/{bucket}/{key}</code></li>
</ul>
-
<p>Sample HTTP DELETE request:</p>
-
<pre><code>curl -i -X DELETE -H "Authorization:OZONE root" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "x-ozone-version: v1" -H "x-ozone-user:bilbo" "http://localhost:9880/volume-of-bilbo/bucket-0/file-0"
</code></pre>
-
<p>this request deletes key <em>/volume-of-bilbo/bucket-0/file-0</em>. The client receives a zero length content result:</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: f8c4a373-dd5f-4e3a-b6c4-ddf7e191fe91
@@ -916,44 +758,33 @@ Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
</code></pre>
-
<h3 id="info-key">Info Key</h3>
-
<p>This API returns information about a given key.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}/{bucket}/{key}?info=key</code></li>
</ul>
-
<p>Query Parameter:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">info</td>
-<td align="left">String, “key”</td>
-<td align="left">Required and enforced with this value.</td>
+<td style="text-align:left">info</td>
+<td style="text-align:left">String, “key”</td>
+<td style="text-align:left">Required and enforced with this value.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP DELETE request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http://localhost:9880/volume-of-bilbo/buket-0/file-0?info=key"
</code></pre>
-
<p>this request returns information of the key <em>/volume-of-bilbo/bucket-0/file-0</em>. The client receives a JSON object listed attributes of the key.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: c674343c-a0f2-49e4-bbd6-daa73e7dc131
@@ -971,56 +802,43 @@ Connection: keep-alive
"keyName" : "file-0"
}
</code></pre>
-
<h3 id="list-keys">List Keys</h3>
-
<p>This API allows user to list keys in a bucket.</p>
-
<p>Schema:</p>
-
<ul>
<li><code>GET /{volume}/{bucket}?prefix=<PREFIX>&max-keys=<MAX_RESULT_SIZE>&prev-key=<PREVIOUS_KEY></code></li>
</ul>
-
<p>Query Parameters:</p>
-
<table>
<thead>
<tr>
-<th align="left">Query Parameter</th>
-<th align="left">Value</th>
-<th align="left">Description</th>
+<th style="text-align:left">Query Parameter</th>
+<th style="text-align:left">Value</th>
+<th style="text-align:left">Description</th>
</tr>
</thead>
-
<tbody>
<tr>
-<td align="left">prefix</td>
-<td align="left">string</td>
-<td align="left">Optional. Only keys with this prefix are included in the result.</td>
+<td style="text-align:left">prefix</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Only keys with this prefix are included in the result.</td>
</tr>
-
<tr>
-<td align="left">max-keys</td>
-<td align="left">int</td>
-<td align="left">Optional. Maximum number of keys included in the result. Default is 1024 if not specified.</td>
+<td style="text-align:left">max-keys</td>
+<td style="text-align:left">int</td>
+<td style="text-align:left">Optional. Maximum number of keys included in the result. Default is 1024 if not specified.</td>
</tr>
-
<tr>
-<td align="left">prev-key</td>
-<td align="left">string</td>
-<td align="left">Optional. Key name from where listing should start, this key is excluded in the result. It must be a valid key name.</td>
+<td style="text-align:left">prev-key</td>
+<td style="text-align:left">string</td>
+<td style="text-align:left">Optional. Key name from where listing should start, this key is excluded in the result. It must be a valid key name.</td>
</tr>
</tbody>
</table>
-
<p>Sample HTTP GET request:</p>
-
<pre><code>curl -i -H "x-ozone-user: bilbo" -H "x-ozone-version: v1" -H "Date: Mon, 26 Jun 2017 04:23:30 GMT" -H "Authorization:OZONE" "http:/localhost:9880/volume-of-bilbo/bucket-0/?max-keys=100&prefix=file"
</code></pre>
-
<p>this request list keys under bucket <em>/volume-of-bilbo/bucket-0</em>, the listing result is filtered by prefix <em>file</em>. The client receives an array of JSON objects, each of them represents the info of a matched key.</p>
-
<pre><code>HTTP/1.1 200 OK
x-ozone-server-name: localhost
x-ozone-request-id: 7f9fc970-9904-4c56-b671-83a086c6f555
diff --git a/docs/0.3.0-alpha/runningviadocker.html b/docs/0.3.0-alpha/runningviadocker.html
index 47b5308..0109514 100644
--- a/docs/0.3.0-alpha/runningviadocker.html
+++ b/docs/0.3.0-alpha/runningviadocker.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,66 +221,42 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Alpha Cluster</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-<p><strong><em>This is an alpha release of Ozone. Please don’t use this release in
-production.</em></strong> Please check the road map page for features under
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
+<p><em><strong>This is an alpha release of Ozone. Please don’t use this release in
+production.</strong></em> Please check the road map page for features under
development.</p>
-
<p>The easiest way to run ozone is to download the release tarball and launch
ozone via Docker. Docker will create a small ozone cluster on your machine,
including the data nodes and ozone services.</p>
-
<h2 id="running-ozone-via-docker">Running Ozone via Docker</h2>
-
<p><strong>This assumes that you have Docker installed on the machine.</strong></p>
-
<ul>
-<li><p>Download the Ozone tarball and untar it.</p></li>
-
-<li><p>Go to the directory where the docker compose files exist and tell
+<li>
+<p>Download the Ozone tarball and untar it.</p>
+</li>
+<li>
+<p>Go to the directory where the docker compose files exist and tell
<code>docker-compose</code> to start Ozone in the background. This will start a small
-ozone instance on your machine.</p></li>
+ozone instance on your machine.</p>
+</li>
</ul>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">cd ozone-0.2.1-SNAPSHOT/compose/ozone/
-
-docker-compose up -d</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>cd ozone-0.2.1-SNAPSHOT/compose/ozone/
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>docker-compose up -d</span></span></code></pre></div>
<p>To verify that ozone is working as expected, let us log into a data node and
run <em>freon</em>, the load generator for Ozone. The <code>exec datanode bash</code> command
will open a bash shell on the datanode. The ozone freon command is executed
within the datanode container. You can quit freon via CTRL-C any time. The
<code>rk</code> profile instructs freon to generate random keys.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose exec datanode bash
-ozone freon rk</code></pre></div>
-
-<p>You can check out the <strong>OzoneManager UI</strong> at <a href="http://localhost:9874/">http://localhost:9874/</a> to see the
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose exec datanode bash
+</span></span><span style="display:flex;"><span>ozone freon rk</span></span></code></pre></div>
+<p>You can check out the <strong>OzoneManager UI</strong> at http://localhost:9874/ to see the
activity generated by freon.
While you are there, please don’t forget to check out the ozone configuration explorer.</p>
-
-<p><strong><em>Congratulations, You have just run your first ozone cluster.</em></strong></p>
-
+<p><em><strong>Congratulations, You have just run your first ozone cluster.</strong></em></p>
<p>To shutdown the cluster, please run
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose down</code></pre></div></p>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose down</span></span></code></pre></div></p>
</div>
</div>
diff --git a/docs/0.3.0-alpha/runningwithhdfs.html b/docs/0.3.0-alpha/runningwithhdfs.html
index 218150a..55a620e 100644
--- a/docs/0.3.0-alpha/runningwithhdfs.html
+++ b/docs/0.3.0-alpha/runningwithhdfs.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,53 +221,26 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Running concurrently with HDFS</h1>
<div class="col-md-9">
- <!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone is designed to work with HDFS. So it is easy to deploy ozone in an
existing HDFS cluster.</p>
-
<p>Ozone does <em>not</em> support security today. It is a work in progress and tracked
- in
+in
<a href="https://issues.apache.org/jira/browse/HDDS-4">HDDS-4</a>. If you enable ozone
in a secure HDFS cluster, for your own protection Ozone will refuse to work.</p>
-
<p>In other words, till Ozone security work is done, Ozone will not work in any
secure clusters.</p>
-
<p>The container manager part of Ozone runs inside DataNodes as a pluggable module.
To activate ozone you should define the service plugin implementation class.</p>
-
-<div class="alert alert-warning" role="alert">
-<b>Important</b>: It should be added to the <b>hdfs-site.xml</b> as the plugin should
-be activated as part of the normal HDFS Datanode bootstrap.
-</div>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>dfs.datanode.plugins<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>org.apache.hadoop.ozone.HddsDatanodeService<span style="color:#f92672"></value></span>
-<span style="color:#f92672"></property></span></code></pre></div>
-
+<!-- raw HTML omitted -->
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span><span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>dfs.datanode.plugins<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>org.apache.hadoop.ozone.HddsDatanodeService<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"></property></span></span></span></code></pre></div>
<p>You also need to add the ozone-datanode-plugin jar file to the classpath:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">export HADOOP_CLASSPATH<span style="color:#f92672">=</span>/opt/ozone/share/hadoop/ozoneplugin/hadoop-ozone-datanode-plugin.jar</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>export HADOOP_CLASSPATH<span style="color:#f92672">=</span>/opt/ozone/share/hadoop/ozoneplugin/hadoop-ozone-datanode-plugin.jar</span></span></code></pre></div>
<p>To start ozone with HDFS you should start the the following components:</p>
-
<ol>
<li>HDFS Namenode (from Hadoop distribution)</li>
<li>HDFS Datanode (from the Hadoop distribution with the plugin on the
@@ -253,16 +248,10 @@ classpath from the Ozone distribution)</li>
<li>Ozone Manager (from the Ozone distribution)</li>
<li>Storage Container manager (from the Ozone distribution)</li>
</ol>
-
<p>Please check the log of the datanode whether the HDDS/Ozone plugin is started or
not. Log of datanode should contain something like this:</p>
-
-<pre><code>2018-09-17 16:19:24 INFO HddsDatanodeService:158 - Started plug-in org.apache.hadoop.ozone.web.OzoneHddsDatanodeService@6f94fb9d
-</code></pre>
-
-<div class="alert alert-warning" role="alert">
-<b>Note:</b> The current version of Ozone is tested with Hadoop 3.1.
-</div>
+<pre tabindex="0"><code>2018-09-17 16:19:24 INFO HddsDatanodeService:158 - Started plug-in org.apache.hadoop.ozone.web.OzoneHddsDatanodeService@6f94fb9d
+</code></pre><!-- raw HTML omitted -->
</div>
</div>
diff --git a/docs/0.3.0-alpha/s3.html b/docs/0.3.0-alpha/s3.html
index ea11aee..685ca31 100644
--- a/docs/0.3.0-alpha/s3.html
+++ b/docs/0.3.0-alpha/s3.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,56 +221,27 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>S3</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone provides S3 compatible REST interface to use the object store data with any S3 compatible tools.</p>
-
<h2 id="getting-started">Getting started</h2>
-
<p>S3 Gateway is a separated component which provides the S3 compatible. It should be started additional to the regular Ozone components.</p>
-
<p>You can start a docker based cluster, including the S3 gateway from the release package.</p>
-
<p>Go to the <code>compose/ozones3</code> directory, and start the server:</p>
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose up -d</code></pre></div>
-<p>You can access the S3 gateway at <code>http://localhost:9878</code></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose up -d
+</span></span></code></pre></div><p>You can access the S3 gateway at <code>http://localhost:9878</code></p>
<h2 id="url-schema">URL Schema</h2>
-
-<p>Ozone S3 gateway supports both the virtual-host-style URL s3 bucket addresses (eg. <a href="http://bucketname.host:9878">http://bucketname.host:9878</a>) and the path-style addresses (eg. <a href="http://host:9878/bucketname">http://host:9878/bucketname</a>)</p>
-
+<p>Ozone S3 gateway supports both the virtual-host-style URL s3 bucket addresses (eg. <a href="http://bucketname.host:9878">http://bucketname.host:9878</a>) and the path-style addresses (eg. http://host:9878/bucketname)</p>
<p>By default it uses the path-style addressing. To use virtual host style URLs set your main domain name in your <code>ozone-site.xml</code>:</p>
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.s3g.domain.name<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>s3g.internal<span style="color:#f92672"></value></span>
-<span style="color:#f92672"></property></span></code></pre></div>
-<h2 id="bucket-browser">Bucket browser</h2>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span><span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.s3g.domain.name<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>s3g.internal<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"></property></span>
+</span></span></code></pre></div><h2 id="bucket-browser">Bucket browser</h2>
<p>Bucket’s could be browsed from the browser with adding <code>?browser=true</code> to the bucket URL.</p>
-
-<p>For example the content of the ‘testbucket’ could be checked from the browser using the URL <a href="http://localhost:9878/testbucket?browser=true">http://localhost:9878/testbucket?browser=true</a></p>
-
+<p>For example the content of the ’testbucket’ could be checked from the browser using the URL http://localhost:9878/testbucket?browser=true</p>
<h2 id="implemented-rest-endpoints">Implemented REST endpoints</h2>
-
<p>Operations on S3Gateway service:</p>
-
<table>
<thead>
<tr>
@@ -256,7 +249,6 @@
<th>Status</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>GET service</td>
@@ -264,9 +256,7 @@
</tr>
</tbody>
</table>
-
<p>Operations on Bucket:</p>
-
<table>
<thead>
<tr>
@@ -275,32 +265,27 @@
<th>Notes</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>GET Bucket (List Objects) Version 2</td>
<td>implemented</td>
<td></td>
</tr>
-
<tr>
<td>HEAD Bucket</td>
<td>implemented</td>
<td></td>
</tr>
-
<tr>
<td>DELETE Bucket</td>
<td>implemented</td>
<td></td>
</tr>
-
<tr>
<td>PUT Bucket (Create bucket)</td>
<td>implemented</td>
<td></td>
</tr>
-
<tr>
<td>Delete Multiple Objects (POST)</td>
<td>implemented</td>
@@ -308,9 +293,7 @@
</tr>
</tbody>
</table>
-
<p>Operation on Objects:</p>
-
<table>
<thead>
<tr>
@@ -319,32 +302,27 @@
<th>Notes</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>PUT Object</td>
<td>implemented</td>
<td></td>
</tr>
-
<tr>
<td>GET Object</td>
<td>implemented</td>
<td>Range headers are not supported</td>
</tr>
-
<tr>
<td>Multipart Uplad</td>
<td>not implemented</td>
<td></td>
</tr>
-
<tr>
<td>DELETE Object</td>
<td>implemented</td>
<td></td>
</tr>
-
<tr>
<td>HEAD Object</td>
<td>implemented</td>
@@ -352,43 +330,25 @@
</tr>
</tbody>
</table>
-
<h2 id="security">Security</h2>
-
<p>Security is not yet implemented, you can <em>use</em> any AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY</p>
-
<p>Note: Ozone has a notion for ‘volumes’ which is missing from the S3 Rest endpoint. Under the hood S3 bucket names are maped to Ozone ‘volume/bucket’ locations (depending from the given authentication information).</p>
-
<p>To show the storage location of a S3 bucket, use the <code>ozone sh bucket path <bucketname></code> command.</p>
-
-<pre><code>aws s3api --endpoint-url http://localhost:9878 create-bucket --bucket=bucket1
+<pre tabindex="0"><code>aws s3api --endpoint-url http://localhost:9878 create-bucket --bucket=bucket1
ozone sh bucket path bucket1
Volume name for S3Bucket is : s3thisisakey
Ozone FileSystem Uri is : o3fs://bucket1.s3thisisakey
-</code></pre>
-
-<h2 id="clients">Clients</h2>
-
+</code></pre><h2 id="clients">Clients</h2>
<h3 id="aws-cli">AWS Cli</h3>
-
<p><code>aws</code> CLI could be used with specifying the custom REST endpoint.</p>
-
-<pre><code>aws s3api --endpoint http://localhost:9878 create-bucket --bucket buckettest
-</code></pre>
-
-<p>Or</p>
-
-<pre><code>aws s3 ls --endpoint http://localhost:9878 s3://buckettest
-</code></pre>
-
-<h3 id="s3-fuse-driver-goofys">S3 Fuse driver (goofys)</h3>
-
+<pre tabindex="0"><code>aws s3api --endpoint http://localhost:9878 create-bucket --bucket buckettest
+</code></pre><p>Or</p>
+<pre tabindex="0"><code>aws s3 ls --endpoint http://localhost:9878 s3://buckettest
+</code></pre><h3 id="s3-fuse-driver-goofys">S3 Fuse driver (goofys)</h3>
<p>Goofys is a S3 FUSE driver. It could be used to any mount any Ozone bucket as posix file system:</p>
-
-<pre><code>goofys --endpoint http://localhost:9878 bucket1 /mount/bucket1
+<pre tabindex="0"><code>goofys --endpoint http://localhost:9878 bucket1 /mount/bucket1
</code></pre>
-
</div>
</div>
</div>
diff --git a/docs/0.3.0-alpha/scmcli.html b/docs/0.3.0-alpha/scmcli.html
index 7dcc9ca..338d4e1 100644
--- a/docs/0.3.0-alpha/scmcli.html
+++ b/docs/0.3.0-alpha/scmcli.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,25 +221,9 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>SCMCLI</h1>
<div class="col-md-9">
- <!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>SCM is the block service for Ozone. It is also the workhorse for ozone. But user process never talks to SCM. However, being able to read the state of SCM is useful.</p>
-
<p>SCMCLI allows the developer to access SCM directly. Please note: Improper usage of this tool can destroy your cluster. Unless you know exactly what you are doing, Please do <em>not</em> use this tool. In other words, this is a developer only tool. We might even remove this command in future to prevent improper use.</p>
</div>
diff --git a/docs/0.3.0-alpha/settings.html b/docs/0.3.0-alpha/settings.html
index af19696..fe142f7 100644
--- a/docs/0.3.0-alpha/settings.html
+++ b/docs/0.3.0-alpha/settings.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,87 +221,62 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Configuration</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>If you are feeling adventurous, you can setup ozone in a real cluster.
Setting up a real cluster requires us to understand the components of Ozone.
Ozone is designed to work concurrently with HDFS. However, Ozone is also
capable of running independently. The components of ozone are the same in both approaches.</p>
-
<h2 id="ozone-components">Ozone Components</h2>
-
<ol>
<li>Ozone Manager - Is the server that is in charge of the namespace of Ozone. Ozone Manager is responsible for all volume, bucket and key operations.</li>
<li>Storage Container Manager - Acts as the block manager. Ozone Manager
requests blocks from SCM, to which clients can write data.</li>
<li>Datanodes - Ozone data node code runs inside the HDFS datanode or in the independent deployment case runs an ozone datanode daemon.</li>
</ol>
-
<h2 id="setting-up-an-ozone-only-cluster">Setting up an Ozone only cluster</h2>
-
<ul>
-<li><p>Please untar the ozone-<version> to the directory where you are going
+<li>
+<p>Please untar the ozone-<!-- raw HTML omitted --> to the directory where you are going
to run Ozone from. We need Ozone jars on all machines in the cluster. So you
-need to do this on all machines in the cluster.</p></li>
-
-<li><p>Ozone relies on a configuration file called <code>ozone-site.xml</code>. To
+need to do this on all machines in the cluster.</p>
+</li>
+<li>
+<p>Ozone relies on a configuration file called <code>ozone-site.xml</code>. To
generate a template that you can replace with proper values, please run the
following command. This will generate a template called <code>ozone-site.xml</code> at
-the specified path (directory).</p></li>
+the specified path (directory).</p>
+</li>
</ul>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone genconf <path></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone genconf <path></span></span></code></pre></div>
<p>Let us look at the settings inside the generated file (ozone-site.xml) and
how they control ozone. Once the right values are defined, this file
needs to be copied to <code>ozone directory/etc/hadoop</code>.</p>
-
<ul>
<li><strong>ozone.enabled</strong> This is the most critical setting for ozone.
Ozone is a work in progress and users have to enable this service explicitly.
By default, Ozone is disabled. Setting this flag to <code>true</code> enables ozone in the
HDFS or Ozone cluster.</li>
</ul>
-
<p>Here is an example,</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.enabled<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>True<span style="color:#f92672"></value></span>
- <span style="color:#f92672"></property></span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span> <span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.enabled<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>True<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"></property></span></span></span></code></pre></div>
<ul>
-<li><strong>ozone.metadata.dirs</strong> Allows Administrators to specify where the
+<li>
+<p><strong>ozone.metadata.dirs</strong> Allows Administrators to specify where the
metadata must reside. Usually you pick your fastest disk (SSD if
you have them on your nodes). OzoneManager, SCM and datanode will write the
metadata to this path. This is a required setting, if this is missing Ozone
-will fail to come up.</li>
-</ul>
-
+will fail to come up.</p>
<p>Here is an example,</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.metadata.dirs<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>/data/disk1/meta<span style="color:#f92672"></value></span>
- <span style="color:#f92672"></property></span></code></pre></div>
-
+</li>
+</ul>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span> <span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.metadata.dirs<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>/data/disk1/meta<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"></property></span></span></span></code></pre></div>
<ul>
<li><strong>ozone.scm.names</strong> Storage container manager(SCM) is a distributed block
service which is used by ozone. This property allows data nodes to discover
@@ -287,40 +284,31 @@ SCM’s address. Data nodes send heartbeat to SCM.
Until HA feature is complete, we configure ozone.scm.names to be a
single machine.</li>
</ul>
-
<p>Here is an example,</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.scm.names<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>scm.hadoop.apache.org<span style="color:#f92672"></value></span>
- <span style="color:#f92672"></property></span>
- </code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span> <span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.scm.names<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>scm.hadoop.apache.org<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"></property></span></span></span></code></pre></div>
<ul>
<li><strong>ozone.scm.datanode.id</strong> Data nodes generate a Unique ID called Datanode
ID. This identity is written to the file specified by this path. <em>Data nodes
will create this path if it doesn’t exist already.</em></li>
</ul>
-
<p>Here is an example,
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.scm.datanode.id<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>/data/disk1/meta/node/datanode.id<span style="color:#f92672"></value></span>
- <span style="color:#f92672"></property></span></code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span> <span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.scm.datanode.id<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>/data/disk1/meta/node/datanode.id<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"></property></span></span></span></code></pre></div></p>
<ul>
<li><strong>ozone.om.address</strong> OM server address. This is used by OzoneClient and
Ozone File System.</li>
</ul>
-
<p>Here is an example,
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>ozone.om.address<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>ozonemanager.hadoop.apache.org<span style="color:#f92672"></value></span>
- <span style="color:#f92672"></property></span></code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span> <span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>ozone.om.address<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>ozonemanager.hadoop.apache.org<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"></property></span></span></span></code></pre></div></p>
<h3 id="ozone-settings-summary">Ozone Settings Summary</h3>
-
<table>
<thead>
<tr>
@@ -329,44 +317,37 @@ Ozone File System.</li>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>ozone.enabled</td>
<td>true</td>
<td>This enables SCM and containers in HDFS cluster.</td>
</tr>
-
<tr>
<td>ozone.metadata.dirs</td>
<td>file path</td>
<td>The metadata will be stored here.</td>
</tr>
-
<tr>
<td>ozone.scm.names</td>
<td>SCM server name</td>
<td>Hostname:port or IP:port address of SCM.</td>
</tr>
-
<tr>
<td>ozone.scm.block.client.address</td>
<td>SCM server name and port</td>
<td>Used by services like OM</td>
</tr>
-
<tr>
<td>ozone.scm.client.address</td>
<td>SCM server name and port</td>
<td>Used by client-side</td>
</tr>
-
<tr>
<td>ozone.scm.datanode.address</td>
<td>SCM server name and port</td>
<td>Used by datanode to talk to SCM</td>
</tr>
-
<tr>
<td>ozone.om.address</td>
<td>OM server name</td>
diff --git a/docs/0.3.0-alpha/sitemap.xml b/docs/0.3.0-alpha/sitemap.xml
index f2c3c02..83ae88b 100644
--- a/docs/0.3.0-alpha/sitemap.xml
+++ b/docs/0.3.0-alpha/sitemap.xml
@@ -1,104 +1,69 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:xhtml="http://www.w3.org/1999/xhtml">
-
<url>
<loc>/</loc>
- <lastmod>2017-09-14T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/runningviadocker.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/buildingsources.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/settings.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/runningwithhdfs.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/realcluster.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/hdds.html</loc>
- <lastmod>2017-09-14T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/ozonemanager.html</loc>
- <lastmod>2017-09-14T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/concepts.html</loc>
- <lastmod>2017-10-10T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/javaapi.html</loc>
- <lastmod>2017-09-14T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/ozonefs.html</loc>
- <lastmod>2017-09-14T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/freon.html</loc>
- <lastmod>2017-09-02T23:58:17-07:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/dozone.html</loc>
- <lastmod>2017-08-10T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/scmcli.html</loc>
- <lastmod>2017-08-10T00:00:00+00:00</lastmod>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/bucketcommands.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/categories.html</loc>
- <priority>0</priority>
- </url>
-
- <url>
+ </url><url>
<loc>/keycommands.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/commandshell.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/rest.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/s3.html</loc>
- </url>
-
- <url>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
+ </url><url>
<loc>/tags.html</loc>
- <priority>0</priority>
- </url>
-
- <url>
+ </url><url>
<loc>/volumecommands.html</loc>
+ <lastmod>2018-11-12T15:21:31-08:00</lastmod>
</url>
-
-</urlset>
\ No newline at end of file
+</urlset>
diff --git a/docs/0.3.0-alpha/tags/index.xml b/docs/0.3.0-alpha/tags/index.xml
index 8a5c1ee..22e514b 100644
--- a/docs/0.3.0-alpha/tags/index.xml
+++ b/docs/0.3.0-alpha/tags/index.xml
@@ -1,14 +1,10 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Tags on Ozone</title>
<link>/tags.html</link>
<description>Recent content in Tags on Ozone</description>
<generator>Hugo -- gohugo.io</generator>
- <language>en-us</language>
-
- <atom:link href="/tags/index.xml" rel="self" type="application/rss+xml" />
-
-
+ <language>en-us</language><atom:link href="/tags/index.xml" rel="self" type="application/rss+xml" />
</channel>
-</rss>
\ No newline at end of file
+</rss>
diff --git a/docs/0.3.0-alpha/volumecommands.html b/docs/0.3.0-alpha/volumecommands.html
index 535745b..5eaaf37 100644
--- a/docs/0.3.0-alpha/volumecommands.html
+++ b/docs/0.3.0-alpha/volumecommands.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -199,27 +221,9 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Volume Commands</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Volume commands generally need administrator privileges. The ozone shell supports the following volume commands.</p>
-
<ul>
<li><a href="#create">create</a></li>
<li><a href="#delete">delete</a></li>
@@ -227,14 +231,10 @@
<li><a href="#list">list</a></li>
<li><a href="#update">update</a></li>
</ul>
-
<h3 id="create">Create</h3>
-
<p>The volume create command allows an administrator to create a volume and
assign it to a user.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -242,37 +242,28 @@ assign it to a user.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>-q, –quota</td>
<td>Optional, This argument that specifies the maximum size this volume can use in the Ozone cluster.</td>
</tr>
-
<tr>
<td>-u, –user</td>
<td>Required, The name of the user who owns this volume. This user can create, buckets and keys on this volume.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the volume.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume create --quota<span style="color:#f92672">=</span>1TB --user<span style="color:#f92672">=</span>bilbo /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume create --quota<span style="color:#f92672">=</span>1TB --user<span style="color:#f92672">=</span>bilbo /hive</span></span></code></pre></div>
<p>The above command will create a volume called <em>hive</em> on the ozone cluster. This
volume has a quota of 1TB, and the owner is <em>bilbo</em>.</p>
-
<h3 id="delete">Delete</h3>
-
<p>The volume delete commands allows an administrator to delete a volume. If the
volume is not empty then this command will fail.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -280,7 +271,6 @@ volume is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -288,18 +278,13 @@ volume is not empty then this command will fail.</p>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume delete /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume delete /hive</span></span></code></pre></div>
<p>The above command will delete the volume hive, if the volume has no buckets
inside it.</p>
-
<h3 id="info">Info</h3>
-
<p>The volume info commands returns the information about the volume including
quota and owner information.
-<strong><em>Params:</em></strong></p>
-
+<em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -307,7 +292,6 @@ quota and owner information.
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -315,25 +299,15 @@ quota and owner information.
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume info /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume info /hive</span></span></code></pre></div>
<p>The above command will print out the information about hive volume.</p>
-
<h3 id="list">List</h3>
-
<p>The volume list command will list the volumes owned by a user.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume list --user hadoop</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume list --user hadoop</span></span></code></pre></div>
<p>The above command will print out all the volumes owned by the user hadoop.</p>
-
<h3 id="update">Update</h3>
-
<p>The volume update command allows changing of owner and quota on a given volume.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -341,29 +315,23 @@ quota and owner information.
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>-q, –quota</td>
<td>Optional, This argument that specifies the maximum size this volume can use in the Ozone cluster.</td>
</tr>
-
<tr>
<td>-u, –user</td>
<td>Optional, The name of the user who owns this volume. This user can create, buckets and keys on this volume.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the volume.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume update --quota<span style="color:#f92672">=</span>10TB /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume update --quota<span style="color:#f92672">=</span>10TB /hive</span></span></code></pre></div>
<p>The above command updates the volume quota to 10TB.</p>
-
<p>You can try out these commands from the docker instance of the <a href="runningviadocker.html">Alpha
Cluster</a>.</p>
diff --git a/docs/0.4.0-alpha/OzoneOverview.svg b/docs/0.4.0-alpha/OzoneOverview.svg
index c7fda5d..9d4660d 100644
--- a/docs/0.4.0-alpha/OzoneOverview.svg
+++ b/docs/0.4.0-alpha/OzoneOverview.svg
@@ -235,4 +235,4 @@
</text>
</g>
</g>
-</svg>
+</svg>
\ No newline at end of file
diff --git a/docs/0.4.0-alpha/auditparser.html b/docs/0.4.0-alpha/auditparser.html
index ddaa3cc..ba13352 100644
--- a/docs/0.4.0-alpha/auditparser.html
+++ b/docs/0.4.0-alpha/auditparser.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,29 +299,12 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Audit Parser</h1>
<div class="col-md-9">
- <!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Audit Parser tool can be used for querying the ozone audit logs.
This tool creates a sqllite database at the specified path. If the database
already exists, it will avoid creating a database.</p>
-
<p>The database contains only one table called <code>audit</code> defined as:</p>
-
<p>CREATE TABLE IF NOT EXISTS audit (
datetime text,
level varchar(7),
@@ -311,24 +316,17 @@ params text,
result varchar(7),
exception text,
UNIQUE(datetime,level,logger,user,ip,op,params,result))</p>
-
<p>Usage:
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone auditparser <path to db file> <span style="color:#f92672">[</span>COMMAND<span style="color:#f92672">]</span> <span style="color:#f92672">[</span>PARAM<span style="color:#f92672">]</span></code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone auditparser <path to db file> <span style="color:#f92672">[</span>COMMAND<span style="color:#f92672">]</span> <span style="color:#f92672">[</span>PARAM<span style="color:#f92672">]</span></span></span></code></pre></div></p>
<p>To load an audit log to database:
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone auditparser <path to db file> load <path to audit log></code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone auditparser <path to db file> load <path to audit log></span></span></code></pre></div>
Load command creates the audit table described above.</p>
-
<p>To run a custom read-only query:
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone auditparser <path to db file> query <<span style="color:#66d9ef">select</span> query enclosed within double quotes></code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone auditparser <path to db file> query <<span style="color:#66d9ef">select</span> query enclosed within double quotes></span></span></code></pre></div></p>
<p>Audit Parser comes with a set of templates(most commonly used queries).</p>
-
<p>To run a template query:
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone auditparser <path to db file> template <templateName></code></pre></div></p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone auditparser <path to db file> template <templateName></span></span></code></pre></div></p>
<p>Following templates are available:</p>
-
<table>
<thead>
<tr>
@@ -337,24 +335,21 @@ Load command creates the audit table described above.</p>
<th>SQL</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>top5users</td>
<td>Top 5 users</td>
<td>select user,count(*) as total from audit group by user order by total DESC limit 5</td>
</tr>
-
<tr>
<td>top5cmds</td>
<td>Top 5 commands</td>
<td>select op,count(*) as total from audit group by op order by total DESC limit 5</td>
</tr>
-
<tr>
<td>top5activetimebyseconds</td>
<td>Top 5 active times, grouped by seconds</td>
-<td>select substr(datetime,1,charindex(‘,’,datetime)-1) as dt,count(*) as thecount from audit group by dt order by thecount DESC limit 5</td>
+<td>select substr(datetime,1,charindex(’,’,datetime)-1) as dt,count(*) as thecount from audit group by dt order by thecount DESC limit 5</td>
</tr>
</tbody>
</table>
diff --git a/docs/0.4.0-alpha/bucketcommands.html b/docs/0.4.0-alpha/bucketcommands.html
index 7114a43..f261f34 100644
--- a/docs/0.4.0-alpha/bucketcommands.html
+++ b/docs/0.4.0-alpha/bucketcommands.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,27 +299,9 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Bucket Commands</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone shell supports the following bucket commands.</p>
-
<ul>
<li><a href="#create">create</a></li>
<li><a href="#delete">delete</a></li>
@@ -305,13 +309,9 @@
<li><a href="#list">list</a></li>
<li><a href="#update">update</a></li>
</ul>
-
<h3 id="create">Create</h3>
-
<p>The bucket create command allows users to create a bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -319,7 +319,6 @@
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -327,19 +326,13 @@
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket create /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket create /hive/jan</span></span></code></pre></div>
<p>The above command will create a bucket called <em>jan</em> in the <em>hive</em> volume.
Since no scheme was specified this command defaults to O3 (RPC) protocol.</p>
-
<h3 id="delete">Delete</h3>
-
<p>The bucket delete command allows users to delete a bucket. If the
bucket is not empty then this command will fail.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -347,7 +340,6 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -355,16 +347,11 @@ bucket is not empty then this command will fail.</p>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket delete /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket delete /hive/jan</span></span></code></pre></div>
<p>The above command will delete <em>jan</em> bucket if it is empty.</p>
-
<h3 id="info">Info</h3>
-
<p>The bucket info commands returns the information about the bucket.
-<strong><em>Params:</em></strong></p>
-
+<em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -372,7 +359,6 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -380,17 +366,11 @@ bucket is not empty then this command will fail.</p>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket info /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket info /hive/jan</span></span></code></pre></div>
<p>The above command will print out the information about <em>jan</em> bucket.</p>
-
<h3 id="list">List</h3>
-
<p>The bucket list command allows users to list the buckets in a volume.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -398,40 +378,30 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>-l, –length</td>
<td>Maximum number of results to return. Default: 100</td>
</tr>
-
<tr>
<td>-p, –prefix</td>
<td>Optional, Only buckets that match this prefix will be returned.</td>
</tr>
-
<tr>
<td>-s, –start</td>
<td>The listing will start from key after the start key.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the <em>volume</em>.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket list /hive</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket list /hive</span></span></code></pre></div>
<p>This command will list all buckets on the volume <em>hive</em>.</p>
-
<h3 id="update">Update</h3>
-
<p>The bucket update command allows changing access permissions on bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -439,37 +409,27 @@ bucket is not empty then this command will fail.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>–addAcl</td>
<td>Optional, Comma separated ACLs that will added to bucket.</td>
</tr>
-
<tr>
<td>–removeAcl</td>
<td>Optional, Comma separated list of acl to remove.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the bucket.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh bucket update --addAcl<span style="color:#f92672">=</span>user:bilbo:rw /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh bucket update --addAcl<span style="color:#f92672">=</span>user:bilbo:rw /hive/jan</span></span></code></pre></div>
<p>The above command gives user bilbo read/write permission to the bucket.</p>
-
<h3 id="path">path</h3>
-
<p>The bucket command to provide ozone mapping for s3 bucket (Created via aws cli)</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone s3 path <<s3Bucket>></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone s3 path <<s3Bucket>></span></span></code></pre></div>
<p>The above command will print VolumeName and the mapping created for s3Bucket.</p>
-
<p>You can try out these commands from the docker instance of the <a href="runningviadocker.html">Alpha
Cluster</a>.</p>
diff --git a/docs/0.4.0-alpha/buildingsources.html b/docs/0.4.0-alpha/buildingsources.html
index adb6920..ac78a84 100644
--- a/docs/0.4.0-alpha/buildingsources.html
+++ b/docs/0.4.0-alpha/buildingsources.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,55 +299,32 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Building from Sources</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-<p><strong><em>This is a guide on how to build the ozone sources. If you are <font
-color="red">not</font>
-planning to build sources yourself, you can safely skip this page.</em></strong></p>
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
+<p><em><strong>This is a guide on how to build the ozone sources. If you are <!-- raw HTML omitted -->not<!-- raw HTML omitted -->
+planning to build sources yourself, you can safely skip this page.</strong></em></p>
<p>If you are a Hadoop ninja, and wise in the ways of Apache, you already know
that a real Apache release is a source release.</p>
-
<p>If you want to build from sources, Please untar the source tarball and run
the ozone build command. This instruction assumes that you have all the
dependencies to build Hadoop on your build machine. If you need instructions
on how to build Hadoop, please look at the Apache Hadoop Website.</p>
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">mvn -f pom.ozone.xml clean package -DskipTests<span style="color:#f92672">=</span>true</code></pre></div>
-<p>This will build an ozone-<version>.tar.gz in your <code>hadoop-ozone/dist/target</code> directory.</p>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>mvn -f pom.ozone.xml clean package -DskipTests<span style="color:#f92672">=</span>true
+</span></span></code></pre></div><p>This will build an ozone-<version>.tar.gz in your <code>hadoop-ozone/dist/target</code> directory.</p>
<p>You can copy this tarball and use this instead of binary artifacts that are
provided along with the official release.</p>
-
<h2 id="how-to-test-the-build">How to test the build</h2>
-
<p>You can run the acceptance tests in the hadoop-ozone directory to make sure
that your build is functional. To launch the acceptance tests, please follow
- the instructions in the <strong>README.md</strong> in the <code>smoketest</code> directory.</p>
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">cd smoketest
-./test.sh</code></pre></div>
-<p>You can also execute only a minimal subset of the tests:</p>
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">cd smoketest
-./test.sh --env ozone basic</code></pre></div>
-<p>Acceptance tests
- will start a small ozone cluster and verify that ozone shell and ozone file
- system is fully functional.</p>
+the instructions in the <strong>README.md</strong> in the <code>smoketest</code> directory.</p>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>cd smoketest
+</span></span><span style="display:flex;"><span>./test.sh
+</span></span></code></pre></div><p>You can also execute only a minimal subset of the tests:</p>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>cd smoketest
+</span></span><span style="display:flex;"><span>./test.sh --env ozone basic
+</span></span></code></pre></div><p>Acceptance tests
+will start a small ozone cluster and verify that ozone shell and ozone file
+system is fully functional.</p>
</div>
</div>
diff --git a/docs/0.4.0-alpha/categories/index.xml b/docs/0.4.0-alpha/categories/index.xml
index 5c6fe28..eca954b 100644
--- a/docs/0.4.0-alpha/categories/index.xml
+++ b/docs/0.4.0-alpha/categories/index.xml
@@ -1,14 +1,10 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Categories on Ozone</title>
<link>/categories.html</link>
<description>Recent content in Categories on Ozone</description>
<generator>Hugo -- gohugo.io</generator>
- <language>en-us</language>
-
- <atom:link href="/categories/index.xml" rel="self" type="application/rss+xml" />
-
-
+ <language>en-us</language><atom:link href="/categories/index.xml" rel="self" type="application/rss+xml" />
</channel>
-</rss>
\ No newline at end of file
+</rss>
diff --git a/docs/0.4.0-alpha/commandshell.html b/docs/0.4.0-alpha/commandshell.html
index 059faaa..e20a945 100644
--- a/docs/0.4.0-alpha/commandshell.html
+++ b/docs/0.4.0-alpha/commandshell.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,31 +299,11 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Ozone CLI</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone has a set of command line tools that can be used to manage ozone.</p>
-
<p>All these commands are invoked via the <code>ozone</code> script.</p>
-
<p>The commands supported by ozone are:</p>
-
<ul>
<li><strong>classpath</strong> - Prints the class path needed to get the hadoop jar and the
required libraries.</li>
@@ -324,56 +326,43 @@ Container Manager.</li>
<li><strong>genconf</strong> - Generate minimally required ozone configs and output to
ozone-site.xml.</li>
</ul>
-
<h2 id="understanding-ozone-command-shell">Understanding Ozone command shell</h2>
-
<p>The most used command when working with Ozone is the Ozone command shell.
Ozone command shell gives a command shell interface to work against
Ozone.</p>
-
<p>The Ozone shell commands take the following format.</p>
-
<blockquote>
<p><em>ozone sh object action url</em></p>
</blockquote>
-
<p><strong>ozone</strong> script is used to invoke all Ozone sub-commands. The ozone shell is
invoked via <code>sh</code> command.</p>
-
<p>The object can be a volume, bucket or a key. The action is various verbs like
- create, list, delete etc.</p>
-
+create, list, delete etc.</p>
<p>Ozone URL can point to a volume, bucket or keys in the following format:</p>
-
<p><em>[scheme][server:port]/volume/bucket/key</em></p>
-
<p>Where,</p>
-
<ol>
-<li><p>Scheme - This should be <code>o3</code> which is the native RPC protocol to access
-Ozone API. The usage of the schema is optional.</p></li>
-
-<li><p>Server:Port - This is the address of the Ozone Manager. This can be server
+<li>
+<p>Scheme - This should be <code>o3</code> which is the native RPC protocol to access
+Ozone API. The usage of the schema is optional.</p>
+</li>
+<li>
+<p>Server:Port - This is the address of the Ozone Manager. This can be server
only, in that case, the default port is used. If this value is omitted
then the defaults specified in the ozone-site.xml will be used for Ozone
-Manager address.</p></li>
+Manager address.</p>
+</li>
</ol>
-
<p>Depending on the call, the volume/bucket/key names will be part of the URL.
Please see volume commands, bucket commands, and key commands section for more
detail.</p>
-
<h2 id="invoking-help">Invoking help</h2>
-
<p>Ozone shell help can be invoked at <em>object</em> level or at <em>action</em> level.
For example:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume --help</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume --help</span></span></code></pre></div>
<p>This will show all possible actions for volumes.</p>
-
<p>or it can be invoked to explain a specific action like
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume create --help</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume create --help</span></span></code></pre></div>
This command will give you command line options of the create command.</p>
</div>
diff --git a/docs/0.4.0-alpha/concepts.html b/docs/0.4.0-alpha/concepts.html
index a1bc869..4ce9731 100644
--- a/docs/0.4.0-alpha/concepts.html
+++ b/docs/0.4.0-alpha/concepts.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,115 +299,60 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Architecture</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone is a redundant, distributed object store build by
leveraging primitives present in HDFS. The primary design point of ozone is scalability, and it aims to scale to billions of objects.</p>
-
<p>Ozone consists of volumes, buckets, and keys. A volume is similar to a home directory in the ozone world. Only an administrator can create it. Volumes are used to store buckets. Once a volume is created users can create as many buckets as needed. Ozone stores data as keys which live inside these buckets.</p>
-
<p>Ozone namespace is composed of many storage volumes. Storage volumes are also used as the basis for storage accounting.</p>
-
<p>To access a key, an Ozone URL has the following format:</p>
-
-<pre><code>http://servername:port/volume/bucket/key
-</code></pre>
-
-<p>Where the server name is the name of a data node, the port is the data node HTTP port. The volume represents the name of the ozone volume; bucket is an ozone bucket created by the user and key represents the file.</p>
-
+<pre tabindex="0"><code>http://servername:port/volume/bucket/key
+</code></pre><p>Where the server name is the name of a data node, the port is the data node HTTP port. The volume represents the name of the ozone volume; bucket is an ozone bucket created by the user and key represents the file.</p>
<p>Please look at the <a href="./commandshell.html#shell">command line interface</a> for more info.</p>
-
<p>Ozone supports both (S3 compatible) REST and RPC protocols. Clients can choose either of these protocols to communicate with Ozone. Please see the <a href="./javaapi.html">client documentation</a> for more details.</p>
-
<p>Ozone separates namespace management and block space management; this helps
ozone to scale much better. The namespace is managed by a daemon called
<a href="./ozonemanager.html">Ozone Manager </a> (OM), and block space is
-managed by <a href="./hdds.html">Storage Container Manager</a> (SCM).</p>
-
+managed by [Storage Container Manager] (/hdds.html) (SCM).</p>
<p>The data nodes provide replication and ability to store blocks; these blocks are stored in groups to reduce the metadata pressure on SCM. This groups of blocks are called storage containers. Hence the block manager is called storage container
manager.</p>
-
<h2 id="ozone-overview">Ozone Overview</h2>
-
<p>
The following diagram is a high-level overview of the core components of Ozone.
</p>
-
-<p><img src="../../OzoneOverview.svg" alt="Architecture diagram" /></p>
-
+<p><img src="../../OzoneOverview.svg" alt="Architecture diagram"></p>
<p>The main elements of Ozone are
:</p>
-
<h3 id="ozone-manager">Ozone Manager
</h3>
-
<p><a href="./ozonemanager.html">Ozone Manager</a> (OM) takes care of the Ozone’s namespace.
All ozone objects like volumes, buckets, and keys are managed by OM. In Short, OM is the metadata manager for Ozone.
OM talks to blockManager(SCM) to get blocks and passes it on to the Ozone
client. Ozone client writes data to these blocks.
OM will eventually be replicated via Apache Ratis for High Availability.
</p>
-
<h3 id="storage-container-manager">Storage Container Manager</h3>
-
<p><a href="./hdds.html">Storage Container Manager</a> (SCM) is the block and cluster manager for Ozone.
SCM along with data nodes offer a service called ‘storage containers’.
A storage container is a group unrelated of blocks that are managed together as a single entity.</p>
-
<p>SCM offers the following abstractions.
</p>
-
-<p><img src="../../SCMBlockDiagram.png" alt="SCM Abstractions" /></p>
-
+<p><img src="../../SCMBlockDiagram.png" alt="SCM Abstractions"></p>
<h3 id="blocks">Blocks</h3>
-
<p>Blocks are similar to blocks in HDFS. They are replicated store of data. Client writes data to blocks.</p>
-
<h3 id="containers">Containers</h3>
-
<p>A collection of blocks replicated and managed together.</p>
-
<h3 id="pipelines">Pipelines</h3>
-
<p>SCM allows each storage container to choose its method of replication.
For example, a storage container might decide that it needs only one copy of a block
and might choose a stand-alone pipeline. Another storage container might want to have a very high level of reliability and pick a RATIS based pipeline. In other words, SCM allows different kinds of replication strategies to co-exist. The client while writing data, chooses a storage container with required properties.</p>
-
<h3 id="pools">Pools</h3>
-
<p>A group of data nodes is called a pool. For scaling purposes,
we define a pool as a set of machines. This makes management of data nodes easier.</p>
-
<h3 id="nodes">Nodes</h3>
-
<p>The data node where data is stored. SCM monitors these nodes via heartbeat.</p>
-
<h3 id="clients">Clients</h3>
-
-<p>Ozone ships with a set of clients. Ozone <a href="./commandshell.html#shell">CLI</a> is the command line interface like ‘hdfs’ command.
<a href="./freon.html">Freon</a> is a load generation tool for Ozone.
</p>
-
+<p>Ozone ships with a set of clients. Ozone <a href="./commandshell.html#shell">CLI</a> is the command line interface like ‘hdfs’ command.
[Freon] (/freon.html) is a load generation tool for Ozone.
</p>
<h2 id="s3-gateway">S3 gateway</h2>
-
<p>Ozone provides and <a href="./s3.html">S3 compatible REST gateway server</a>. All of the main S3 features are supported and any S3 compatible client library can be used.</p>
-
<h3 id="ozone-file-system">Ozone File System</h3>
-
<p><a href="./ozonefs.html">Ozone file system</a> is a Hadoop compatible file system. This allows Hadoop services and applications like Hive and Spark to run against
Ozone without any change. (For example: you can use <code>hdfs dfs -ls o3fs://...</code> instead of <code>hdfs dfs -ls hdfs://...</code>)</p>
-
<h3 id="ozone-client">Ozone Client</h3>
-
<p>This is similar to DFSClient in HDFS. This is the standard client to talk to Ozone. All other components that we have discussed so far rely on Ozone client. Ozone client supports RPC protocol.</p>
</div>
diff --git a/docs/0.4.0-alpha/css/ozonedoc.css b/docs/0.4.0-alpha/css/ozonedoc.css
index 8e762bb..bc546f5 100644
--- a/docs/0.4.0-alpha/css/ozonedoc.css
+++ b/docs/0.4.0-alpha/css/ozonedoc.css
@@ -146,4 +146,4 @@ a:hover {
h4 {
font-weight: bold;
-}
+}
\ No newline at end of file
diff --git a/docs/0.4.0-alpha/dozone.html b/docs/0.4.0-alpha/dozone.html
index 19288df..32a4350 100644
--- a/docs/0.4.0-alpha/dozone.html
+++ b/docs/0.4.0-alpha/dozone.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,90 +299,44 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Dozone & Dev Tools</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Dozone stands for docker for ozone. Ozone supports docker to make it easy to develop and test ozone. Starting a docker-based ozone container is simple.</p>
-
<p>In the <code>compose/ozone</code> directory there are two files that define the docker and ozone settings.</p>
-
<p>Developers can</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">cd compose/ozone</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>cd compose/ozone</span></span></code></pre></div>
<p>and simply run</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose up -d</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose up -d</span></span></code></pre></div>
<p>to run a ozone cluster on docker.</p>
-
<p>This command will launch OM, SCM and a data node.</p>
-
-<p>To access the OM UI, one can view <a href="http://localhost:9874">http://localhost:9874</a>.</p>
-
+<p>To access the OM UI, one can view http://localhost:9874.</p>
<p><em>Please note</em>: dozone does not map the data node ports to the 9864. Instead, it maps to the ephemeral port range. So many examples in the command shell will not work if you run those commands from the host machine. To find out where the data node port is listening, you can run the <code>docker ps</code> command or always ssh into a container before running ozone commands.</p>
-
<p>To shutdown a running docker-based ozone cluster, please run</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose down</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose down</span></span></code></pre></div>
<h2 id="adding-more-config-settings">Adding more config settings</h2>
-
<p>The file called <code>docker-config</code> contains all ozone specific config settings. This file is processed to create the ozone-site.xml.</p>
-
-<h2 id="useful-docker-ozone-commands">Useful Docker & Ozone Commands</h2>
-
+<h2 id="useful-docker--ozone-commands">Useful Docker & Ozone Commands</h2>
<p>If you make any modifications to ozone, the simplest way to test it is to run freon and unit tests.</p>
-
<p>Here are the instructions to run freon in a docker-based cluster.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose exec datanode bash</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose exec datanode bash</span></span></code></pre></div>
<p>This will open a bash shell on the data node container.
Now we can execute freon for load generation.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span></span></span></code></pre></div>
<p>Here is a set of helpful commands for working with docker for ozone.
To check the status of the components:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose ps</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose ps</span></span></code></pre></div>
<p>To get logs from a specific node/service:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose logs scm</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose logs scm</span></span></code></pre></div>
<p>As the WebUI ports are forwarded to the external machine, you can check the web UI:</p>
-
<ul>
-<li>For the Storage Container Manager: <a href="http://localhost:9876">http://localhost:9876</a></li>
-<li>For the Ozone Manager: <a href="http://localhost:9874">http://localhost:9874</a></li>
+<li>For the Storage Container Manager: http://localhost:9876</li>
+<li>For the Ozone Manager: http://localhost:9874</li>
<li>For the Datanode: check the port with <code>docker ps</code> (as there could be multiple data nodes, ports are mapped to the ephemeral port range)</li>
</ul>
-
<p>You can start multiple data nodes with:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose scale datanode<span style="color:#f92672">=</span><span style="color:#ae81ff">3</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose scale datanode<span style="color:#f92672">=</span><span style="color:#ae81ff">3</span></span></span></code></pre></div>
<p>You can test the commands from the <a href="./commandshell.html#shell">Ozone CLI</a> after opening a new bash shell in one of the containers:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">docker-compose exec datanode bash</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>docker-compose exec datanode bash</span></span></code></pre></div>
</div>
</div>
diff --git a/docs/0.4.0-alpha/freon.html b/docs/0.4.0-alpha/freon.html
index 78c36d0..aa2fb05 100644
--- a/docs/0.4.0-alpha/freon.html
+++ b/docs/0.4.0-alpha/freon.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,60 +299,34 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Freon</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="overview">Overview</h2>
-
<p>Freon is a load-generator for Ozone. This tool is used for testing the functionality of ozone.</p>
-
<h3 id="random-keys">Random keys</h3>
-
<p>In randomkeys mode, the data written into ozone cluster is randomly generated.
Each key will be of size 10 KB.</p>
-
<p>The number of volumes/buckets/keys can be configured. The replication type and
factor (eg. replicate with ratis to 3 nodes) Also can be configured.</p>
-
<p>For more information use</p>
-
<p><code>bin/ozone freon --help</code></p>
-
<h3 id="example">Example</h3>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span> --replicationType<span style="color:#f92672">=</span>RATIS --factor<span style="color:#f92672">=</span>THREE</code>< [...]
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">***************************************************
-Status: Success
-Git Base Revision: 48aae081e5afacbb3240657556b26c29e61830c3
-Number of Volumes created: <span style="color:#ae81ff">10</span>
-Number of Buckets created: <span style="color:#ae81ff">100</span>
-Number of Keys added: <span style="color:#ae81ff">1000</span>
-Ratis replication factor: THREE
-Ratis replication type: RATIS
-Average Time spent in volume creation: <span style="color:#ae81ff">00</span>:00:00,035
-Average Time spent in bucket creation: <span style="color:#ae81ff">00</span>:00:00,319
-Average Time spent in key creation: <span style="color:#ae81ff">00</span>:00:03,659
-Average Time spent in key write: <span style="color:#ae81ff">00</span>:00:10,894
-Total bytes written: <span style="color:#ae81ff">10240000</span>
-Total Execution time: <span style="color:#ae81ff">00</span>:00:16,898
-***********************</code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone freon randomkeys --numOfVolumes<span style="color:#f92672">=</span><span style="color:#ae81ff">10</span> --numOfBuckets <span style="color:#ae81ff">10</span> --numOfKeys <span style="color:#ae81ff">10</span> --replicationType<span style="color:#f92672">=</span>RATIS --factor< [...]
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>***************************************************
+</span></span><span style="display:flex;"><span>Status: Success
+</span></span><span style="display:flex;"><span>Git Base Revision: 48aae081e5afacbb3240657556b26c29e61830c3
+</span></span><span style="display:flex;"><span>Number of Volumes created: <span style="color:#ae81ff">10</span>
+</span></span><span style="display:flex;"><span>Number of Buckets created: <span style="color:#ae81ff">100</span>
+</span></span><span style="display:flex;"><span>Number of Keys added: <span style="color:#ae81ff">1000</span>
+</span></span><span style="display:flex;"><span>Ratis replication factor: THREE
+</span></span><span style="display:flex;"><span>Ratis replication type: RATIS
+</span></span><span style="display:flex;"><span>Average Time spent in volume creation: 00:00:00,035
+</span></span><span style="display:flex;"><span>Average Time spent in bucket creation: 00:00:00,319
+</span></span><span style="display:flex;"><span>Average Time spent in key creation: 00:00:03,659
+</span></span><span style="display:flex;"><span>Average Time spent in key write: 00:00:10,894
+</span></span><span style="display:flex;"><span>Total bytes written: <span style="color:#ae81ff">10240000</span>
+</span></span><span style="display:flex;"><span>Total Execution time: 00:00:16,898
+</span></span><span style="display:flex;"><span>***********************</span></span></code></pre></div>
</div>
</div>
diff --git a/docs/0.4.0-alpha/genconf.html b/docs/0.4.0-alpha/genconf.html
index 83b1f90..3528745 100644
--- a/docs/0.4.0-alpha/genconf.html
+++ b/docs/0.4.0-alpha/genconf.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,26 +299,10 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Generate Configurations</h1>
<div class="col-md-9">
- <!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Genconf tool generates a template ozone-site.xml file at the specified path.
This template file can be edited to replace with proper values.</p>
-
<p><code>ozone genconf <path></code></p>
</div>
diff --git a/docs/0.4.0-alpha/hdds.html b/docs/0.4.0-alpha/hdds.html
index ca5e87a..3e04a19 100644
--- a/docs/0.4.0-alpha/hdds.html
+++ b/docs/0.4.0-alpha/hdds.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,38 +299,16 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Hadoop Distributed Data Store</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="scm-overview">SCM Overview</h2>
-
<p>Storage Container Manager or SCM is a very important component of ozone. SCM
offers block and container-based services to Ozone Manager. A container is a
collection of unrelated blocks under ozone. SCM and data nodes work together
to maintain the replication levels needed by the cluster.</p>
-
<p>It is easier to look at a putKey operation to understand the role that SCM plays.</p>
-
<p>To put a key, a client makes a call to OM with the following arguments.</p>
-
<p>– putKey(keyName, data, pipeline type, replication count)</p>
-
<ol>
<li>keyName - refers to the file name.</li>
<li>data - The data that the client wants to write.</li>
@@ -317,21 +317,16 @@ refers to the replication strategy used for replicating a block. Ozone
currently supports Stand Alone and Ratis as two different pipeline types.</li>
<li>replication count - This specifies how many copies of the block replica should be maintained.</li>
</ol>
-
<p>In most cases, the client does not specify the pipeline type and replication
- count. The default pipeline type and replication count are used.</p>
-
+count. The default pipeline type and replication count are used.</p>
<p>Ozone Manager when it receives the putKey call, makes a call to SCM asking
for a pipeline instance with the specified property. So if the client asked
for RATIS replication strategy and a replication count of three, then OM
requests SCM to return a set of data nodes that meet this capability.</p>
-
<p>If SCM can find this a pipeline ( that is a set of data nodes) that can meet
the requirement from the client, then those nodes are returned to OM. OM will
persist this info and return a tuple consisting of {BlockID, ContainerName, and Pipeline}.</p>
-
<p>If SCM is not able to find a pipeline, then SCM creates a logical pipeline and then returns it.</p>
-
<p>SCM manages blocks, containers, and pipelines. To return healthy pipelines,
SCM also needs to understand the node health. So SCM listens to heartbeats
from data nodes and acts as the node manager too.</p>
diff --git a/docs/0.4.0-alpha/index.html b/docs/0.4.0-alpha/index.html
index f344489..92cfd74 100644
--- a/docs/0.4.0-alpha/index.html
+++ b/docs/0.4.0-alpha/index.html
@@ -3,7 +3,7 @@
<!DOCTYPE html>
<html lang="en">
<head>
- <meta name="generator" content="Hugo 0.55.3" />
+ <meta name="generator" content="Hugo 0.108.0">
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
@@ -18,6 +18,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -276,40 +298,20 @@
</div>
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h1 id="apache-hadoop-ozone">Apache Hadoop Ozone</h1>
-
<p>Ozone is a scalable, distributed object store for Hadoop. Applications like
Apache Spark, Hive and YARN, can run against Ozone without any
-modifications. Ozone comes with a <a href="./javaapi.html">Java client library</a>, a <a href="./s3.html">S3</a> and a <a href="./commandshell.html#shell">command line interface</a> which makes it easy to use Ozone.</p>
-
+modifications. Ozone comes with a <a href="./javaapi.html">Java client library</a>, a <a href="./s3.html">S3</a> and a [command line interface]
+(/commandshell.html#shell) which makes it easy to use Ozone.</p>
<p>Ozone consists of volumes, buckets, and Keys:</p>
-
<ul>
<li>Volumes are similar to user accounts. Only administrators can create or delete volumes.</li>
<li>Buckets are similar to directories. A bucket can contain any number of keys, but buckets cannot contain other buckets.</li>
<li>Keys are similar to files. A bucket can contain any number of keys.</li>
</ul>
-
-<p><a href="./runningviadocker.html"><button class="btn btn-danger btn-lg">Getting started</button></a></p>
+<p><!-- raw HTML omitted --><!-- raw HTML omitted -->Getting started<!-- raw HTML omitted --><!-- raw HTML omitted --></p>
diff --git a/docs/0.4.0-alpha/index.xml b/docs/0.4.0-alpha/index.xml
index d4a4351..cd0a64c 100644
--- a/docs/0.4.0-alpha/index.xml
+++ b/docs/0.4.0-alpha/index.xml
@@ -1,16 +1,11 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Ozone Overview on Ozone</title>
<link>/</link>
<description>Recent content in Ozone Overview on Ozone</description>
<generator>Hugo -- gohugo.io</generator>
- <language>en-us</language>
- <lastBuildDate>Tue, 18 Dec 2018 00:00:00 +0000</lastBuildDate>
-
- <atom:link href="/index.xml" rel="self" type="application/rss+xml" />
-
-
+ <language>en-us</language><atom:link href="/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>Ozone File System</title>
<link>/ozonefs.html</link>
@@ -50,7 +45,7 @@ If you want to build from sources, Please untar the source tarball and run the o
<guid>/settings.html</guid>
<description>If you are feeling adventurous, you can setup ozone in a real cluster. Setting up a real cluster requires us to understand the components of Ozone. Ozone is designed to work concurrently with HDFS. However, Ozone is also capable of running independently. The components of ozone are the same in both approaches.
-Ozone Components Ozone Manager - Is the server that is in charge of the namespace of Ozone.</description>
+Ozone Components Ozone Manager - Is the server that is in charge of the namespace of Ozone. Ozone Manager is responsible for all volume, bucket and key operations.</description>
</item>
<item>
@@ -104,7 +99,8 @@ Ozone&rsquo;s namespace is a collection of volumes or is a forest instead of
<guid>/ozonesecurityarchitecture.html</guid>
<description>Security in Ozone Starting with badlands release (ozone-0.4.0-alpha) ozone cluster can be secured against external threats. Specifically it can be configured for following security features:
- Authentication Authorization Audit Transparent Data Encryption (TDE) Authentication Kerberos Similar to hadoop, Ozone allows kerberos-based authentication. So one way to setup identities for all the daemons and clients is to create kerberos keytabs and configure it like any other service in hadoop.</description>
+Authentication Authorization Audit Transparent Data Encryption (TDE) Authentication Kerberos Similar to hadoop, Ozone allows kerberos-based authentication. So one way to setup identities for all the daemons and clients is to create kerberos keytabs and configure it like any other service in hadoop.
+Tokens Tokens are widely used in Hadoop to achieve lightweight authentication without compromising on security.</description>
</item>
<item>
@@ -114,8 +110,8 @@ Ozone&rsquo;s namespace is a collection of volumes or is a forest instead of
<guid>/setupsecureozone.html</guid>
<description>Setup secure ozone cluster To enable security in ozone cluster ozone.security.enabled should be set to true.
- Property Value ozone.security.enabled true Kerberos Configuration for service daemons:
- Property Description hdds.scm.kerberos.principal The SCM service principal. Ex scm/HOST@REALM.COM hdds.scm.kerberos.keytab.file The keytab file used by SCM daemon to login as its service principal. ozone.</description>
+Property Value ozone.security.enabled true Kerberos Configuration for service daemons:
+Property Description hdds.scm.kerberos.principal The SCM service principal. Ex scm/HOST@REALM.COM hdds.scm.kerberos.keytab.file The keytab file used by SCM daemon to login as its service principal. ozone.om.kerberos.principal The OzoneManager service principal. Ex om/_HOST@REALM.COM ozone.om.kerberos.keytab.file The keytab file used by SCM daemon to login as its service principal. hdds.</description>
</item>
<item>
@@ -136,8 +132,7 @@ ozone genconf &lt;path&gt;</description>
<guid>/auditparser.html</guid>
<description>Audit Parser tool can be used for querying the ozone audit logs. This tool creates a sqllite database at the specified path. If the database already exists, it will avoid creating a database.
The database contains only one table called audit defined as:
-CREATE TABLE IF NOT EXISTS audit ( datetime text, level varchar(7), logger varchar(7), user text, ip text, op text, params text, result varchar(7), exception text, UNIQUE(datetime,level,logger,user,ip,op,params,result))
-Usage: ozone auditparser &lt;path to db file&gt; [COMMAND] [PARAM]</description>
+CREATE TABLE IF NOT EXISTS audit ( datetime text, level varchar(7), logger varchar(7), user text, ip text, op text, params text, result varchar(7), exception text, UNIQUE(datetime,level,logger,user,ip,op,params,result))</description>
</item>
<item>
@@ -206,9 +201,10 @@ SCMCLI allows the developer to access SCM directly. Please note: Improper usage
<guid>/bucketcommands.html</guid>
<description>Ozone shell supports the following bucket commands.
- create delete info list update Create The bucket create command allows users to create a bucket.
+create delete info list update Create The bucket create command allows users to create a bucket.
Params:
- Arguments Comment Uri The name of the bucket in /volume/bucket format. ozone sh bucket create /hive/jan The above command will create a bucket called jan in the hive volume. Since no scheme was specified this command defaults to O3 (RPC) protocol.</description>
+Arguments Comment Uri The name of the bucket in /volume/bucket format. ozone sh bucket create /hive/jan The above command will create a bucket called jan in the hive volume. Since no scheme was specified this command defaults to O3 (RPC) protocol.
+Delete The bucket delete command allows users to delete a bucket.</description>
</item>
<item>
@@ -218,9 +214,9 @@ Params:
<guid>/keycommands.html</guid>
<description>Ozone shell supports the following key commands.
- get put delete info list rename Get The key get command downloads a key from Ozone cluster to local file system.
+get put delete info list rename Get The key get command downloads a key from Ozone cluster to local file system.
Params:
- Arguments Comment Uri The name of the key in /volume/bucket/key format. FileName Local file to download the key to. ozone sh key get /hive/jan/sales.orc sales.orc Downloads the file sales.</description>
+Arguments Comment Uri The name of the key in /volume/bucket/key format. FileName Local file to download the key to. ozone sh key get /hive/jan/sales.orc sales.orc Downloads the file sales.orc from the /hive/jan bucket and writes to the local file sales.orc.</description>
</item>
<item>
@@ -231,7 +227,7 @@ Params:
<guid>/prometheus.html</guid>
<description>Prometheus is an open-source monitoring server developed under under the Cloud Native Computing Foundation.
Ozone supports Prometheus out of the box. The servers start a prometheus compatible metrics endpoint where all the available hadoop metrics are published in prometheus exporter format.
-Prerequisites Install the and start an Ozone cluster. Download the prometheus binary. Monitoring with prometheus (1) To enable the Prometheus metrics endpoint you need to add a new configuration to the ozone-site.</description>
+Prerequisites Install the and start an Ozone cluster. Download the prometheus binary. Monitoring with prometheus (1) To enable the Prometheus metrics endpoint you need to add a new configuration to the ozone-site.</description>
</item>
<item>
@@ -243,7 +239,7 @@ Prerequisites Install the and start an Ozone cluster. Download the prometheus b
<description>Ozone has a set of command line tools that can be used to manage ozone.
All these commands are invoked via the ozone script.
The commands supported by ozone are:
- classpath - Prints the class path needed to get the hadoop jar and the required libraries. dtutil - Operations related to delegation tokens fs - Runs a command on ozone file system. datanode - Via daemon command, the HDDS data nodes can be started or stopped.</description>
+classpath - Prints the class path needed to get the hadoop jar and the required libraries. dtutil - Operations related to delegation tokens fs - Runs a command on ozone file system. datanode - Via daemon command, the HDDS data nodes can be started or stopped.</description>
</item>
<item>
@@ -266,9 +262,9 @@ docker-compose up -d You can access the S3 gateway at http://localhost:9878</des
<guid>/sparkozonefsk8s.html</guid>
<description>Using Ozone from Apache Spark This recipe shows how Ozone object store can be used from Spark using:
- OzoneFS (Hadoop compatible file system) Hadoop 2.7 (included in the Spark distribution) Kubernetes Spark scheduler Local spark client Requirements Download latest Spark and Ozone distribution and extract them. This method is tested with the spark-2.4.0-bin-hadoop2.7 distribution.
+OzoneFS (Hadoop compatible file system) Hadoop 2.7 (included in the Spark distribution) Kubernetes Spark scheduler Local spark client Requirements Download latest Spark and Ozone distribution and extract them. This method is tested with the spark-2.4.0-bin-hadoop2.7 distribution.
You also need the following:
- A container repository to push and pull the spark+ozone images.</description>
+A container repository to push and pull the spark+ozone images. (In this recipe we will use the dockerhub) A repo/name for the custom containers (in this recipe myrepo/ozone-spark) A dedicated namespace in kubernetes (we use yournamespace in this recipe) Create the docker image for drivers Create the base Spark driver/executor image First of all create a docker image with the Spark image creator.</description>
</item>
<item>
@@ -278,10 +274,10 @@ You also need the following:
<guid>/volumecommands.html</guid>
<description>Volume commands generally need administrator privileges. The ozone shell supports the following volume commands.
- create delete info list update Create The volume create command allows an administrator to create a volume and assign it to a user.
+create delete info list update Create The volume create command allows an administrator to create a volume and assign it to a user.
Params:
- Arguments Comment -q, &ndash;quota Optional, This argument that specifies the maximum size this volume can use in the Ozone cluster. -u, &ndash;user Required, The name of the user who owns this volume.</description>
+Arguments Comment -q, &ndash;quota Optional, This argument that specifies the maximum size this volume can use in the Ozone cluster. -u, &ndash;user Required, The name of the user who owns this volume.</description>
</item>
</channel>
-</rss>
\ No newline at end of file
+</rss>
diff --git a/docs/0.4.0-alpha/javaapi.html b/docs/0.4.0-alpha/javaapi.html
index 32d7abc..5c1e52e 100644
--- a/docs/0.4.0-alpha/javaapi.html
+++ b/docs/0.4.0-alpha/javaapi.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,144 +299,102 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Java API</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="introduction">Introduction</h2>
-
<p>Ozone ships with its own client library that supports RPC. For generic use cases the S3
compatible REST interface also can be used instead of the Ozone client.</p>
-
<h2 id="creating-an-ozone-client">Creating an Ozone client</h2>
-
<p>The Ozone client factory creates the ozone client. To get a RPC client we can call</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getRpcClient</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getRpcClient</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>If the user want to create a client based on the configuration, then they can
call.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>and an appropriate client based on configuration will be returned.</p>
-
<h2 id="writing-data-using-ozone-client">Writing data using Ozone Client</h2>
-
<p>The hierarchy of data inside ozone is a volume, bucket and a key. A volume
is a collection of buckets. A bucket is a collection of keys. To write data
to the ozone, you need a volume, bucket and a key.</p>
-
<h3 id="creating-a-volume">Creating a Volume</h3>
-
<p>Once we have a client, we need to get a reference to the ObjectStore. This
is done via</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java">ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span>ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span></span></span></code></pre></div>
<p>An object store represents an active cluster against which the client is working.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java"><span style="color:#75715e">// Let us create a volume to store our game assets.
-</span><span style="color:#75715e">// This uses default arguments for creating that volume.
-</span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#e6db74">"assets"</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Let us verify that the volume got created.
-</span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#e6db74">"assets"</span><span style="color:#f92672">);</span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#75715e">// Let us create a volume to store our game assets.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e">// This uses default arguments for creating that volume.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#e6db74">"assets"</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us verify that the volume got created.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#e6db74">"assets"</span><span style="color:#f92672">);</span></span></span></code></pre></div>
<p>It is possible to pass an array of arguments to the createVolume by creating volume arguments.</p>
-
<h3 id="creating-a-bucket">Creating a Bucket</h3>
-
<p>Once you have a volume, you can create buckets inside the volume.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">// Let us create a bucket called videos.
-assets.createBucket<span style="color:#f92672">(</span><span style="color:#e6db74">"videos"</span><span style="color:#f92672">)</span>;
-OzoneBucket video <span style="color:#f92672">=</span> assets.getBucket<span style="color:#f92672">(</span><span style="color:#e6db74">"videos"</span><span style="color:#f92672">)</span>;</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>// Let us create a bucket called videos.
+</span></span><span style="display:flex;"><span>assets.createBucket<span style="color:#f92672">(</span><span style="color:#e6db74">"videos"</span><span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>OzoneBucket video <span style="color:#f92672">=</span> assets.getBucket<span style="color:#f92672">(</span><span style="color:#e6db74">"videos"</span><span style="color:#f92672">)</span>;</span></span></code></pre></div>
<p>At this point we have a usable volume and a bucket. Our volume is called assets and bucket is called videos.</p>
-
<p>Now we can create a Key.</p>
-
<h3 id="reading-and-writing-a-key">Reading and Writing a Key</h3>
-
<p>With a bucket object the users can now read and write keys. The following code reads a video called intro.mp4 from the local disk and stores in the video bucket that we just created.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">// read data from the file, this is a user provided <span style="color:#66d9ef">function</span>.
-byte <span style="color:#f92672">[]</span> videoData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span><span style="color:#f92672">)</span>;
-
-// Create an output stream and write data.
-OzoneOutputStream videoStream <span style="color:#f92672">=</span> video.createKey<span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span>, <span style="color:#ae81ff">1048576</span><span style="color:#f92672">)</span>;
-videoStream.write<span style="color:#f92672">(</span>videoData<span style="color:#f92672">)</span>;
-
-// Close the stream when it is <span style="color:#66d9ef">done</span>.
-videoStream.close<span style="color:#f92672">()</span>;
-
-
-// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
-// Let us allocate a byte array to hold the video first.
-byte<span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> new byte<span style="color:#f92672">[(</span>int<span style="color:#f92672">)</span><span style="color:#ae81ff">1048576</span><span style="color:#f92672">]</span>;
-OzoneInputStream introStream <span style="color:#f92672">=</span> video.readKey<span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span><span style="color:#f92672">)</span>;
-// read intro.mp4 into the data buffer
-introStream.read<span style="color:#f92672">(</span>data<span style="color:#f92672">)</span>;
-introStream.close<span style="color:#f92672">()</span>;</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>// read data from the file, this is a user provided <span style="color:#66d9ef">function</span>.
+</span></span><span style="display:flex;"><span>byte <span style="color:#f92672">[]</span> videoData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span><span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>// Create an output stream and write data.
+</span></span><span style="display:flex;"><span>OzoneOutputStream videoStream <span style="color:#f92672">=</span> video.createKey<span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span>, 1048576<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>videoStream.write<span style="color:#f92672">(</span>videoData<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>// Close the stream when it is <span style="color:#66d9ef">done</span>.
+</span></span><span style="display:flex;"><span>videoStream.close<span style="color:#f92672">()</span>;
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
+</span></span><span style="display:flex;"><span>// Let us allocate a byte array to hold the video first.
+</span></span><span style="display:flex;"><span>byte<span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> new byte<span style="color:#f92672">[(</span>int<span style="color:#f92672">)</span>1048576<span style="color:#f92672">]</span>;
+</span></span><span style="display:flex;"><span>OzoneInputStream introStream <span style="color:#f92672">=</span> video.readKey<span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span><span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>// read intro.mp4 into the data buffer
+</span></span><span style="display:flex;"><span>introStream.read<span style="color:#f92672">(</span>data<span style="color:#f92672">)</span>;
+</span></span><span style="display:flex;"><span>introStream.close<span style="color:#f92672">()</span>;</span></span></code></pre></div>
<p>Here is a complete example of the code that we just wrote. Please note the close functions being called in this program.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java"><span style="color:#75715e">// Let us create a client
-</span><span style="color:#75715e"></span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span>
-
-<span style="color:#75715e">// Get a reference to the ObjectStore using the client
-</span><span style="color:#75715e"></span>ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span>
-
-<span style="color:#75715e">// Let us create a volume to store our game assets.
-</span><span style="color:#75715e">// This default arguments for creating that volume.
-</span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#e6db74">"assets"</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Let us verify that the volume got created.
-</span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#e6db74">"assets"</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Let us create a bucket called videos.
-</span><span style="color:#75715e"></span>assets<span style="color:#f92672">.</span><span style="color:#a6e22e">createBucket</span><span style="color:#f92672">(</span><span style="color:#e6db74">"videos"</span><span style="color:#f92672">);</span>
-OzoneBucket video <span style="color:#f92672">=</span> assets<span style="color:#f92672">.</span><span style="color:#a6e22e">getBucket</span><span style="color:#f92672">(</span><span style="color:#e6db74">"videos"</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// read data from the file, this is assumed to be a user provided function.
-</span><span style="color:#75715e"></span><span style="color:#66d9ef">byte</span> <span style="color:#f92672">[]</span> videoData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span><span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Create an output stream and write data.
-</span><span style="color:#75715e"></span>OzoneOutputStream videoStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">createKey</span><span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span><span style="color:#f92672">,</span> 1048576<span style="color:#f92672">);</span>
-videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">write</span><span style="color:#f92672">(</span>videoData<span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Close the stream when it is done.
-</span><span style="color:#75715e"></span>videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
-
-
-<span style="color:#75715e">// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
-</span><span style="color:#75715e">// Let us allocate a byte array to hold the video first.
-</span><span style="color:#75715e"></span>
-<span style="color:#66d9ef">byte</span><span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> <span style="color:#66d9ef">new</span> <span style="color:#66d9ef">byte</span><span style="color:#f92672">[(</span><span style="color:#66d9ef">int</span><span style="color:#f92672">)</span>1048576<span style="color:#f92672">];</span>
-OzoneInputStream introStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">readKey</span><span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span><span style="color:#f92672">);</span>
-introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">read</span><span style="color:#f92672">(</span>data<span style="color:#f92672">);</span>
-
-<span style="color:#75715e">// Close the stream when it is done.
-</span><span style="color:#75715e"></span>introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
-
-<span style="color:#75715e">// Close the client.
-</span><span style="color:#75715e"></span>ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span></code></pre></div>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#75715e">// Let us create a client
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneClient ozClient <span style="color:#f92672">=</span> OzoneClientFactory<span style="color:#f92672">.</span><span style="color:#a6e22e">getClient</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Get a reference to the ObjectStore using the client
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>ObjectStore objectStore <span style="color:#f92672">=</span> ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">getObjectStore</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us create a volume to store our game assets.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e">// This default arguments for creating that volume.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">createVolume</span><span style="color:#f92672">(</span><span style="color:#e6db74">"assets"</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us verify that the volume got created.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneVolume assets <span style="color:#f92672">=</span> objectStore<span style="color:#f92672">.</span><span style="color:#a6e22e">getVolume</span><span style="color:#f92672">(</span><span style="color:#e6db74">"assets"</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us create a bucket called videos.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>assets<span style="color:#f92672">.</span><span style="color:#a6e22e">createBucket</span><span style="color:#f92672">(</span><span style="color:#e6db74">"videos"</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>OzoneBucket video <span style="color:#f92672">=</span> assets<span style="color:#f92672">.</span><span style="color:#a6e22e">getBucket</span><span style="color:#f92672">(</span><span style="color:#e6db74">"videos"</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// read data from the file, this is assumed to be a user provided function.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">byte</span> <span style="color:#f92672">[]</span> videoData <span style="color:#f92672">=</span> readFile<span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Create an output stream and write data.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>OzoneOutputStream videoStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">createKey</span><span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span><span style="color:#f92672">,</span> <span style="color:#ae81ff">1048576</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">write</span><span style="color:#f92672">(</span>videoData<span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Close the stream when it is done.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>videoStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e">// Let us allocate a byte array to hold the video first.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>
+</span></span><span style="display:flex;"><span><span style="color:#66d9ef">byte</span><span style="color:#f92672">[]</span> data <span style="color:#f92672">=</span> <span style="color:#66d9ef">new</span> <span style="color:#66d9ef">byte</span><span style="color:#f92672">[(</span><span style="color:#66d9ef">int</span><span style="color:#f92672">)</span><span style="color:#ae81ff">1048576</span><span style="color:#f92672">];</span>
+</span></span><span style="display:flex;"><span>OzoneInputStream introStream <span style="color:#f92672">=</span> video<span style="color:#f92672">.</span><span style="color:#a6e22e">readKey</span><span style="color:#f92672">(</span><span style="color:#e6db74">"intro.mp4"</span><span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">read</span><span style="color:#f92672">(</span>data<span style="color:#f92672">);</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Close the stream when it is done.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>introStream<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span>
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">// Close the client.
+</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>ozClient<span style="color:#f92672">.</span><span style="color:#a6e22e">close</span><span style="color:#f92672">();</span></span></span></code></pre></div>
</div>
</div>
diff --git a/docs/0.4.0-alpha/keycommands.html b/docs/0.4.0-alpha/keycommands.html
index 5ddc607..5d4acb1 100644
--- a/docs/0.4.0-alpha/keycommands.html
+++ b/docs/0.4.0-alpha/keycommands.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,27 +299,9 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Key Commands</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>Ozone shell supports the following key commands.</p>
-
<ul>
<li><a href="#get">get</a></li>
<li><a href="#put">put</a></li>
@@ -306,13 +310,9 @@
<li><a href="#list">list</a></li>
<li><a href="#rename">rename</a></li>
</ul>
-
<h3 id="get">Get</h3>
-
<p>The key get command downloads a key from Ozone cluster to local file system.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -320,30 +320,23 @@
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
<td>The name of the key in <strong>/volume/bucket/key</strong> format.</td>
</tr>
-
<tr>
<td>FileName</td>
<td>Local file to download the key to.</td>
</tr>
</tbody>
</table>
-
-<p><div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key get /hive/jan/sales.orc sales.orc</code></pre></div>
+<p><div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key get /hive/jan/sales.orc sales.orc</span></span></code></pre></div>
Downloads the file sales.orc from the <em>/hive/jan</em> bucket and writes to the
local file sales.orc.</p>
-
<h3 id="put">Put</h3>
-
<p>Uploads a file from the local file system to the specified bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -351,34 +344,26 @@ local file sales.orc.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
<td>The name of the key in <strong>/volume/bucket/key</strong> format.</td>
</tr>
-
<tr>
<td>FileName</td>
<td>Local file to upload.</td>
</tr>
-
<tr>
<td>-r, –replication</td>
<td>Optional, Number of copies, ONE or THREE are the options. Picks up the default from cluster configuration.</td>
</tr>
</tbody>
</table>
-
-<p><div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key put /hive/jan/corrected-sales.orc sales.orc</code></pre></div>
+<p><div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key put /hive/jan/corrected-sales.orc sales.orc</span></span></code></pre></div>
The above command will put the sales.orc as a new key into <em>/hive/jan/corrected-sales.orc</em>.</p>
-
<h3 id="delete">Delete</h3>
-
<p>The key delete command removes the key from the bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -386,7 +371,6 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -394,16 +378,11 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key delete /hive/jan/corrected-sales.orc</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key delete /hive/jan/corrected-sales.orc</span></span></code></pre></div>
<p>The above command deletes the key <em>/hive/jan/corrected-sales.orc</em>.</p>
-
<h3 id="info">Info</h3>
-
<p>The key info commands returns the information about the key.
-<strong><em>Params:</em></strong></p>
-
+<em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -411,7 +390,6 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
@@ -419,18 +397,12 @@ The above command will put the sales.orc as a new key into <em>/hive/jan/correct
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key info /hive/jan/sales.orc</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key info /hive/jan/sales.orc</span></span></code></pre></div>
<p>The above command will print out the information about <em>/hive/jan/sales.orc</em>
key.</p>
-
<h3 id="list">List</h3>
-
<p>The key list command allows user to list all keys in a bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -438,40 +410,30 @@ key.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>-l, –length</td>
<td>Maximum number of results to return. Default: 1000</td>
</tr>
-
<tr>
<td>-p, –prefix</td>
<td>Optional, Only buckets that match this prefix will be returned.</td>
</tr>
-
<tr>
<td>-s, –start</td>
<td>The listing will start from key after the start key.</td>
</tr>
-
<tr>
<td>Uri</td>
<td>The name of the <em>volume</em>.</td>
</tr>
</tbody>
</table>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key list /hive/jan</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key list /hive/jan</span></span></code></pre></div>
<p>This command will list all keys in the bucket <em>/hive/jan</em>.</p>
-
<h3 id="rename">Rename</h3>
-
<p>The <code>key rename</code> command changes the name of an existing key in the specified bucket.</p>
-
-<p><strong><em>Params:</em></strong></p>
-
+<p><em><strong>Params:</strong></em></p>
<table>
<thead>
<tr>
@@ -479,28 +441,23 @@ key.</p>
<th>Comment</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>Uri</td>
<td>The name of the bucket in <strong>/volume/bucket</strong> format.</td>
</tr>
-
<tr>
<td>FromKey</td>
<td>The existing key to be renamed</td>
</tr>
-
<tr>
<td>ToKey</td>
<td>The new desired name of the key</td>
</tr>
</tbody>
</table>
-
-<p><div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh key rename /hive/jan sales.orc new_name.orc</code></pre></div>
+<p><div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh key rename /hive/jan sales.orc new_name.orc</span></span></code></pre></div>
The above command will rename <code>sales.orc</code> to <code>new_name.orc</code> in the bucket <code>/hive/jan</code>.</p>
-
<p>You can try out these commands from the docker instance of the <a href="runningviadocker.html">Alpha
Cluster</a>.</p>
diff --git a/docs/0.4.0-alpha/ozonefs.html b/docs/0.4.0-alpha/ozonefs.html
index 2b7aeaf..0f4f69e 100644
--- a/docs/0.4.0-alpha/ozonefs.html
+++ b/docs/0.4.0-alpha/ozonefs.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,105 +299,58 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Ozone File System</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<p>There are many Hadoop compatible files systems under Hadoop. Hadoop compatible file systems ensures that storage backends like Ozone can easily be integrated into Hadoop eco-system.</p>
-
<h2 id="setting-up-the-ozone-file-system">Setting up the Ozone file system</h2>
-
<p>To create an ozone file system, we have to choose a bucket where the file system would live. This bucket will be used as the backend store for OzoneFileSystem. All the files and directories will be stored as keys in this bucket.</p>
-
<p>Please run the following commands to create a volume and bucket, if you don’t have them already.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone sh volume create /volume
-ozone sh bucket create /volume/bucket</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>ozone sh volume create /volume
+</span></span><span style="display:flex;"><span>ozone sh bucket create /volume/bucket</span></span></code></pre></div>
<p>Once this is created, please make sure that bucket exists via the listVolume or listBucket commands.</p>
-
<p>Please add the following entry to the core-site.xml.</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>fs.o3fs.impl<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>org.apache.hadoop.fs.ozone.OzoneFileSystem<span style="color:#f92672"></value></span>
-<span style="color:#f92672"></property></span>
-<span style="color:#f92672"><property></span>
- <span style="color:#f92672"><name></span>fs.defaultFS<span style="color:#f92672"></name></span>
- <span style="color:#f92672"><value></span>o3fs://bucket.volume<span style="color:#f92672"></value></span>
-<span style="color:#f92672"></property></span></code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-xml" data-lang="xml"><span style="display:flex;"><span><span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>fs.o3fs.impl<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>org.apache.hadoop.fs.ozone.OzoneFileSystem<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"></property></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><name></span>fs.defaultFS<span style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span style="color:#f92672"><value></span>o3fs://bucket.volume<span style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span><span style="color:#f92672"></property></span></span></span></code></pre></div>
<p>This will make this bucket to be the default file system for HDFS dfs commands and register the o3fs file system type.</p>
-
<p>You also need to add the ozone-filesystem.jar file to the classpath:</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">export HADOOP_CLASSPATH<span style="color:#f92672">=</span>/opt/ozone/share/ozonefs/lib/hadoop-ozone-filesystem-lib-current*.jar:$HADOOP_CLASSPATH</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>export HADOOP_CLASSPATH<span style="color:#f92672">=</span>/opt/ozone/share/ozonefs/lib/hadoop-ozone-filesystem-lib-current*.jar:$HADOOP_CLASSPATH</span></span></code></pre></div>
<p>Once the default Filesystem has been setup, users can run commands like ls, put, mkdir, etc.
For example,</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">hdfs dfs -ls /</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>hdfs dfs -ls /</span></span></code></pre></div>
<p>or</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">hdfs dfs -mkdir /users</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>hdfs dfs -mkdir /users</span></span></code></pre></div>
<p>Or put command etc. In other words, all programs like Hive, Spark, and Distcp will work against this file system.
Please note that any keys created/deleted in the bucket using methods apart from OzoneFileSystem will show up as directories and files in the Ozone File System.</p>
-
<p>Note: Bucket and volume names are not allowed to have a period in them.
Moreover, the filesystem URI can take a fully qualified form with the OM host and port as a part of the path following the volume name.
For example,</p>
-
-<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">hdfs dfs -ls o3fs://bucket.volume.om-host.example.com:5678/key</code></pre></div>
-
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>hdfs dfs -ls o3fs://bucket.volume.om-host.example.com:5678/key</span></span></code></pre></div>
<h2 id="supporting-older-hadoop-version-legacy-jar-basicozonefilesystem">Supporting older Hadoop version (Legacy jar, BasicOzoneFilesystem)</h2>
-
<p>There are two ozonefs files, both of them include all the dependencies:</p>
-
<ul>
<li>share/ozone/lib/hadoop-ozone-filesystem-lib-current-VERSION.jar</li>
<li>share/ozone/lib/hadoop-ozone-filesystem-lib-legacy-VERSION.jar</li>
</ul>
-
<p>The first one contains all the required dependency to use ozonefs with a
- compatible hadoop version (hadoop 3.2).</p>
-
+compatible hadoop version (hadoop 3.2).</p>
<p>The second one contains all the dependency in an internal, separated directory,
- and a special class loader is used to load all the classes from the location.</p>
-
+and a special class loader is used to load all the classes from the location.</p>
<p>With this method the hadoop-ozone-filesystem-lib-legacy.jar can be used from
- any older hadoop version (eg. hadoop 3.1, hadoop 2.7 or spark+hadoop 2.7)</p>
-
+any older hadoop version (eg. hadoop 3.1, hadoop 2.7 or spark+hadoop 2.7)</p>
<p>Similar to the dependency jar, there are two OzoneFileSystem implementation.</p>
-
<p>For hadoop 3.0 and newer, you can use <code>org.apache.hadoop.fs.ozone.OzoneFileSystem</code>
- which is a full implementation of the Hadoop compatible File System API.</p>
-
+which is a full implementation of the Hadoop compatible File System API.</p>
<p>For Hadoop 2.x you should use the Basic version: <code>org.apache.hadoop.fs.ozone.BasicOzoneFileSystem</code>.</p>
-
<p>This is the same implementation but doesn’t include the features/dependencies which are added with
- Hadoop 3.0. (eg. FS statistics, encryption zones).</p>
-
+Hadoop 3.0. (eg. FS statistics, encryption zones).</p>
<h3 id="summary">Summary</h3>
-
<p>The following table summarize which jar files and implementation should be used:</p>
-
<table>
<thead>
<tr>
@@ -384,37 +359,40 @@ For example,</p>
<th>OzoneFileSystem implementation</th>
</tr>
</thead>
-
<tbody>
<tr>
<td>3.2</td>
<td>filesystem-lib-current</td>
<td>org.apache.hadoop.fs.ozone.OzoneFileSystem</td>
</tr>
-
<tr>
<td>3.1</td>
<td>filesystem-lib-legacy</td>
<td>org.apache.hadoop.fs.ozone.OzoneFileSystem</td>
</tr>
-
<tr>
<td>2.9</td>
<td>filesystem-lib-legacy</td>
<td>org.apache.hadoop.fs.ozone.BasicOzoneFileSystem</td>
</tr>
-
<tr>
<td>2.7</td>
<td>filesystem-lib-legacy</td>
<td>org.apache.hadoop.fs.ozone.BasicOzoneFileSystem</td>
</tr>
+<tr>
+<td>With this method the hadoop-ozone-filesystem-lib-legacy.jar can be used from</td>
+<td></td>
+<td></td>
+</tr>
+<tr>
+<td>any older hadoop version (eg. hadoop 2.7 or spark+hadoop 2.7)</td>
+<td></td>
+<td></td>
+</tr>
</tbody>
</table>
-<p>With this method the hadoop-ozone-filesystem-lib-legacy.jar can be used from
- any older hadoop version (eg. hadoop 2.7 or spark+hadoop 2.7)</p>
-
</div>
</div>
</div>
diff --git a/docs/0.4.0-alpha/ozonemanager.html b/docs/0.4.0-alpha/ozonemanager.html
index 6d91bc8..a55c208 100644
--- a/docs/0.4.0-alpha/ozonemanager.html
+++ b/docs/0.4.0-alpha/ozonemanager.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,72 +299,33 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Ozone Manager</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h2 id="om-overview">OM Overview</h2>
-
<p>Ozone Manager or OM is the namespace manager for Ozone. The clients (RPC clients, Rest proxy, Ozone file system, etc.) communicate with OM to create and delete various ozone objects.</p>
-
<p>Each ozone volume is the root of a namespace under OM. This is very different from HDFS which provides a single rooted file system.</p>
-
<p>Ozone’s namespace is a collection of volumes or is a forest instead of a
single rooted tree as in HDFS. This property makes it easy to deploy multiple
- OMs for scaling, this feature is under development.</p>
-
+OMs for scaling, this feature is under development.</p>
<h2 id="om-metadata">OM Metadata</h2>
-
<p>Conceptually, OM maintains a list of volumes, buckets, and keys. For each user, it maintains a list of volumes. For each volume, the list of buckets and for each bucket the list of keys.</p>
-
<p>Right now, OM is a single instance service. Ozone already relies on Apache Ratis (A Replicated State Machine based on Raft protocol). OM will be extended to replicate all its metadata via Ratis. With that, OM will be highly available.</p>
-
<h2 id="om-ui">OM UI</h2>
-
-<p>OM supports a simple UI for the time being. The default port of OM is 9874. To access the OM UI, the user can connect to <a href="http://OM:port">http://OM:port</a> or for a concrete example,</p>
-
-<pre><code>http://omserver:9874/
-</code></pre>
-
-<p>OM UI primarily tries to measure load and latency of OM. The first section of OM UI relates to the number of operations seen by the cluster broken down by the object, operation and whether the operation was successful.</p>
-
+<p>OM supports a simple UI for the time being. The default port of OM is 9874. To access the OM UI, the user can connect to http://OM:port or for a concrete example,</p>
+<pre tabindex="0"><code>http://omserver:9874/
+</code></pre><p>OM UI primarily tries to measure load and latency of OM. The first section of OM UI relates to the number of operations seen by the cluster broken down by the object, operation and whether the operation was successful.</p>
<p>The latter part of the UI is focused on latency and number of operations that OM is performing.</p>
-
<p>One of the hardest problems in HDFS world is discovering the numerous settings offered to tune HDFS. Ozone solves that problem by tagging the configs. To discover settings, click on “Common Tools”->Config. This will take you to the ozone config UI.</p>
-
<h2 id="config-ui">Config UI</h2>
-
<p>The ozone config UI is a matrix with row representing the tags, and columns representing All, OM and SCM.</p>
-
<p>Suppose a user wanted to discover the required settings for ozone. Then the user can tick the checkbox that says “Required.”
This will filter out all “Required” settings along with the description of what each setting does.</p>
-
<p>The user can combine different checkboxes and UI will combine the results. That is, If you have more than one row selected, then all keys for those chosen tags are displayed together.</p>
-
<p>We are hopeful that this leads to a more straightforward way of discovering settings that manage ozone.</p>
-
<h2 id="om-and-scm">OM and SCM</h2>
-
<p><a href="./hdds.html">Storage container manager</a> or (SCM) is the block manager
- for ozone. When a client requests OM for a set of data nodes to write data, OM talks to SCM and gets a block.</p>
-
+for ozone. When a client requests OM for a set of data nodes to write data, OM talks to SCM and gets a block.</p>
<p>A block returned by SCM contains a pipeline, which is a set of nodes that we participate in that block replication.</p>
-
<p>So OM is dependent on SCM for reading and writing of Keys. However, OM is independent of SCM while doing metadata operations like ozone volume or bucket operations.</p>
</div>
diff --git a/docs/0.4.0-alpha/ozonesecurityarchitecture.html b/docs/0.4.0-alpha/ozonesecurityarchitecture.html
index f49bf0f..cf63a86 100644
--- a/docs/0.4.0-alpha/ozonesecurityarchitecture.html
+++ b/docs/0.4.0-alpha/ozonesecurityarchitecture.html
@@ -17,6 +17,28 @@
<link href="css/ozonedoc.css" rel="stylesheet">
+
+ <script>
+ var _paq = window._paq = window._paq || [];
+
+
+
+ _paq.push(['disableCookies']);
+
+
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="//analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '34']);
+ var d=document, g=d.createElement('script'),
+s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+
+
</head>
@@ -277,112 +299,64 @@
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
<h1>Ozone Security Overview</h1>
<div class="col-md-9">
-
-
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
+ <!-- raw HTML omitted -->
+<!-- raw HTML omitted -->
<h1 id="security-in-ozone">Security in Ozone</h1>
-
<p>Starting with badlands release (ozone-0.4.0-alpha) ozone cluster can be secured against external threats. Specifically it can be configured for following security features:</p>
-
<ol>
<li>Authentication</li>
<li>Authorization</li>
<li>Audit</li>
<li>Transparent Data Encryption (TDE)</li>
</ol>
-
<h2 id="authentication">Authentication</h2>
-
<h3 id="kerberos">Kerberos</h3>
-
<p>Similar to hadoop, Ozone allows kerberos-based authentication. So one way to setup identities for all the daemons and clients is to create kerberos keytabs and configure it like any other service in hadoop.</p>
-
<h3 id="tokens">Tokens</h3>
-
<p>Tokens are widely used in Hadoop to achieve lightweight authentication without compromising on security. Main motivation for using tokens inside Ozone is to prevent the unauthorized access while keeping the protocol lightweight and without sharing secret over the wire. Ozone utilizes three types of token:</p>
-
<h4 id="delegation-token">Delegation token</h4>
-
<p>Once client establishes their identity via kerberos they can request a delegation token from OzoneManager. This token can be used by a client to prove its identity until the token expires. Like Hadoop delegation tokens, an Ozone delegation token has 3 important fields:</p>
-
<ol>
<li><strong>Renewer</strong>: User responsible for renewing the token.</li>
<li><strong>Issue date</strong>: Time at which token was issued.</li>
<li><strong>Max date</strong>: Time after which token can’t be renewed.</li>
</ol>
-
<p>Token operations like get, renew and cancel can only be performed over an Kerberos authenticated connection. Clients can use delegation token to establish connection with OzoneManager and perform any file system/object store related operations like, listing the objects in a bucket or creating a volume etc.</p>
-
<h4 id="block-tokens">Block Tokens</h4>
-
<p>Block tokens are similar to delegation tokens in sense that they are signed by OzoneManager. Block tokens are created by OM (OzoneManager) when a client request involves interaction with DataNodes such as read/write Ozone keys.</p>
-
<p>Unlike delegation tokens there is no client API to request block tokens. Instead, they are handed transparently to client along with key/block locations. Block tokens are validated by Datanodes when receiving read/write requests from clients. Block token can’t be renewed explicitly by client. Client with expired block token will need to refetch the key/block locations to get new block tokens.</p>
-
<h4 id="s3token">S3Token</h4>
-
<p>Like block tokens S3Tokens are handled transparently for clients. It is signed by S3secret created by client. S3Gateway creates this token for every s3 client request. To create an S3Token user must have a S3 secret.</p>
-
<h3 id="certificates">Certificates</h3>
-
<p>Apart from kerberos and tokens Ozone utilizes certificate based authentication for Ozone service components. To enable this, SCM (StorageContainerManager) bootstraps itself as an Certificate Authority when security is enabled. This allows all daemons inside Ozone to have an SCM signed certificate. Below is brief descriptions of steps involved:</p>
-
<ol>
<li>Datanodes and OzoneManagers submits a CSR (certificate signing request) to SCM.</li>
<li>SCM verifies identity of DN (Datanode) or OM via Kerberos and generates a certificate.</li>
<li>This certificate is used by OM and DN to prove their identities.</li>
<li>Datanodes use OzoneManager certificate to validate block tokens. This is possible because both of them trust SCM signed certificates. (i.e OzoneManager and Datanodes)</li>
</ol>
-
<h2 id="authorization">Authorization</h2>
-
<p>Ozone provides a pluggable API to control authorization of all client related operations. Default implementation allows every request. Clearly it is not meant for production environments. To configure a more fine grained policy one may configure Ranger plugin for Ozone. Since it is a pluggable module clients can also implement their own custom authorization policy and configure it using <code>ozone.acl.authorizer.class</code>.</p>
-
<h2 id="audit">Audit</h2>
-
<p>Ozone provides ability to audit all read & write operations to OM, SCM and Datanodes. Ozone audit leverages the Marker feature which enables user to selectively audit only READ or WRITE operations by a simple config change without restarting the service(s).</p>
-
<p>To enable/disable audit of READ operations, set filter.read.onMatch to NEUTRAL or DENY respectively. Similarly, the audit of WRITE operations can be controlled using filter.write.onMatch.</p>
-
... 9267 lines suppressed ...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ozone.apache.org
For additional commands, e-mail: commits-help@ozone.apache.org