You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@crail.apache.org by pe...@apache.org on 2018/09/06 11:01:08 UTC
[3/5] incubator-crail git commit: Documentation: readthedocs
Documentation
Documentation: readthedocs Documentation
New Documentation using readthedocs based on github README.
Extended and adapted to latest Crail version.
Will be hosted at http://incubator-crail.readthedocs.io
https://issues.apache.org/jira/projects/CRAIL/issues/CRAIL-24
Signed-off-by: Jonas Pfefferle <pe...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/incubator-crail/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-crail/commit/f4d3147a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-crail/tree/f4d3147a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-crail/diff/f4d3147a
Branch: refs/heads/master
Commit: f4d3147afb1dd1003495dfa97abe89a2a3c16568
Parents: 7342de9
Author: Jonas Pfefferle <pe...@apache.org>
Authored: Fri Aug 10 10:25:01 2018 +0200
Committer: Jonas Pfefferle <pe...@apache.org>
Committed: Thu Sep 6 12:59:41 2018 +0200
----------------------------------------------------------------------
doc/Makefile | 20 ++++
doc/logo/crail_logo.png | Bin 0 -> 14782 bytes
doc/source/conf.py | 168 +++++++++++++++++++++++++++
doc/source/config.rst | 237 +++++++++++++++++++++++++++++++++++++++
doc/source/contact.rst | 6 +
doc/source/contribute.rst | 4 +
doc/source/deploy.rst | 9 ++
doc/source/docker.rst | 93 +++++++++++++++
doc/source/index.rst | 34 ++++++
doc/source/introduction.rst | 15 +++
doc/source/run.rst | 126 +++++++++++++++++++++
doc/source/source.rst | 24 ++++
12 files changed, 736 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/Makefile
----------------------------------------------------------------------
diff --git a/doc/Makefile b/doc/Makefile
new file mode 100644
index 0000000..2a475c3
--- /dev/null
+++ b/doc/Makefile
@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS =
+SPHINXBUILD = sphinx-build
+SPHINXPROJ = ApacheCrailincubating
+SOURCEDIR = source
+BUILDDIR = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+ @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+ @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/logo/crail_logo.png
----------------------------------------------------------------------
diff --git a/doc/logo/crail_logo.png b/doc/logo/crail_logo.png
new file mode 100644
index 0000000..1ca0ce5
Binary files /dev/null and b/doc/logo/crail_logo.png differ
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/source/conf.py
----------------------------------------------------------------------
diff --git a/doc/source/conf.py b/doc/source/conf.py
new file mode 100644
index 0000000..a9d35e9
--- /dev/null
+++ b/doc/source/conf.py
@@ -0,0 +1,168 @@
+# -*- coding: utf-8 -*-
+#
+# Configuration file for the Sphinx documentation builder.
+#
+# This file does only contain a selection of the most common options. For a
+# full list see the documentation:
+# http://www.sphinx-doc.org/en/master/config
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = u'Apache Crail incubating'
+html_logo = '../logo/crail_logo.png'
+copyright = u'2018, Apache Software Foundation'
+author = u'_'
+
+# The short X.Y version
+version = u''
+# The full version, including alpha/beta/rc tags
+release = u'1.0'
+
+
+# -- General configuration ---------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#
+# needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+ 'sphinx.ext.autodoc',
+ 'sphinx.ext.autosectionlabel',
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The master toctree document.
+master_doc = 'index'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path .
+exclude_patterns = []
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages. See the documentation for
+# a list of builtin themes.
+#
+import os
+on_rtd = os.environ.get('READTHEDOCS') == 'True'
+if on_rtd:
+ html_theme = 'default'
+else:
+ import sphinx_rtd_theme
+ html_theme = 'sphinx_rtd_theme'
+ html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further. For a list of options available for each theme, see the
+# documentation.
+#
+# html_theme_options = {}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Custom sidebar templates, must be a dictionary that maps document names
+# to template names.
+#
+# The default sidebars (for documents that don't match any pattern) are
+# defined by theme itself. Builtin themes are using these templates by
+# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
+# 'searchbox.html']``.
+#
+# html_sidebars = {}
+
+
+# -- Options for HTMLHelp output ---------------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'ApacheCrailincubatingdoc'
+
+
+# -- Options for LaTeX output ------------------------------------------------
+
+latex_elements = {
+ # The paper size ('letterpaper' or 'a4paper').
+ #
+ # 'papersize': 'letterpaper',
+
+ # The font size ('10pt', '11pt' or '12pt').
+ #
+ # 'pointsize': '10pt',
+
+ # Additional stuff for the LaTeX preamble.
+ #
+ # 'preamble': '',
+
+ # Latex figure (float) alignment
+ #
+ # 'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+# author, documentclass [howto, manual, or own class]).
+latex_documents = [
+ (master_doc, 'ApacheCrailincubating.tex', u'Apache Crail incubating Documentation',
+ u'\\_', 'manual'),
+]
+
+
+# -- Options for manual page output ------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+ (master_doc, 'apachecrailincubating', u'Apache Crail incubating Documentation',
+ [author], 1)
+]
+
+
+# -- Options for Texinfo output ----------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+# dir menu entry, description, category)
+texinfo_documents = [
+ (master_doc, 'ApacheCrailincubating', u'Apache Crail incubating Documentation',
+ author, 'ApacheCrailincubating', 'One line description of project.',
+ 'Miscellaneous'),
+]
+
+
+# -- Extension configuration -------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/source/config.rst
----------------------------------------------------------------------
diff --git a/doc/source/config.rst b/doc/source/config.rst
new file mode 100644
index 0000000..f97c08f
--- /dev/null
+++ b/doc/source/config.rst
@@ -0,0 +1,237 @@
+Configuration
+=============
+
+To configure Crail use the \*.template files as a basis and modify it to match your environment.
+Set the :code:`$CRAIL_HOME` environment variable to your Crail deployment's path.
+
+.. code-block:: bash
+
+ cd $CRAIL_HOME/conf
+ mv crail-site.conf.template crail-site.conf
+ mv crail-env.sh.template crail-env.sh
+ mv core-site.xml.template core-site.xml
+ mv slaves.template slaves
+
+**Note:** *Docker containers can be configured by using config files above. However it is only recommended for complex configurations. See* :ref:`Docker <Docker>` *for details.*
+
+The purpuse of each of these files are:
+
+* :ref:`crail-site.conf`: Configuration of the file system, data tiers and RPC
+* :ref:`crail-env.sh`: Allows to pass additional JVM arguments
+* :ref:`core-site.xml`: Configuration of the HDFS adapter
+* :ref:`slaves`: Used by the start-crail.sh script to ease running Crail on multiple machines
+
+crail-site.conf
+---------------
+There are a general file system properties and specific properties for the different storage tiers. Typical properties you might want to change are:
+
+=============================== ====================== ======================================
+Property Default Value Description
+=============================== ====================== ======================================
+:code:`crail.namenode.address` crail://localhost:9060 Namenode hostname and port
+:code:`crail.cachelimit` 1073741824 Size (byte) of client buffer cache
+:code:`crail.cachepath` /dev/hugepages/cache Hugepage path to client buffer cache
+=============================== ====================== ======================================
+
+Advanced properties (*Only modify if you know what you are doing*):
+
+===================================== ========================================= ===================================================
+Property Default Value Description
+===================================== ========================================= ===================================================
+:code:`crail.directorydepth` 16 Maximum depth of directory tree
+:code:`crail.tokenexpiration` 10 Seconds write token is valid
+:code:`crail.blocksize` 1048576 Size (byte) of block
+:code:`crail.user` crail Username used for HDFS adapter
+:code:`crail.debug` false Enable debug output
+:code:`crail.statistics` true Collect statistics
+:code:`crail.rpctimeout` 1000 RPC timeout in milliseconds
+:code:`crail.datatimeout` 1000 Data operation timeout in milliseconds
+:code:`crail.buffersize` 1048576 Size (byte) of buffer (buffered stream)
+:code:`crail.slicesize` 524288 Size (byte) of slice (transfer unit)
+:code:`crail.singleton` true Only create a single instance of the FS
+:code:`crail.regionsize` 1073741824 Size (byte) of allocation unit (Cache)
+:code:`crail.directoryrecord` 512 Size (byte) of directory entry
+:code:`crail.directoryrandomize` true Randomize iteration of directories
+:code:`crail.cacheimpl` org.apache.crail.memory.MappedBufferCache Client buffer cache implementation
+:code:`crail.namenode.fileblocks` 16 File
+:code:`crail.namenode.blockselection` roundrobin Block selection algorithm: roundrobin or random
+===================================== ========================================= ===================================================
+
+RPC
+~~~
+
+Crail's modular architecture allows to plugin different kinds of RPC implementations. The :code:`crail.namenode.rpctype` property
+is used to configure the RPC implementation. We currently offer two implementations:
+
+* A TCP implementation based on `narpc <https://github.com/zrlio/narpc>`_ (default):
+ **org.apache.crail.namenode.rpc.tcp.tcpnamenode**
+* A RDMA implementation based on `darpc <https://github.com/zrlio/darpc>`_:
+ **org.apache.crail.namenode.rpc.darpc.DaRPCNameNode**
+
+
+Logging
+'''''''
+
+To allow shutting down the namenode without loosing data Crail offers namenode logging.
+It can be enabled by setting a path to the log file with :code:`crail.namenode.log`.
+
+**Note:** *this feature is experimental and should be used with caution*
+
+Storage Tiers
+~~~~~~~~~~~~~
+
+Crail offers multiple types of datanode dependent on your network and storage requirements:
+
+(a) TCP storage tier backed by DRAM (default)
+(b) RDMA storage tier backed by DRAM
+(c) NVMe over Fabrics storage tier, typically backed by NVMe drives
+
+Crail allows to use multiple storage tier types together, e.g. to store hot data on
+DRAM and cold data on NVMe, or extend your DRAM by NVMe storage. Storage types can be
+configured as a comma separated list by setting the :code:`crail.storage.types` property:
+
+(a) TCP: **org.apache.crail.storage.tcp.TcpStorageTier**
+(b) RDMA: **org.apache.crail.storage.rdma.RdmaStorageTier**
+(c) NVMf: **org.apache.crail.storage.nvmf.NvmfStorageTier**
+
+Each of the storage types in the list defines a storage class, starting from storage class 0.
+Types can appear multiple times to allow defining multiple storage classes for a type.
+The maximum number of storage classes needs to be specified with the
+:code:`crail.storage.classes` property (default = 1).
+In the default configuration storage classes are used in incremental order, i.e.
+storage class 0 is used until no more space is left then storage class 1 is used and so on.
+However filesystem nodes (e.g. files) can also be created on a particular storage class and
+can be configured to inherit the storage class of its container. The default storage
+class of `/` is 0 however it can be configured via :code:`crail.storage.rootclass`.
+
+Storage tiers send keep alive messages to the namenode to indicate that they are still
+running and no error has occured. The interval in which keep alive message are
+send can be configured in seconds with :code:`crail.storage.keepalive`.
+
+Some of the configuration properties can be set via the command line when starting
+a storage tier. Refer to :ref:`Run` for details.
+
+TCP Tier
+''''''''
+
+The TCP storage tier (org.apache.crail.storage.tcp.TcpStorageTier) is backed by DRAM. The following
+properties can be set to configure the storage tier:
+
+======================================== ====================== ============================================================
+Property Default Value Description
+======================================== ====================== ============================================================
+:code:`crail.storage.tcp.interface` eth0 Network interface to bind to
+:code:`crail.storage.tcp.storagelimit` 1073741824 Size (Bytes) of DRAM to provide, multiple of allocation size
+:code:`crail.storage.tcp.datapath` /dev/hugepages/data Hugepage path to data
+======================================== ====================== ============================================================
+
+Advanced properties:
+
+======================================== ====================== ==============================================
+Property Default Value Description
+======================================== ====================== ==============================================
+:code:`crail.storage.tcp.port` 50020 Port to listen on
+:code:`crail.storage.tcp.allocationsize` crail.regionsize Allocation unit
+:code:`crail.storage.tcp.queuedepth` 16 Data operation queue depth (single connection)
+:code:`crail.storage.tcp.cores` 1 Threads to process requests
+======================================== ====================== ==============================================
+
+
+RDMA Tier
+'''''''''
+
+The RDMA storage tier (org.apache.crail.storage.rdma.RdmaStorageTier) is backed by DRAM. The following
+properties can be set to configure the storage tier:
+
+======================================== ====================== ============================================================
+Property Default Value Description
+======================================== ====================== ============================================================
+:code:`crail.storage.rdma.interface` eth0 Network interface to bind to
+:code:`crail.storage.rdma.storagelimit` 1073741824 Size (Bytes) of DRAM to provide; multiple of allocation size
+:code:`crail.storage.rdma.datapath` /dev/hugepages/data Hugepage path to data
+======================================== ====================== ============================================================
+
+Advanced properties:
+
+========================================= ====================== ========================================================
+Property Default Value Description
+========================================= ====================== ========================================================
+:code:`crail.storage.rdma.port` 50020 Port to listen on
+:code:`crail.storage.rdma.allocationsize` crail.regionsize Allocation unit
+:code:`crail.storage.rdma.localmap` true Use mmap if client is colocated with data tier
+:code:`crail.storage.rdma.queuesize` 32 Data operation queue depth (single connection)
+:code:`crail.storage.rdma.type` passive Operation type: passive or active (see DiSNI)
+:code:`crail.storage.rdma.persistent` false Allow restarting a data tier if namenode logging is used
+:code:`crail.storage.rdma.backlog` 100 Listen backlog
+:code:`crail.storage.rdma.connecttimeout` 1000 Connect timeout in milliseconds
+========================================= ====================== ========================================================
+
+NVMf Tier
+'''''''''
+
+The NVMf storage tier (org.apache.crail.storage.nvmf.NvmfStorageTier) is typically backed by NVMe drives. However some target
+implementations support using any block device. Unlike the RDMA and TCP storage tier the NVMf storage tier is not involved
+in any data operation but only is used to provide metadata information. Crail uses the `jNVMf <https://github.com/zrlio/jNVMf>`_
+library to connect to a standard NVMf target to gain metadata information about the storage and provide the information to the namenode.
+Clients directly connect to the NVMf target. Crail has been tested to run with the Linux kernel, SPDK and Mellanox ConnectX-5
+offloading target.
+
+The following properties can be set to configure the storage tier:
+
+======================================== ========================== ============================================================
+Property Default Value Description
+======================================== ========================== ============================================================
+:code:`crail.storage.nvmf.ip` localhost IP/hostname of NVMf target
+:code:`crail.storage.nvmf.port` 50025 Port of NVMf target
+:code:`crail.storage.nvmf.nqn` nqn.2017-06.io.crail:cnode NVMe qualified name of NVMf controller
+:code:`crail.storage.nvmf.namespace` 1 Namespace of NVMe device
+======================================== ========================== ============================================================
+
+Advanced properties:
+
+=========================================== ====================== ==========================================================
+Property Default Value Description
+=========================================== ====================== ==========================================================
+:code:`crail.storage.nvmf.allocationsize` crail.regionsize Allocation unit
+:code:`crail.storage.nvmf.queueSize` 64 NVMf submission queue size
+:code:`crail.storage.nvmf.stagingcachesize` 262144 Staging cache size (byte) for read-modify-write operations
+=========================================== ====================== ==========================================================
+
+crail-env.sh
+------------
+
+Modify crail-env.sh to pass additional JVM arguments to :code:`crail` respectively
+:code:`start-crail.sh`.
+
+It is recommended to increase heap (e.g. :code:`-Xmx24g`) and young generation heap size
+(e.g. :code:`-Xmn16g`) for the namenodes and TCP datanodes to improve performance
+for large deployments.
+
+core-site.xml
+-------------
+
+To configure the HDFS adapter modify core-site.xml. For example the Crail shell
+:code:`crail fs` uses the HDFS adapter thus requiring the core-site.xml file to
+be setup. Modify :code:`fs.defaultFS` to match :code:`crail.namenode.address` in
+:ref:`crail-site.conf`. The default is:
+
+.. code-block:: xml
+
+ <property>
+ <name>fs.defaultFS</name>
+ <value>crail://localhost:9060</value>
+ </property>
+
+
+slaves
+------
+
+The slaves file can be used to ease starting Crail on larger deployments. Refer
+to :ref:`Run` for details. Each line should contain a hostname where a storage
+tier is supposed to be started. Make sure the hostname allows passwordless ssh
+connections. Note that the hostnames are not used by the storage tier itself but
+only by the start/stop-crail.sh scripts to start and stop storage tiers.
+IP/hostname of the storage tiers or any other configuration option are either
+passed by command line arguments or via :ref:`crail-site.conf`.
+Command line arguments can be configured in the slaves file following the hostname.
+
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/source/contact.rst
----------------------------------------------------------------------
diff --git a/doc/source/contact.rst b/doc/source/contact.rst
new file mode 100644
index 0000000..280d642
--- /dev/null
+++ b/doc/source/contact.rst
@@ -0,0 +1,6 @@
+Contact
+=======
+
+Feel free to ask questions any questions on our mailing list: dev@crail.apache.org
+
+If you find any issues please report them at https://issues.apache.org/jira/projects/CRAIL/issues
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/source/contribute.rst
----------------------------------------------------------------------
diff --git a/doc/source/contribute.rst b/doc/source/contribute.rst
new file mode 100644
index 0000000..1a67d33
--- /dev/null
+++ b/doc/source/contribute.rst
@@ -0,0 +1,4 @@
+Contribute
+==========
+
+For any potential changes/proposals we recommend that you open a `JIRA ticket <https://issues.apache.org/jira/projects/CRAIL/issues>`_ to have a discussion. After making necessary code changes, please open a pull request at `Github <https://github.com/apache/incubator-crail>`_, and update the JIRA.
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/source/deploy.rst
----------------------------------------------------------------------
diff --git a/doc/source/deploy.rst b/doc/source/deploy.rst
new file mode 100644
index 0000000..ed2c228
--- /dev/null
+++ b/doc/source/deploy.rst
@@ -0,0 +1,9 @@
+Deploy Crail
+============
+
+.. TODO: fill in binary release url
+
+Download the latest binary image from here and :ref:`configure <Configuration>` it.
+Alternatively you can :ref:`build from source <Building from source>` or use our :ref:`Docker container image <Docker>`.
+
+
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/source/docker.rst
----------------------------------------------------------------------
diff --git a/doc/source/docker.rst b/doc/source/docker.rst
new file mode 100644
index 0000000..9b178d4
--- /dev/null
+++ b/doc/source/docker.rst
@@ -0,0 +1,93 @@
+Docker
+======
+
+The easiest way to run Crail is to use Docker images. We provide two preconfigured
+Docker images:
+
+(1) TCP/DRAM: **apache/crail**
+(2) RDMA/DRAM: **apache/crail_rdma**
+
+If you want to run other or more complex configurations you can use either image
+as a basis and provide your own configuration files.
+Refer to :ref:`Own configurations` for details.
+
+(1) and 2. share the following configuration parameters:
+
+======================================== ====================== ==============================================
+Property Default Value Description
+======================================== ====================== ==============================================
+:code:`NAMENODE_HOST` localhost Namenode hostname/ip to bind to
+:code:`NAMENODE_PORT` 9060 Namenode port to listen on
+:code:`INTERFACE` eth0 Datanode network interface to listen on
+:code:`DATAPATH` /dev/hugepages/data Datanode hugepage path to data
+:code:`STORAGELIMIT` 1073741824 Datanode Size (Bytes) of DRAM to provide
+:code:`CACHEPATH` /dev/hugepages/cache Client/Datanode hugepage path to buffer cache
+:code:`CACHELIMIT` 0 Size (Bytes) of hugepage buffer cache
+======================================== ====================== ==============================================
+
+These properties can be specified as environment variables when
+starting a Docker image with :code:`-e <property>=<value>`.
+
+TCP image
+---------
+
+To run Crail you first need to start the namenode. For example
+
+.. code-block:: bash
+
+ docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 apache/crail namenode
+
+starts a TCP namenode using Docker's host network configuration on host02 on interface eth5.
+The TCP tier allows for other network configurations.
+Refer to https://docs.docker.com/network/ for details.
+
+Once the namenode has started start a Crail TCP storage tier (preferably on a different node).
+For example
+
+
+.. code-block:: bash
+
+ docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 apache/crail datanode
+
+starts a TCP datanode with 1GB of storage (default) listening on eth5. It is recommended
+to put the data directory on a hugetlb mount. You can do this by passing an
+mounted hugetlb directory on the host as a volume to Docker. For example
+
+
+.. code-block:: bash
+
+ docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 -v /dev/hugepages:/dev/hugepages apache/crail datanode
+
+passes the hugetlb mount /dev/hugepages to the container.
+
+RDMA image
+----------
+
+To run Crail/RDMA/DRAM you can start the namenode as follows:
+
+.. code-block:: bash
+
+ docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 --cap-add=IPC_LOCK --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/rdma_cm -v /dev/hugepages:/dev/hugepages apache/crail_rdma namenode
+
+This starts a namenode on host02 using the host's network on eth5. Note that the
+uverbs device needs to match the interface.
+
+To run a RDMA storage tier:
+
+
+.. code-block:: bash
+
+ docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 --cap-add=IPC_LOCK --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/rdma_cm -v /dev/hugepages:/dev/hugepages apache/crail_rdma datanode
+
+**Note:** *The RDMA docker image provides default RDMA provider libraries from Ubuntu 18.04.
+They might not be compatible with your host's RDMA stack.
+To install your own RDMA libraries use -v or create your own Docker image from crail_rdma.*
+
+Own configurations
+------------------
+
+If you want to run more complex configurations that are not covered by the options above you have two options:
+
+(1) Create your own Docker image by creating a Dockerfile and using the crail images as a source. You can then change the configuration by e.g. copying your own config into the image
+(2) Pass your config as a volume with :code:`-v <local_path>:<docker_path>`
+
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/source/index.rst
----------------------------------------------------------------------
diff --git a/doc/source/index.rst b/doc/source/index.rst
new file mode 100644
index 0000000..5e8a965
--- /dev/null
+++ b/doc/source/index.rst
@@ -0,0 +1,34 @@
+Apache Crail incubating
+=======================
+*Apache Crail (Incubating) is a high-performance distributed data store designed for fast sharing of ephemeral data in distributed data processing workloads.*
+
+.. toctree::
+ :maxdepth: 1
+ :caption: Overview
+
+ introduction
+
+.. toctree::
+ :maxdepth: 1
+ :caption: Deployment
+
+ deploy
+ source
+ docker
+ config
+ run
+
+.. toctree::
+ :maxdepth: 1
+ :caption: Getting started
+
+.. toctree::
+ :maxdepth: 1
+ :caption: API
+
+.. toctree::
+ :maxdepth: 1
+ :caption: Help
+
+ contribute
+ contact
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/source/introduction.rst
----------------------------------------------------------------------
diff --git a/doc/source/introduction.rst b/doc/source/introduction.rst
new file mode 100644
index 0000000..46d1dba
--- /dev/null
+++ b/doc/source/introduction.rst
@@ -0,0 +1,15 @@
+Introduction
+============
+
+
+Apache Crail (Incubating) is a fast multi-tiered distributed storage system designed from ground up for high-performance network and storage hardware. The unique features of Crail include:
+
+* Zero-copy network access from userspace
+* Integration of multiple storage tiers such DRAM, flash and disaggregated shared storage
+* Ultra-low latencies for both meta data and data operations. For instance: opening, reading and closing a small file residing in the distributed DRAM tier less than 10 microseconds, which is in the same ballpark as some of the fastest RDMA-based key/value stores
+* High-performance sequential read/write operations: For instance: read operations on large files residing in the distributed DRAM tier are typically limited only by the performance of the network
+* Very low CPU consumption: a single core sharing both application and file system client can drive sequential read/write operations at the speed of up to 100Gbps and more
+* Asynchronous API leveraging the asynchronous nature of RDMA-based networking hardware
+* Extensible plugin architecture: new storage tiers tailored to specific hardware can be added easily
+
+Crail is implemented in Java offering a Java API which integrates directly with the Java off-heap memory. Crail is designed for performance critical temporary data within a scope of a rack or two.
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/source/run.rst
----------------------------------------------------------------------
diff --git a/doc/source/run.rst b/doc/source/run.rst
new file mode 100644
index 0000000..f998cf8
--- /dev/null
+++ b/doc/source/run.rst
@@ -0,0 +1,126 @@
+Run
+===
+
+For all deployments, make sure the :code:`$CRAIL_HOME` environment variable is
+set on each machine to point to the top level Crail directory.
+
+Starting Crail manually
+-----------------------
+The simplest way to run Crail is to start it manually on just a handful nodes.
+You will need to start the Crail namenode, plus at least one datanode.
+To start the namenode execute the following command on the host that is configured to be the namenode:
+
+.. code-block:: bash
+
+ $CRAIL_HOME/crail namenode
+
+To start a datanode run the following command on a host in the cluster
+(ideally this is a different physical machine than the one running the namenode):
+
+.. code-block:: bash
+
+ $CRAIL_HOME/crail datanode
+
+Now you should have a small deployment up with just one datanode.
+In this case the datanode is of type TCP/DRAM, which is the default datnode.
+If you want to start a different storage tier you can do so by passing a specific
+storage tier type.
+You can find a list of supported storage tiers :ref:`here <Storage Tiers>`. For example:
+
+.. code-block:: bash
+
+ $CRAIL_HOME/crail datanode -t org.apache.crail.storage.nvmf.NvmfStorageTier
+
+starts the NVMf datanode. Note that configuration in :ref:`crail-site.conf` needs
+to have the specific properties set of this type of datanode, in order for this to work.
+Some storage tiers allow to set :ref:`configuration <Storage Tier Command Line>`
+properties on the command line which can be appended after :code:`--` to the command line, e.g.:
+
+.. code-block:: bash
+
+ $CRAIL_HOME/crail datanode -t org.apache.crail.storage.nvmf.NvmfStorageTier -- -a 192.168.0.2
+
+Each storage tier instance can only belong to one storage class however the same
+storage tier type can belong to multiple storage classes. Refer to :ref:`Storage Tiers`
+for details. If there is only one storage class per type the storage class
+is picked by the order in which they appear in :code:`crail.storage.types`
+(:ref:`crail-site.conf`). Use :code:`-c <storage_class>` To start a storage tier
+in a specific storage class, e.g.:
+
+
+.. code-block:: bash
+
+ $CRAIL_HOME/crail datanode -t org.apache.crail.storage.nvmf.NvmfStorageTier -c 1
+
+starts a NVMf storage tier in storage class 1 (storage classes start from 0).
+
+Storage Tier Command Line
+--------------------------
+
+Command line arguments of the storage tiers override configuration
+properties in :ref:`crail-site.conf`. Refer to :ref:`crail-site.conf` for
+a detailed explanation of the properties and their default values.
+
+TCP
+~~~
+
+================== =================================
+Argument crail-site.conf
+================== =================================
+:code:`-p <port>` :code:`crail.storage.tcp.port`
+:code:`-c <cores>` :code:`crail.storage.tcp.cores`
+================== =================================
+
+RDMA
+~~~~
+
+====================== =====================================
+Argument crail-site.conf
+====================== =====================================
+:code:`-i <interface>` :code:`crail.storage.rdma.interface`
+:code:`-p <port>` :code:`crail.storage.rdma.port`
+:code:`-s` :code:`crail.storage.rdma.persistent`
+====================== =====================================
+
+NVMf
+~~~~
+
+========================= =====================================
+Argument crail-site.conf/Description
+========================= =====================================
+:code:`-a <ip/hostname>` :code:`crail.storage.nvmf.ip`
+:code:`-p <port>` :code:`crail.storage.nvmf.port`
+:code:`-nqn <nqn>` :code:`crail.storage.nvmf.nqn`
+:code:`-n <namespace_id>` Namespace id to use (default 1)
+========================= =====================================
+
+Larger deployments
+------------------
+To run larger deployments start Crail using
+
+.. code-block:: bash
+
+ $CRAIL_HOME/start-crail.sh
+
+Similarly, Crail can be stopped by using
+
+.. code-block:: bash
+
+ $CRAIL_HOME/stop-crail.sh
+
+For this to work include the list of machines to start datanodes in the :ref:`slaves` file.
+You can start multiple datanode of different types on the same host as follows:
+
+.. code-block:: bash
+
+ host02
+ host02 -t org.apache.crail.storage.nvmf.NvmfStorageTier -- -a 192.168.0.2
+ host03
+
+In this example, we are configuring a Crail cluster with 2 physical hosts but 3 datanodes and two different storage tiers.
+
+Starting Crail in Docker
+------------------------
+
+Refer to :ref:`Docker` for how to run Crail in a Docker container.
+
http://git-wip-us.apache.org/repos/asf/incubator-crail/blob/f4d3147a/doc/source/source.rst
----------------------------------------------------------------------
diff --git a/doc/source/source.rst b/doc/source/source.rst
new file mode 100644
index 0000000..661d71c
--- /dev/null
+++ b/doc/source/source.rst
@@ -0,0 +1,24 @@
+Building from source
+====================
+
+Follow the steps below to build Crail from source.
+
+Requirements
+------------
+
+* Java 8 or higher
+* RDMA-based network, e.g., Infiniband, iWARP, RoCE. There are two options to run Crail without RDMA networking hardware: (a) use SoftiWARP, (b) us the TCP/DRAM storage tier
+* Libdisni.so, available as part of `DiSNI <https://github.com/zrlio/disni>`_
+
+Building
+--------
+
+To build Crail from source using `Apache Maven <http://maven.apache.org>`_ execute the following steps:
+
+1. (a) Clone from Apache: :code:`git clone http://git-wip-us.apache.org/repos/asf/incubator-crail.git` or
+ (b) Clone from Github: :code:`git clone https://github.com/apache/incubator-crail` or
+ (c) Download and unpack the latest source release from `here <http://crail.apache.org/download>`_
+2. Run: :code:`mvn -DskipTests install`
+3. Copy tarball to the cluster and unpack it using :code:`tar xvfz crail-XX-bin.tar.gz`
+
+**Note:** *later, when deploying Crail, make sure libdisni.so is part of your LD_LIBRARY_PATH. The easiest way to make it work is to copy libdisni.so into crail-1.0/lib*