You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@warble.apache.org by hu...@apache.org on 2018/06/25 15:04:39 UTC
[incubator-warble-server] branch master updated: Start on some
documentation
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-warble-server.git
The following commit(s) were added to refs/heads/master by this push:
new 907fd3f Start on some documentation
907fd3f is described below
commit 907fd3f96f62d4ea6875b5ca185101ebaeaa6102
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Mon Jun 25 10:04:25 2018 -0500
Start on some documentation
---
docs/.gitignore | 1 +
docs/Makefile | 20 +++
docs/source/_static/images/warble-logo.png | Bin 0 -> 267056 bytes
docs/source/conf.py | 170 +++++++++++++++++++++++++
docs/source/index.rst | 17 +++
docs/source/setup.rst | 196 +++++++++++++++++++++++++++++
6 files changed, 404 insertions(+)
diff --git a/docs/.gitignore b/docs/.gitignore
new file mode 100644
index 0000000..378eac2
--- /dev/null
+++ b/docs/.gitignore
@@ -0,0 +1 @@
+build
diff --git a/docs/Makefile b/docs/Makefile
new file mode 100644
index 0000000..68be7b6
--- /dev/null
+++ b/docs/Makefile
@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS =
+SPHINXBUILD = sphinx-build
+SPHINXPROJ = ApacheWarble
+SOURCEDIR = source
+BUILDDIR = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+ @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+ @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
diff --git a/docs/source/_static/images/warble-logo.png b/docs/source/_static/images/warble-logo.png
new file mode 100644
index 0000000..ced2f98
Binary files /dev/null and b/docs/source/_static/images/warble-logo.png differ
diff --git a/docs/source/conf.py b/docs/source/conf.py
new file mode 100644
index 0000000..6bd322a
--- /dev/null
+++ b/docs/source/conf.py
@@ -0,0 +1,170 @@
+# -*- coding: utf-8 -*-
+#
+# Apache Warble documentation build configuration file, created by
+# sphinx-quickstart on Thu Jan 11 06:05:51 2018.
+#
+# This file is execfile()d with the current directory set to its
+# containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+
+
+# -- General configuration ------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#
+# needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = ['sphinx.ext.todo',
+ 'sphinx.ext.imgmath']
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = u'Apache Warble'
+copyright = u'2018, The Apache Warble Community'
+author = u'The Apache Warble Community'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+version = u'0.1'
+# The full version, including alpha/beta/rc tags.
+release = u'0.1'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This patterns also effect to html_static_path and html_extra_path
+exclude_patterns = []
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = True
+
+
+# -- Options for HTML output ----------------------------------------------
+
+# The theme to use for HTML and HTML Help pages. See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'sphinx_rtd_theme'
+html_logo = '_static/images/warble-logo.png'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further. For a list of options available for each theme, see the
+# documentation.
+#
+# html_theme_options = {}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Custom sidebar templates, must be a dictionary that maps document names
+# to template names.
+#
+# This is required for the alabaster theme
+# refs: http://alabaster.readthedocs.io/en/latest/installation.html#sidebars
+html_sidebars = {
+ '**': [
+ 'relations.html', # needs 'show_related': True theme option to display
+ 'searchbox.html',
+ ]
+}
+
+
+# -- Options for HTMLHelp output ------------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'ApacheWarbledoc'
+
+
+# -- Options for LaTeX output ---------------------------------------------
+
+latex_elements = {
+ # The paper size ('letterpaper' or 'a4paper').
+ #
+ # 'papersize': 'letterpaper',
+
+ # The font size ('10pt', '11pt' or '12pt').
+ #
+ # 'pointsize': '10pt',
+
+ # Additional stuff for the LaTeX preamble.
+ #
+ # 'preamble': '',
+
+ # Latex figure (float) alignment
+ #
+ # 'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+# author, documentclass [howto, manual, or own class]).
+latex_documents = [
+ (master_doc, 'ApacheWarble.tex', u'Apache Warble Documentation',
+ u'The Apache Warble Community', 'manual'),
+]
+
+
+# -- Options for manual page output ---------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+ (master_doc, 'apachewarble', u'Apache Warble Documentation',
+ [author], 1)
+]
+
+
+# -- Options for Texinfo output -------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+# dir menu entry, description, category)
+texinfo_documents = [
+ (master_doc, 'ApacheWarble', u'Apache Warble Documentation',
+ author, 'ApacheWarble', 'One line description of project.',
+ 'Miscellaneous'),
+]
+
+
+
diff --git a/docs/source/index.rst b/docs/source/index.rst
new file mode 100644
index 0000000..abed454
--- /dev/null
+++ b/docs/source/index.rst
@@ -0,0 +1,17 @@
+
+Welcome to Apache Warble's documentation!
+=========================================
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Contents:
+
+ setup
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
diff --git a/docs/source/setup.rst b/docs/source/setup.rst
new file mode 100644
index 0000000..449fdc1
--- /dev/null
+++ b/docs/source/setup.rst
@@ -0,0 +1,196 @@
+Setting up Apache Warble
+========================
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Contents:
+
+
+****************************
+Understanding the Components
+****************************
+
+Warble currently consists of two major components:
+
+The Warble Master Server (warble-server)
+ This is the main database and UI Server. It serves as the hub for the
+ nodes/agents to connect to, and provides the overall management of
+ hosts, tests, as well as the visualizations and API end points.
+
+The Warble Node Applications (warble-node)
+ This is a daemon with a collection of test classes used to test
+ external hosts for various services and/or response values. Nodes
+ send results back to the master, which then processes and responds
+ accordingly (for instance, in the case of downtime).
+
+A third major component, the Warble Agent Applications, are being
+worked on, but is not completed.
+
+**********************
+Component Requirements
+**********************
+
+################
+Server Component
+################
+
+The main Warble Server is a hub for nodes/agents and tests, and as such, is
+generally speaking only needed on one machine. It is recommended that, for larger
+instances of warble, you place the application on a machine or VM with
+sufficient resources to handle the database load and memory requirements.
+
+We will be working towards a multi-master setup option, but that is
+currently not available.
+
+As a rule of thumb, the Server does not require a lot of disk space
+(enough to hold the compiled database and timeseries), but it does require CPU and RAM.
+The nodes/agents require virtually no disk space, as all test results are sent
+to the master server for storage.
+
+#################
+Node Component
+#################
+
+The node component can either consist of one instance, or be spread
+out across multiple machines for a distributed test coverage.
+Nodes will auto-adjust the test speed to match the number of CPU cores available to it;
+a node with two cores available will run up to 256 simultaneous jobs, whereas a scanner with
+eight cores would run up to 1024 simultaneous jobs to speed up processing.
+A node will typically require somewhere between 256 and 512MB of memory,
+and thus can safely run on a VM with 2GB memory (or less).
+
+
+********************
+Source Code Location
+********************
+
+.. This needs to change once we have released Warble
+
+*Apache Warble does not currently have any releases.*
+*You are however welcome to try out the development version.*
+
+For the time being, we recommend that you use the ``master`` branch for
+testing Warble. This applies to both scanners and the server.
+
+The Warble Server can be found via our source repository at
+https://github.com/apache/incubator-warble-server
+
+The Warble Node Application can be found via:
+https://github.com/apache/incubator-warble-node
+
+
+*********************
+Installing the Server
+*********************
+
+###############
+Pre-requisites
+###############
+
+Before you install the Warble Server, please ensure you have the
+following components installed and set up:
+
+- A web server of your choice (Apache HTTP Server, NGINX, lighttp etc)
+- Python 3.4 or newer with the following libraries installed:
+- - yaml
+- - certifi
+- - sqlite3
+- - bcrypt
+- Gunicorn for Python 3.x (often called gunicorn3) or mod_wsgi
+
+###########################################
+Configuring and Priming the Warble Server
+###########################################
+Once you have the components installed and Warble Server downloaded, you will
+need to prime the databases and create a configuration file.
+
+Assuming you wish to install warble in /opt/warble, you would set it
+up by issuing the following:
+
+- ``git clone https://github.com/apache/incubator-warble-server.git /opt/warble``
+- ``cd /opt/warble/setup``
+- ``python3 setup.py``
+- Enter the configuration parameters the setup process asks for
+
+This will set up the database, the configuration file, and create your
+initial administrator account for the UI. You can later on do additional
+configuration of the data server by editing the ``api/yaml/warble.yaml``
+file.
+
+#####################
+Setting up the Web UI
+#####################
+
+Once you have finished the initial setup, you will need to enable the
+web UI. Warble is built as a WSGI application, and as such you can
+use mod_wsgi for apache, or proxy to Gunicorn. In this example, we will
+be using the Apache HTTP Server and proxy to Gunicorn:
+
+- Make sure you have mod_proxy and mod_proxy_http loaded (on
+ debian/ubuntu, you would run: `a2enmod proxy_http`)
+- Set up a virtual host in Apache:
+
+::
+
+ <VirtualHost *:80>
+ # Set this to your domain, or add warble.localhost to /etc/hosts
+ ServerName warble.localhost
+ DocumentRoot /opt/warble/ui/
+ # Proxy to gunicorn for /api/ below:
+ ProxyPass /api/ http://localhost:8000/api/
+ </VirtualHost>
+
+- Launch gunicorn as a daemon on port 8000:
+
+::
+
+ cd /opt/warble/api
+ gunicorn -w 10 -b 127.0.0.1:8000 handler:application -t 120 -D
+
+Once httpd is (re)started, you should be able to browse to your new
+Warble instance.
+
+
+*******************
+Installing Nodes
+*******************
+
+##############
+Pre-requisites
+##############
+
+
+The Warble Nodes rely on the following packages:
+
+- Python >= 3.4 with the following packages:
+- - python3-yaml
+- - python3-ldap
+- - python3-dns
+
+Custom node tests may require additional packages.
+
+###########################
+Configuring a node
+###########################
+
+First, check out the node source in a file path of your choosing:
+
+``git clone https://github.com/apache/incubator-warble-node.git``
+
+Then edit the ``conf/config.yaml`` file to point towards the
+proper Warble Master server.
+
+Then fire up the node software as a daemon:
+
+``python3 node.py start``
+
+Warble Node apps will, when run the first time, set up an async
+key pair for encryption and verification, and request a spot in
+the Warble Master node registry. Spots are verified/approved in the
+Warble UI, and once completed, the node will receive an API key
+that corresponds with its ID and key pair, and get to work.
+It is worth noting, that the Warble node software needs write access
+to the configuration directory on disk, so it can store the API key and
+async key pair.
+
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@warble.apache.org
For additional commands, e-mail: commits-help@warble.apache.org