You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@metron.apache.org by ce...@apache.org on 2017/03/17 14:47:09 UTC
[01/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Repository: incubator-metron
Updated Branches:
refs/heads/master f1435a6a7 -> 39059e5be
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-platform/metron-parsers/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-parsers/index.html b/site/current-book/metron-platform/metron-parsers/index.html
new file mode 100644
index 0000000..10bbd15
--- /dev/null
+++ b/site/current-book/metron-platform/metron-parsers/index.html
@@ -0,0 +1,691 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Parsers</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Parsers</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-down"></i>
+ Platform</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-platform/metron-api/index.html" title="Api">
+ <i class="none"></i>
+ Api</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-common/index.html" title="Common">
+ <i class="none"></i>
+ Common</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
+ <i class="none"></i>
+ Data-management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
+ <i class="none"></i>
+ Enrichment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
+ <i class="none"></i>
+ Indexing</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-management/index.html" title="Management">
+ <i class="none"></i>
+ Management</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Parsers</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
+ <i class="none"></i>
+ Pcap-backend</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Parsers</h1>
+<p><a name="Parsers"></a></p>
+<div class="section">
+<h2><a name="Introduction"></a>Introduction</h2>
+<p>Parsers are pluggable components which are used to transform raw data (textual or raw bytes) into JSON messages suitable for downstream enrichment and indexing. </p>
+<p>There are two general types types of parsers:</p>
+
+<ul>
+
+<li>A parser written in Java which conforms to the <tt>MessageParser</tt> interface. This kind of parser is optimized for speed and performance and is built for use with higher velocity topologies. These parsers are not easily modifiable and in order to make changes to them the entire topology need to be recompiled.</li>
+
+<li>A general purpose parser. This type of parser is primarily designed for lower-velocity topologies or for quickly standing up a parser for a new telemetry before a permanent Java parser can be written for it. As of the time of this writing, we have:
+
+<ul>
+
+<li>Grok parser: <tt>org.apache.metron.parsers.GrokParser</tt> with possible <tt>parserConfig</tt> entries of
+
+<ul>
+
+<li><tt>grokPath</tt> : The path in HDFS (or in the Jar) to the grok statement</li>
+
+<li><tt>patternLabel</tt> : The pattern label to use from the grok statement</li>
+
+<li><tt>timestampField</tt> : The field to use for timestamp</li>
+
+<li><tt>timeFields</tt> : A list of fields to be treated as time</li>
+
+<li><tt>dateFormat</tt> : The date format to use to parse the time fields</li>
+
+<li><tt>timezone</tt> : The timezone to use. <tt>UTC</tt> is default.</li>
+ </ul></li>
+
+<li>CSV Parser: <tt>org.apache.metron.parsers.csv.CSVParser</tt> with possible <tt>parserConfig</tt> entries of
+
+<ul>
+
+<li><tt>timestampFormat</tt> : The date format of the timestamp to use. If unspecified, the parser assumes the timestamp is ms since unix epoch.</li>
+
+<li><tt>columns</tt> : A map of column names you wish to extract from the CSV to their offsets (e.g. <tt>{ 'name' : 1, 'profession' : 3}</tt> would be a column map for extracting the 2nd and 4th columns from a CSV)</li>
+
+<li><tt>separator</tt> : The column separator, <tt>,</tt> by default. just</li>
+ </ul></li>
+ </ul></li>
+</ul></div>
+<div class="section">
+<h2><a name="Parser_Architecture"></a>Parser Architecture</h2>
+<p><img src="../../images/parser_arch.png" alt="Architecture" /></p>
+<p>Data flows through the parser bolt via kafka and into the <tt>enrichments</tt> topology in kafka. Errors are collected with the context of the error (e.g. stacktrace) and original message causing the error and sent to an <tt>error</tt> queue. Invalid messages as determined by global validation functions are sent to an <tt>invalid</tt> queue. </p></div>
+<div class="section">
+<h2><a name="Message_Format"></a>Message Format</h2>
+<p>All Metron messages follow a specific format in order to ingest a message. If a message does not conform to this format it will be dropped and put onto an error queue for further examination. The message must be of a JSON format and must have a JSON tag message like so:</p>
+
+<div class="source">
+<div class="source">
+<pre>{"message" : message content}
+
+</pre></div></div>
+<p>Where appropriate there is also a standardization around the 5-tuple JSON fields. This is done so the topology correlation engine further down stream can correlate messages from different topologies by these fields. We are currently working on expanding the message standardization beyond these fields, but this feature is not yet availabe. The standard field names are as follows:</p>
+
+<ul>
+
+<li>ip_src_addr: layer 3 source IP</li>
+
+<li>ip_dst_addr: layer 3 dest IP</li>
+
+<li>ip_src_port: layer 4 source port</li>
+
+<li>ip_dst_port: layer 4 dest port</li>
+
+<li>protocol: layer 4 protocol</li>
+
+<li>timestamp (epoch)</li>
+
+<li>original_string: A human friendly string representation of the message</li>
+</ul>
+<p>The timestamp and original_string fields are madatory. The remaining standard fields are optional. If any of the optional fields are not applicable then the field should be left out of the JSON.</p>
+<p>So putting it all together a typical Metron message with all 5-tuple fields present would look like the following:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+"message":
+{"ip_src_addr": xxxx,
+"ip_dst_addr": xxxx,
+"ip_src_port": xxxx,
+"ip_dst_port": xxxx,
+"protocol": xxxx,
+"original_string": xxx,
+"additional-field 1": xxx,
+}
+
+}
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Global_Configuration"></a>Global Configuration</h2>
+<p>See the “<a href="../metron-common/index.html">Global Configuration</a>” section.</p></div>
+<div class="section">
+<h2><a name="Parser_Configuration"></a>Parser Configuration</h2>
+<p>The configuration for the various parser topologies is defined by JSON documents stored in zookeeper.</p>
+<p>The document is structured in the following way</p>
+
+<ul>
+
+<li><tt>parserClassName</tt> : The fully qualified classname for the parser to be used.</li>
+
+<li><tt>filterClassName</tt> : The filter to use. This may be a fully qualified classname of a Class that implements the <tt>org.apache.metron.parsers.interfaces.MessageFilter<JSONObject></tt> interface. Message Filters are intended to allow the user to ignore a set of messages via custom logic. The existing implementations are:
+
+<ul>
+
+<li><tt>STELLAR</tt> : Allows you to apply a stellar statement which returns a boolean, which will pass every message for which the statement returns <tt>true</tt>. The Stellar statement that is to be applied is specified by the <tt>filter.query</tt> property in the <tt>parserConfig</tt>. Example Stellar Filter which includes messages which contain a the <tt>field1</tt> field:</li>
+ </ul></li>
+</ul>
+
+<div class="source">
+<div class="source">
+<pre> {
+ "filterClassName" : "STELLAR"
+ ,"parserConfig" : {
+ "filter.query" : "exists(field1)"
+ }
+ }
+</pre></div></div>
+
+<ul>
+
+<li><tt>sensorTopic</tt> : The kafka topic to send the parsed messages to.</li>
+
+<li><tt>parserConfig</tt> : A JSON Map representing the parser implementation specific configuration.</li>
+
+<li><tt>fieldTransformations</tt> : An array of complex objects representing the transformations to be done on the message generated from the parser before writing out to the kafka topic.</li>
+</ul>
+<p>The <tt>fieldTransformations</tt> is a complex object which defines a transformation which can be done to a message. This transformation can </p>
+
+<ul>
+
+<li>Modify existing fields to a message</li>
+
+<li>Add new fields given the values of existing fields of a message</li>
+
+<li>Remove existing fields of a message</li>
+</ul>
+<div class="section">
+<h3><a name="fieldTransformation_configuration"></a><tt>fieldTransformation</tt> configuration</h3>
+<p>The format of a <tt>fieldTransformation</tt> is as follows:</p>
+
+<ul>
+
+<li><tt>input</tt> : An array of fields or a single field representing the input. This is optional; if unspecified, then the whole message is passed as input.</li>
+
+<li><tt>output</tt> : The outputs to produce from the transformation. If unspecified, it is assumed to be the same as inputs.</li>
+
+<li><tt>transformation</tt> : The fully qualified classname of the transformation to be used. This is either a class which implements <tt>FieldTransformation</tt> or a member of the <tt>FieldTransformations</tt> enum.</li>
+
+<li><tt>config</tt> : A String to Object map of transformation specific configuration.</li>
+</ul>
+<p>The currently implemented fieldTransformations are:</p>
+
+<ul>
+
+<li><tt>REMOVE</tt> : This transformation removes the specified input fields. If you want a conditional removal, you can pass a Metron Query Language statement to define the conditions under which you want to remove the fields.</li>
+</ul>
+<p>Consider the following simple configuration which will remove <tt>field1</tt> unconditionally:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+...
+ "fieldTransformations" : [
+ {
+ "input" : "field1"
+ , "transformation" : "REMOVE"
+ }
+ ]
+}
+</pre></div></div>
+<p>Consider the following simple sensor parser configuration which will remove <tt>field1</tt> whenever <tt>field2</tt> exists and whose corresponding equal to ‘foo’:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+...
+ "fieldTransformations" : [
+ {
+ "input" : "field1"
+ , "transformation" : "REMOVE"
+ , "config" : {
+ "condition" : "exists(field2) and field2 == 'foo'"
+ }
+ }
+ ]
+}
+</pre></div></div>
+
+<ul>
+
+<li><tt>IP_PROTOCOL</tt> : This transformation maps IANA protocol numbers to consistent string representations.</li>
+</ul>
+<p>Consider the following sensor parser config to map the <tt>protocol</tt> field to a textual representation of the protocol:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+...
+ "fieldTransformations" : [
+ {
+ "input" : "protocol"
+ , "transformation" : "IP_PROTOCOL"
+ }
+ ]
+}
+</pre></div></div>
+<p>This transformation would transform <tt>{ "protocol" : 6, "source.type" : "bro", ... }</tt> into <tt>{ "protocol" : "TCP", "source.type" : "bro", ...}</tt></p>
+
+<ul>
+
+<li><tt>STELLAR</tt> : This transformation executes a set of transformations expressed as <a href="../metron-common/index.html">Stellar Language</a> statements.</li>
+</ul>
+<p>Consider the following sensor parser config to add three new fields to a message:</p>
+
+<ul>
+
+<li><tt>utc_timestamp</tt> : The unix epoch timestamp based on the <tt>timestamp</tt> field, a <tt>dc</tt> field which is the data center the message comes from and a <tt>dc2tz</tt> map mapping data centers to timezones</li>
+
+<li><tt>url_host</tt> : The host associated with the url in the <tt>url</tt> field</li>
+
+<li><tt>url_protocol</tt> : The protocol associated with the url in the <tt>url</tt> field</li>
+</ul>
+
+<div class="source">
+<div class="source">
+<pre>{
+...
+ "fieldTransformations" : [
+ {
+ "transformation" : "STELLAR"
+ ,"output" : [ "utc_timestamp", "url_host", "url_protocol" ]
+ ,"config" : {
+ "utc_timestamp" : "TO_EPOCH_TIMESTAMP(timestamp, 'yyyy-MM-dd
+HH:mm:ss', MAP_GET(dc, dc2tz, 'UTC') )"
+ ,"url_host" : "URL_TO_HOST(url)"
+ ,"url_protocol" : "URL_TO_PROTOCOL(url)"
+ }
+ }
+ ]
+ ,"parserConfig" : {
+ "dc2tz" : {
+ "nyc" : "EST"
+ ,"la" : "PST"
+ ,"london" : "UTC"
+ }
+ }
+}
+</pre></div></div>
+<p>Note that the <tt>dc2tz</tt> map is in the parser config, so it is accessible in the functions.</p></div>
+<div class="section">
+<h3><a name="An_Example_Configuration_for_a_Sensor"></a>An Example Configuration for a Sensor</h3>
+<p>Consider the following example configuration for the <tt>yaf</tt> sensor:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "parserClassName":"org.apache.metron.parsers.GrokParser",
+ "sensorTopic":"yaf",
+ "fieldTransformations" : [
+ {
+ "input" : "protocol"
+ ,"transformation": "IP_PROTOCOL"
+ }
+ ],
+ "parserConfig":
+ {
+ "grokPath":"/patterns/yaf",
+ "patternLabel":"YAF_DELIMITED",
+ "timestampField":"start_time",
+ "timeFields": ["start_time", "end_time"],
+ "dateFormat":"yyyy-MM-dd HH:mm:ss.S"
+ }
+}
+</pre></div></div></div></div>
+<div class="section">
+<h2><a name="Parser_Adapters"></a>Parser Adapters</h2>
+<p>Parser adapters are loaded dynamically in each Metron topology. They are defined in the Parser Config (defined above) JSON file in Zookeeper.</p>
+<div class="section">
+<h3><a name="Java_Parser_Adapters"></a>Java Parser Adapters</h3>
+<p>Java parser adapters are indended for higher-velocity topologies and are not easily changed or extended. As the adoption of Metron continues we plan on extending our library of Java adapters to process more log formats. As of this moment the Java adapters included with Metron are:</p>
+
+<ul>
+
+<li>org.apache.metron.parsers.ise.BasicIseParser : Parse ISE messages</li>
+
+<li>org.apache.metron.parsers.bro.BasicBroParser : Parse Bro messages</li>
+
+<li>org.apache.metron.parsers.sourcefire.BasicSourcefireParser : Parse Sourcefire messages</li>
+
+<li>org.apache.metron.parsers.lancope.BasicLancopeParser : Parse Lancope messages</li>
+</ul></div>
+<div class="section">
+<h3><a name="Grok_Parser_Adapters"></a>Grok Parser Adapters</h3>
+<p>Grok parser adapters are designed primarly for someone who is not a Java coder for quickly standing up a parser adapter for lower velocity topologies. Grok relies on Regex for message parsing, which is much slower than purpose-built Java parsers, but is more extensible. Grok parsers are defined via a config file and the topplogy does not need to be recombiled in order to make changes to them. An example of a Grok perser is:</p>
+
+<ul>
+
+<li>org.apache.metron.parsers.GrokParser</li>
+</ul>
+<p>For more information on the Grok project please refer to the following link:</p>
+<p><a class="externalLink" href="https://github.com/thekrakken/java-grok">https://github.com/thekrakken/java-grok</a></p>
+<p><a name="Starting_the_Parser_Topology"></a></p>
+<h1>Starting the Parser Topology</h1>
+<p>Starting a particular parser topology on a running Metron deployment is as easy as running the <tt>start_parser_topology.sh</tt> script located in <tt>$METRON_HOME/bin</tt>. This utility will allow you to configure and start the running topology assuming that the sensor specific parser configuration exists within zookeeper.</p>
+<p>The usage for <tt>start_parser_topology.sh</tt> is as follows:</p>
+
+<div class="source">
+<div class="source">
+<pre>usage: start_parser_topology.sh
+ -e,--extra_topology_options <JSON_FILE> Extra options in the form
+ of a JSON file with a map
+ for content.
+ -esc,--extra_kafka_spout_config <JSON_FILE> Extra spout config options
+ in the form of a JSON file
+ with a map for content.
+ Possible keys are:
+ retryDelayMaxMs,retryDelay
+ Multiplier,retryInitialDel
+ ayMs,stateUpdateIntervalMs
+ ,bufferSizeBytes,fetchMaxW
+ ait,fetchSizeBytes,maxOffs
+ etBehind,metricsTimeBucket
+ SizeInSecs,socketTimeoutMs
+ -ewnt,--error_writer_num_tasks <NUM_TASKS> Error Writer Num Tasks
+ -ewp,--error_writer_p <PARALLELISM_HINT> Error Writer Parallelism
+ Hint
+ -h,--help This screen
+ -iwnt,--invalid_writer_num_tasks <NUM_TASKS> Invalid Writer Num Tasks
+ -iwp,--invalid_writer_p <PARALLELISM_HINT> Invalid Message Writer
+ Parallelism Hint
+ -k,--kafka <BROKER_URL> Kafka Broker URL
+ -mt,--message_timeout <TIMEOUT_IN_SECS> Message Timeout in Seconds
+ -mtp,--max_task_parallelism <MAX_TASK> Max task parallelism
+ -na,--num_ackers <NUM_ACKERS> Number of Ackers
+ -nw,--num_workers <NUM_WORKERS> Number of Workers
+ -pnt,--parser_num_tasks <NUM_TASKS> Parser Num Tasks
+ -pp,--parser_p <PARALLELISM_HINT> Parser Parallelism Hint
+ -s,--sensor <SENSOR_TYPE> Sensor Type
+ -snt,--spout_num_tasks <NUM_TASKS> Spout Num Tasks
+ -sp,--spout_p <SPOUT_PARALLELISM_HINT> Spout Parallelism Hint
+ -t,--test <TEST> Run in Test Mode
+ -z,--zk <ZK_QUORUM> Zookeeper Quroum URL
+ (zk1:2181,zk2:2181,...
+</pre></div></div></div></div>
+<div class="section">
+<h2><a name="The_--extra_kafka_spout_config_Option"></a>The <tt>--extra_kafka_spout_config</tt> Option</h2>
+<p>These options are intended to configure the Storm Kafka Spout more completely. These options can be specified in a JSON file containing a map associating the kafka spout configuration parameter to a value. The range of values possible to configure are:</p>
+
+<ul>
+
+<li>retryDelayMaxMs</li>
+
+<li>retryDelayMultiplier</li>
+
+<li>retryInitialDelayMs</li>
+
+<li>stateUpdateIntervalMs</li>
+
+<li>bufferSizeBytes</li>
+
+<li>fetchMaxWait</li>
+
+<li>fetchSizeBytes</li>
+
+<li>maxOffsetBehind</li>
+
+<li>metricsTimeBucketSizeInSecs</li>
+
+<li>socketTimeoutMs</li>
+</ul>
+<p>These are described in some detail <a class="externalLink" href="https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_storm-user-guide/content/storm-kafka-api-ref.html">here</a>.</p>
+<p>For instance, creating a JSON file which will set the <tt>bufferSizeBytes</tt> to 2MB and <tt>retryDelayMaxMs</tt> to 2000 would look like</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "bufferSizeBytes" : 2000000,
+ "retryDelayMaxMs" : 2000
+}
+</pre></div></div>
+<p>This would be loaded by passing the file as argument to <tt>--extra_kafka_spout_config</tt></p></div>
+<div class="section">
+<h2><a name="The_--extra_topology_options_Option"></a>The <tt>--extra_topology_options</tt> Option</h2>
+<p>These options are intended to be Storm configuration options and will live in a JSON file which will be loaded into the Storm config. For instance, if you wanted to set a storm property on the config called <tt>topology.ticks.tuple.freq.secs</tt> to 1000 and <tt>storm.local.dir</tt> to <tt>/opt/my/path</tt> you could create a file called <tt>custom_config.json</tt> containing </p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "topology.ticks.tuple.freq.secs" : 1000,
+ "storm.local.dir" : "/opt/my/path"
+}
+</pre></div></div>
+<p>and pass <tt>--extra_topology_options custom_config.json</tt> to <tt>start_parser_topology.sh</tt>.</p>
+<p><a name="Notes_on_Performance_Tuning"></a></p>
+<h1>Notes on Performance Tuning</h1>
+<p>Default installed Metron is untuned for production deployment. There are a few knobs to tune to get the most out of your system.</p></div>
+<div class="section">
+<h2><a name="Kafka_Queue"></a>Kafka Queue</h2>
+<p>The kafka queue associated with your parser is a collection point for all of the data sent to your parser. As such, make sure that the number of partitions in the kafka topic is sufficient to handle the throughput that you expect from your parser topology.</p></div>
+<div class="section">
+<h2><a name="Parser_Topology"></a>Parser Topology</h2>
+<p>The enrichment topology as started by the <tt>$METRON_HOME/bin/start_parser_topology.sh</tt> script uses a default of one executor per bolt. In a real production system, this should be customized by modifying the arguments sent to this utility.</p>
+
+<ul>
+
+<li>Topology Wide
+
+<ul>
+
+<li><tt>--num_workers</tt> : The number of workers for the topology</li>
+
+<li><tt>--num_ackers</tt> : The number of ackers for the topology</li>
+ </ul></li>
+
+<li>The Kafka Spout
+
+<ul>
+
+<li><tt>--spout_num_tasks</tt> : The number of tasks for the spout</li>
+
+<li><tt>--spout_p</tt> : The parallelism hint for the spout</li>
+
+<li>Ensure that the spout has enough parallelism so that it can dedicate a worker per partition in your kafka topic.</li>
+ </ul></li>
+
+<li>The Parser Bolt
+
+<ul>
+
+<li><tt>--parser_num_tasks</tt> : The number of tasks for the parser bolt</li>
+
+<li><tt>--parser_p</tt> : The parallelism hint for the spout</li>
+
+<li>This is bolt that gets the most processing, so ensure that it is configured with sufficient parallelism to match your throughput expectations.</li>
+ </ul></li>
+
+<li>The Error Message Writer Bolt
+
+<ul>
+
+<li><tt>--error_writer_num_tasks</tt> : The number of tasks for the error writer bolt</li>
+
+<li><tt>--error_writer_p</tt> : The parallelism hint for the error writer bolt</li>
+ </ul></li>
+
+<li>The Invalid Message Writer Bolt
+
+<ul>
+
+<li><tt>--invalid_writer_num_tasks</tt> : The number of tasks for the error writer bolt</li>
+
+<li><tt>--invalid_writer_p</tt> : The parallelism hint for the error writer bolt</li>
+ </ul></li>
+</ul>
+<p>Finally, if workers and executors are new to you, the following might be of use to you:</p>
+
+<ul>
+
+<li><a class="externalLink" href="http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/">Understanding the Parallelism of a Storm Topology</a></li>
+</ul></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-platform/metron-pcap-backend/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-pcap-backend/index.html b/site/current-book/metron-platform/metron-pcap-backend/index.html
new file mode 100644
index 0000000..a36dc50
--- /dev/null
+++ b/site/current-book/metron-platform/metron-pcap-backend/index.html
@@ -0,0 +1,376 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Metron PCAP Backend</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Metron PCAP Backend</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-down"></i>
+ Platform</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-platform/metron-api/index.html" title="Api">
+ <i class="none"></i>
+ Api</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-common/index.html" title="Common">
+ <i class="none"></i>
+ Common</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
+ <i class="none"></i>
+ Data-management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
+ <i class="none"></i>
+ Enrichment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
+ <i class="none"></i>
+ Indexing</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-management/index.html" title="Management">
+ <i class="none"></i>
+ Management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
+ <i class="none"></i>
+ Parsers</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Pcap-backend</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Metron PCAP Backend</h1>
+<p><a name="Metron_PCAP_Backend"></a></p>
+<p>The purpose of the Metron PCAP backend is to create a storm topology capable of ingesting rapidly raw packet capture data directly into HDFS from Kafka.</p>
+<div class="section">
+<h2><a name="The_Sensors_Feeding_Kafka"></a>The Sensors Feeding Kafka</h2>
+<p>This component must be fed by fast packet capture components upstream via Kafka. The two supported components shipped with Metron are as follows:</p>
+
+<ul>
+
+<li>The pycapa <a href="../../metron-sensors/pycapa/index.html">tool</a> aimed at low-volume packet capture</li>
+
+<li>The <a class="externalLink" href="http://dpdk.org/">DPDK</a> based <a href="../../metron-sensors/fastcapa/index.html">tool</a> aimed at high-volume packet capture</li>
+</ul>
+<p>Both of these sensors feed kafka raw packet data directly into Kafka. The format of the record structure that this component expects is the following:</p>
+
+<ul>
+
+<li>A key which is the byte representation of a 64-bit <tt>unsigned long</tt> representing a time-unit since the unix epoch</li>
+
+<li>A value which is the raw packet data without header (either global pcap header or packet header)</li>
+</ul></div>
+<div class="section">
+<h2><a name="The_PCAP_Topology"></a>The PCAP Topology</h2>
+<p>The structure of the topology is extremely simple. In fact, it is a spout-only topology. The <tt>Storm Kafka</tt> spout is used but extended to allow a callback to be used rather than having a separate bolt. </p>
+<p>The following happens as part of this spout for each packet:</p>
+
+<ul>
+
+<li>A custom <tt>Scheme</tt> is used which attaches the appropriate headers to the packet (both global and packet headers) using the timestamp in the key and the raw packet data in the value.</li>
+
+<li>A callback is called which appends the packet data to a sequence file in HDFS.</li>
+</ul></div>
+<div class="section">
+<h2><a name="The_Files_on_HDFS"></a>The Files on HDFS</h2>
+<p>The sequence files on HDFS fit the following pattern: <tt>$BASE_PATH/pcap_$TOPIC_$TS_$PARTITION_$UUID</tt></p>
+<p>where</p>
+
+<ul>
+
+<li><tt>BASE_PATH</tt> is the base path to where pcap data is stored in HDFS</li>
+
+<li><tt>TOPIC</tt> is the kafka topic</li>
+
+<li><tt>TS</tt> is the timestamp, in nanoseconds since the unix epoch</li>
+
+<li><tt>PARTITION</tt> is the kafka partition</li>
+
+<li><tt>UUID</tt> the UUID for the storm worker</li>
+</ul>
+<p>These files contain a set of packet data with headers on them in sequence files.</p></div>
+<div class="section">
+<h2><a name="Configuration"></a>Configuration</h2>
+<p>The configuration file for the Flux topology is located at <tt>$METRON_HOME/config/etc/env/pcap.properties</tt> and the possible options are as follows:</p>
+
+<ul>
+
+<li><tt>spout.kafka.topic.pcap</tt> : The kafka topic to listen to</li>
+
+<li><tt>kafka.zk</tt> : The comma separated zookeeper quorum (i.e. host:2181,host2:2181)</li>
+
+<li><tt>kafka.pcap.start</tt> : One of <tt>START</tt>, <tt>END</tt>, <tt>WHERE_I_LEFT_OFF</tt> representing where to start listening on the queue.</li>
+
+<li><tt>kafka.pcap.numPackets</tt> : The number of packets to keep in one file.</li>
+
+<li><tt>kafka.pcap.maxTimeMS</tt> : The number of packets to keep in one file in terms of duration (in milliseconds). For instance, you may only want to keep an hour’s worth of packets in a given file.</li>
+
+<li><tt>kafka.pcap.ts_scheme</tt> : One of <tt>FROM_KEY</tt> or <tt>FROM_VALUE</tt>. You really only want <tt>FROM_KEY</tt> as that fits the current tooling. <tt>FROM_VALUE</tt> assumes that fully headerized packets are coming in on the value, which is legacy.</li>
+
+<li><tt>kafka.pcap.out</tt> : The directory in HDFS to store the packet capture data</li>
+
+<li><tt>kafka.pcap.ts_granularity</tt> : The granularity of timing used in the timestamps. One of <tt>MILLISECONDS</tt>, <tt>MICROSECONDS</tt>, or <tt>NANOSECONDS</tt> representing milliseconds, microseconds or nanoseconds since the unix epoch (respectively).</li>
+</ul></div>
+<div class="section">
+<h2><a name="Starting_the_Topology"></a>Starting the Topology</h2>
+<p>To assist in starting the topology, a utility script which takes no arguments has been created to make this very simple. Simply, execute <tt>$METRON_HOME/bin/start_pcap_topology.sh</tt>.</p></div>
+<div class="section">
+<h2><a name="Utilities"></a>Utilities</h2>
+<div class="section">
+<h3><a name="Inspector_Utility"></a>Inspector Utility</h3>
+<p>In order to ensure that data can be read back out, a utility, <tt>$METRON_HOME/bin/pcap_inspector.sh</tt> has been created to read portions of the sequence files.</p>
+
+<div class="source">
+<div class="source">
+<pre>usage: PcapInspector
+ -h,--help Generate Help screen
+ -i,--input <SEQ_FILE> Input sequence file on HDFS
+ -n,--num_packets <N> Number of packets to dump
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Query_Filter_Utility"></a>Query Filter Utility</h3>
+<p>This tool exposes the two methods for filtering PCAP data via a command line tool:</p>
+
+<ul>
+
+<li>fixed</li>
+
+<li>query (Metron Stellar)</li>
+</ul>
+<p>The tool is executed via </p>
+
+<div class="source">
+<div class="source">
+<pre>${metron_home}/bin/pcap_query.sh [fixed|query]
+</pre></div></div>
+<div class="section">
+<h4><a name="Usage"></a>Usage</h4>
+
+<div class="source">
+<div class="source">
+<pre>usage: Fixed filter options
+ -bop,--base_output_path <arg> Query result output path. Default is
+ '/tmp'
+ -bp,--base_path <arg> Base PCAP data path. Default is
+ '/apps/metron/pcap'
+ -da,--ip_dst_addr <arg> Destination IP address
+ -df,--date_format <arg> Date format to use for parsing start_time
+ and end_time. Default is to use time in
+ millis since the epoch.
+ -dp,--ip_dst_port <arg> Destination port
+ -et,--end_time <arg> Packet end time range. Default is current
+ system time.
+ -nr,--num_reducers <arg> The number of reducers to use. Default
+ is 10.
+ -h,--help Display help
+ -ir,--include_reverse Indicates if filter should check swapped
+ src/dest addresses and IPs
+ -p,--protocol <arg> IP Protocol
+ -sa,--ip_src_addr <arg> Source IP address
+ -sp,--ip_src_port <arg> Source port
+ -st,--start_time <arg> (required) Packet start time range.
+</pre></div></div>
+
+<div class="source">
+<div class="source">
+<pre>usage: Query filter options
+ -bop,--base_output_path <arg> Query result output path. Default is
+ '/tmp'
+ -bp,--base_path <arg> Base PCAP data path. Default is
+ '/apps/metron/pcap'
+ -df,--date_format <arg> Date format to use for parsing start_time
+ and end_time. Default is to use time in
+ millis since the epoch.
+ -et,--end_time <arg> Packet end time range. Default is current
+ system time.
+ -nr,--num_reducers <arg> The number of reducers to use. Default
+ is 10.
+ -h,--help Display help
+ -q,--query <arg> Query string to use as a filter
+ -st,--start_time <arg> (required) Packet start time range.
+</pre></div></div></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-sensors/fastcapa/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-sensors/fastcapa/index.html b/site/current-book/metron-sensors/fastcapa/index.html
new file mode 100644
index 0000000..a12f6e5
--- /dev/null
+++ b/site/current-book/metron-sensors/fastcapa/index.html
@@ -0,0 +1,195 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Fastcapa</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Fastcapa</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-down"></i>
+ Sensors</a>
+ <ul class="nav nav-list">
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Fastcapa</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/pycapa/index.html" title="Pycapa">
+ <i class="none"></i>
+ Pycapa</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Fastcapa</h1>
+<p>Fastcapa is an Apache Metron sensor that performs fast network packet capture by leveraging Linux kernel-bypass and user space networking technology. </p>
+<p>The sensor will bind to a network interface, capture network packets, and send the raw packet data to Kafka. This provides a scalable mechanism for ingesting high-volumes of network packet data into a Hadoop-y cluster.</p>
+<p>Fastcapa leverages the Data Plane Development Kit (<a class="externalLink" href="http://dpdk.org/">DPDK</a>). DPDK is a set of libraries and drivers to perform fast packet processing in Linux user space. </p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>The quickest way to get up and running is to use a Virtualbox environment on your local machine. The necessary files and instructions to do this are located at <a href="../../metron-deployment/vagrant/fastcapa-test-platform/index.html"><tt>metron-deployment/vagrant/fastcapa-vagrant</tt></a>. </p></div>
+<div class="section">
+<h2><a name="Installation"></a>Installation</h2>
+<p>The process of installing Fastcapa has a fair number of steps and involves building DPDK, loading specific kernel modules, enabling huge page memory, and binding compatible network interface cards.</p>
+<p>The best documentation is code that actually does this for you. An Ansible role that performs the entire installation procedure can be found at <a href="../../metron-deployment/roles/fastcapa/index.html"><tt>metron-deployment/roles/fastcapa</tt></a>. Use this to install Fastcapa or as a guide for manual installation.</p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-sensors/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-sensors/index.html b/site/current-book/metron-sensors/index.html
new file mode 100644
index 0000000..a340bbe
--- /dev/null
+++ b/site/current-book/metron-sensors/index.html
@@ -0,0 +1,193 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Metron Sensors</title>
+ <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../css/site.css" />
+ <link rel="stylesheet" href="../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Metron Sensors</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="icon-chevron-down"></i>Sensors</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../metron-sensors/fastcapa/index.html" title="Fastcapa">
+ <i class="none"></i>
+ Fastcapa</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-sensors/pycapa/index.html" title="Pycapa">
+ <i class="none"></i>
+ Pycapa</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Metron Sensors</h1>
+<p><a name="Metron_Sensors"></a></p>
+
+<ul>
+
+<li>Fast CAPA</li>
+
+<li>Py CAPA</li>
+</ul>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-sensors/pycapa/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-sensors/pycapa/index.html b/site/current-book/metron-sensors/pycapa/index.html
new file mode 100644
index 0000000..0841338
--- /dev/null
+++ b/site/current-book/metron-sensors/pycapa/index.html
@@ -0,0 +1,262 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Pycapa</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Pycapa</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-down"></i>
+ Sensors</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-sensors/fastcapa/index.html" title="Fastcapa">
+ <i class="none"></i>
+ Fastcapa</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Pycapa</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Pycapa</h1>
+<p><a name="Pycapa"></a></p>
+<div class="section">
+<h2><a name="Overview"></a>Overview</h2>
+<p>Pycapa performs network packet capture, both off-the-wire and from Kafka, which is useful for the testing and development of <a class="externalLink" href="https://github.com/apache/incubator-metron">Apache Metron</a>. It is not intended for production use. The tool will capture packets from a specified interface and push them into a Kafka Topic.</p></div>
+<div class="section">
+<h2><a name="Installation"></a>Installation</h2>
+
+<div class="source">
+<div class="source">
+<pre>pip install -r requirements.txt
+python setup.py install
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Usage"></a>Usage</h2>
+
+<div class="source">
+<div class="source">
+<pre>$ pycapa --help
+usage: pycapa [-h] [-p] [-c] [-k KAFKA_BROKERS] [-t TOPIC] [-n PACKET_COUNT]
+ [-d DEBUG] [-i INTERFACE]
+
+optional arguments:
+ -h, --help show this help message and exit
+ -p, --producer sniff packets and send to kafka
+ -c, --consumer read packets from kafka
+ -k KAFKA_BROKERS, --kafka KAFKA_BROKERS
+ kafka broker(s)
+ -t TOPIC, --topic TOPIC
+ kafka topic
+ -n PACKET_COUNT, --number PACKET_COUNT
+ number of packets to consume
+ -d DEBUG, --debug DEBUG
+ debug every X packets
+ -i INTERFACE, --interface INTERFACE
+ interface to listen on
+</pre></div></div>
+<p>Pycapa has two primary runtime modes.</p>
+<div class="section">
+<h3><a name="Producer_Mode"></a>Producer Mode</h3>
+<p>Pycapa can be configured to capture packets from a network interface and then forward those packets to a Kafka topic. The following example will capture packets from the <tt>eth0</tt> network interface and forward those to a Kafka topic called <tt>pcap</tt> running on <tt>localhost</tt>.</p>
+
+<div class="source">
+<div class="source">
+<pre>pycapa --producer --kafka localhost:9092 --topic pcap -i eth0
+</pre></div></div>
+<p>To output debug messages every 100 captured packets, run the following.</p>
+
+<div class="source">
+<div class="source">
+<pre>pycapa --producer --kafka localhost:9092 --topic pcap -i eth0 --debug 100
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Consumer_Mode"></a>Consumer Mode</h3>
+<p>Pycapa can be configured to consume packets from a Kafka topic and then write those packets to a <a class="externalLink" href="https://wiki.wireshark.org/Development/LibpcapFileFormat">libpcap-compliant file</a>. To read 100 packets from a kafka topic and then write those to a <a class="externalLink" href="https://wiki.wireshark.org/Development/LibpcapFileFormat">libpcap-compliant file</a>, run the following command. The file <tt>out.pcap</tt> can then be opened with a tool such as Wireshark for further validation.</p>
+
+<div class="source">
+<div class="source">
+<pre>pycapa --consumer --kafka localhost:9092 --topic pcap --n 100 > out.pcap
+</pre></div></div>
+<p>To consume packets from Kafka continuously and print debug messages every 10 packets, run the following command. </p>
+
+<div class="source">
+<div class="source">
+<pre>pycapa --consumer --kafka localhost:9092 --topic pcap --debug 10
+</pre></div></div></div></div>
+<div class="section">
+<h2><a name="Dependencies"></a>Dependencies</h2>
+
+<ul>
+
+<li><a class="externalLink" href="https://github.com/dpkp/kafka-python">kafka-python</a></li>
+
+<li><a class="externalLink" href="https://github.com/CoreSecurity/pcapy">pcapy</a></li>
+</ul></div>
+<div class="section">
+<h2><a name="Implementation"></a>Implementation</h2>
+<p>When run in Producer Mode, Pycapa embeds the raw network packet data in the Kafka message. The message key contains the timestamp indicating when the packet was captured in microseconds from the epoch. This value is in network byte order.</p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/documentation/index.md
----------------------------------------------------------------------
diff --git a/site/documentation/index.md b/site/documentation/index.md
index 2c548e9..28660c8 100644
--- a/site/documentation/index.md
+++ b/site/documentation/index.md
@@ -74,25 +74,35 @@ title: Apache Metron (Incubating) Documentation
<section class="events-section info-block darken" id="releases">
<div class="wrapper downloads">
<div class="text-center">
- <h4>Current Release</h4>
+ <h4>Current Release: 0.3.1</h4>
</div>
<div class="content-960 hover-btn text-center">
<ul>
<li>
- <a href="http://www.apache.org/dyn/closer.cgi/incubator/metron/0.3.0/apache-metron-0.3.0-incubating.tar.gz" target="new">
- apache-metron-0.3.0-incubating.tar.gz
+ <a href="http://www.apache.org/dyn/closer.cgi/incubator/metron/0.3.1/apache-metron-0.3.1-incubating.tar.gz" target="new">
+ apache-metron-0.3.1-incubating.tar.gz
</a>
- [ <a href="https://dist.apache.org/repos/dist/release/incubator/metron/0.3.0/apache-metron-0.3.0-incubating.tar.gz.asc" target="new">
+ [ <a href="https://dist.apache.org/repos/dist/release/incubator/metron/0.3.1/apache-metron-0.3.1-incubating.tar.gz.asc" target="new">
PGP
</a>]
- [ <a href="https://dist.apache.org/repos/dist/release/incubator/metron/0.3.0/apache-metron-0.3.0-incubating.tar.gz.sha" target="new">
+ [ <a href="https://dist.apache.org/repos/dist/release/incubator/metron/0.3.1/apache-metron-0.3.1-incubating.tar.gz.sha" target="new">
SHA512
</a>]
- [ <a href="https://dist.apache.org/repos/dist/release/incubator/metron/0.3.0/apache-metron-0.3.0-incubating.tar.gz.md5" target="new">
+ [ <a href="https://dist.apache.org/repos/dist/release/incubator/metron/0.3.1/apache-metron-0.3.1-incubating.tar.gz.md5" target="new">
MD5
</a>]
+ [ <a href="/current-book/index.html" target="new"> Book </a>]
</li>
</ul>
</div>
</div>
-</section>
\ No newline at end of file
+
+ <div class="wrapper downloads">
+ <div class="text-center">
+ <h4>Older Releases</h4>
+ </div>
+ <div class="content-960 hover-btn text-center">
+ If you need access to older releases they can be found in the <a href="https://archive.apache.org/dist/incubator/metron/">release archives</a>.
+ </div>
+ </div>
+</section>
[10/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-analytics/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/index.html b/site/current-book/metron-analytics/index.html
new file mode 100644
index 0000000..f5a9628
--- /dev/null
+++ b/site/current-book/metron-analytics/index.html
@@ -0,0 +1,210 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Metron Analytics</title>
+ <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../css/site.css" />
+ <link rel="stylesheet" href="../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Metron Analytics</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="icon-chevron-down"></i>Analytics</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../metron-analytics/metron-maas-service/index.html" title="Maas-service">
+ <i class="none"></i>
+ Maas-service</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-analytics/metron-profiler/index.html" title="Profiler">
+ <i class="none"></i>
+ Profiler</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-analytics/metron-profiler-client/index.html" title="Profiler-client">
+ <i class="none"></i>
+ Profiler-client</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-analytics/metron-statistics/index.html" title="Statistics">
+ <i class="icon-chevron-right"></i>
+ Statistics</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Metron Analytics</h1>
+<p><a name="Metron_Analytics"></a></p>
+<p>Metron analytics consists of:</p>
+
+<ul>
+
+<li>Model-as-a-Service (MAAS) access to Machine Learning services</li>
+
+<li>Profiler and Profiler Client</li>
+
+<li>Statistics</li>
+</ul>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-analytics/metron-maas-service/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/metron-maas-service/index.html b/site/current-book/metron-analytics/metron-maas-service/index.html
new file mode 100644
index 0000000..66716a1
--- /dev/null
+++ b/site/current-book/metron-analytics/metron-maas-service/index.html
@@ -0,0 +1,474 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Model Management Infrastructure</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Model Management Infrastructure</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-down"></i>
+ Analytics</a>
+ <ul class="nav nav-list">
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Maas-service</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-profiler/index.html" title="Profiler">
+ <i class="none"></i>
+ Profiler</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client">
+ <i class="none"></i>
+ Profiler-client</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-statistics/index.html" title="Statistics">
+ <i class="icon-chevron-right"></i>
+ Statistics</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Model Management Infrastructure</h1>
+<p><a name="Model_Management_Infrastructure"></a></p>
+<div class="section">
+<h2><a name="Introduction"></a>Introduction</h2>
+<p>One of the main features envisioned and requested is the ability to augment the threat intelligence and enrichment processes with insights derived from machine learning or statistical models. The challenges with this sort of infrastructure are</p>
+
+<ul>
+
+<li>Applying the model may be sufficiently computationally/resource intensive that we need to support scaling via load balancing, which will require service discovery and management.</li>
+
+<li>Models require out of band and frequent training to react to growing threats and new patterns that emerge.</li>
+
+<li>Models should be language/environment agnostic as much as possible. These should include small-data and big-data libraries and languages.</li>
+</ul>
+<p>To support a high throughput environment that is manageable, it is evident that </p>
+
+<ul>
+
+<li>Multiple versions of models will need to be exposed</li>
+
+<li>Deployment should happen using Yarn to manage resources</li>
+
+<li>Clients should have new model endpoints pushed to them</li>
+</ul></div>
+<div class="section">
+<h2><a name="Architecture"></a>Architecture</h2>
+<p><img src="../../images/maas_arch.png" alt="Architecture" /></p>
+<p>To support these requirements, the following components have been created:</p>
+
+<ul>
+
+<li>A Yarn application which will listen for model deployment requests and upon execution, register their endpoints in zookeeper:
+
+<ul>
+
+<li>Operation type: ADD, REMOVE, LIST</li>
+
+<li>Model Name</li>
+
+<li>Model Version</li>
+
+<li>Memory requirements (in megabytes)</li>
+
+<li>Number of instances</li>
+ </ul></li>
+
+<li>A command line deployment client which will localize the model payload onto HDFS and submit a model request</li>
+
+<li>A Java client which will interact with zookeeper and receive updates about model state changes (new deployments, removals, etc.)</li>
+
+<li>A series of Stellar functions for interacting with models deployed via the Model as a Service infrastructure.</li>
+</ul></div>
+<div class="section">
+<h2><a name="maas_service.sh"></a><tt>maas_service.sh</tt></h2>
+<p>The <tt>maas_service.sh</tt> script starts the Yarn application which will listen for requests. Right now the queue for the requests is a distributed queue stored in <a class="externalLink" href="http://curator.apache.org/curator-recipes/distributed-queue.html">zookeeper</a> for convenience.</p>
+
+<div class="source">
+<div class="source">
+<pre>./maas_service.sh
+usage: MaaSClient
+ -c,--create Flag to indicate whether to create
+ the domain specified with -domain.
+ -d,--domain <arg> ID of the timeline domain where the
+ timeline entities will be put
+ -e,--shell_env <arg> Environment for shell script.
+ Specified as env_key=env_val pairs
+ -h,--help This screen
+ -j,--jar <arg> Jar file containing the application
+ master
+ -l4j,--log_properties <arg> log4j.properties file
+ -ma,--modify_acls <arg> Users and groups that allowed to
+ modify the timeline entities in the
+ given domain
+ -mc,--master_vcores <arg> Amount of virtual cores to be
+ requested to run the application
+ master
+ -mm,--master_memory <arg> Amount of memory in MB to be
+ requested to run the application
+ master
+ -nle,--node_label_expression <arg> Node label expression to determine
+ the nodes where all the containers
+ of this application will be
+ allocated, "" means containers can
+ be allocated anywhere, if you don't
+ specify the option, default
+ node_label_expression of queue will
+ be used.
+ -q,--queue <arg> RM Queue in which this application
+ is to be submitted
+ -t,--timeout <arg> Application timeout in milliseconds
+ -va,--view_acls <arg> Users and groups that allowed to
+ view the timeline entities in the
+ given domain
+ -zq,--zk_quorum <arg> Zookeeper Quorum
+ -zr,--zk_root <arg> Zookeeper Root
+</pre></div></div></div>
+<div class="section">
+<h2><a name="maas_deploy.sh"></a><tt>maas_deploy.sh</tt></h2>
+<p>The <tt>maas_deploy.sh</tt> script allows users to deploy models and their collateral from their local disk to the cluster. It is assumed that the </p>
+
+<ul>
+
+<li>Collateral has exactly one <tt>.sh</tt> script capable of starting the endpoint</li>
+
+<li>The model service executable will expose itself as a URL endpoint (e.g. as a REST interface, but not necessarily)</li>
+
+<li>The model service executable will write out to local disk a JSON blob indicating the endpoint (see <a class="externalLink" href="https://gist.github.com/cestella/cba10aff0f970078a4c2c8cade3a4d1a#file-dga-py-L21">here</a> for an example mock service using Python and Flask).</li>
+</ul>
+
+<div class="source">
+<div class="source">
+<pre>./maas_deploy.sh
+usage: ModelSubmission
+ -h,--help This screen
+ -hmp,--hdfs_model_path <arg> Model Path (HDFS)
+ -lmp,--local_model_path <arg> Model Path (local)
+ -m,--memory <arg> Memory for container
+ -mo,--mode <arg> ADD, LIST or REMOVE
+ -n,--name <arg> Model Name
+ -ni,--num_instances <arg> Number of model instances
+ -v,--version <arg> Model version
+ -zq,--zk_quorum <arg> Zookeeper Quorum
+ -zr,--zk_root <arg> Zookeeper Root
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Stellar_Integration"></a>Stellar Integration</h2>
+<p>Two Stellar functions have been added to provide the ability to call out to models deployed via Model as a Service. One aimed at recovering a load balanced endpoint of a deployed model given the name and, optionally, the version. The second is aimed at calling that endpoint assuming that it is exposed as a REST endpoint.</p>
+
+<ul>
+
+<li><tt>MAAS_MODEL_APPLY(endpoint, function?, model_args)</tt> : Returns the output of a model deployed via model which is deployed at endpoint. <tt>endpoint</tt> is a map containing <tt>name</tt>, <tt>version</tt>, <tt>url</tt> for the REST endpoint, <tt>function</tt> is the endpoint path and is optional, and <tt>model_args</tt> is a dictionary of arguments for the model (these become request params).</li>
+
+<li><tt>MAAS_GET_ENDPOINT(model_name, model_version?)</tt> : Inspects zookeeper and returns a map containing the <tt>name</tt>, <tt>version</tt> and <tt>url</tt> for the model referred to by <tt>model_name</tt> and <tt>model_version</tt>. If <tt>model_version</tt> is not specified, the most current model associated with <tt>model_name</tt> is returned. In the instance where more than one model is deployed, a random one is selected with uniform probability.</li>
+</ul>
+<p><a name="Example"></a></p>
+<h1>Example</h1>
+<p>Let’s augment the <tt>squid</tt> proxy sensor to use a model that will determine if the destination host is a domain generating algorithm. For the purposes of demonstration, this algorithm is super simple and is implemented using Python with a REST interface exposed via the Flask python library.</p></div>
+<div class="section">
+<h2><a name="Install_Prerequisites_and_Mock_DGA_Service"></a>Install Prerequisites and Mock DGA Service</h2>
+<p>Now let’s install some prerequisites:</p>
+
+<ul>
+
+<li>Flask via <tt>yum install python-flask</tt></li>
+
+<li>Jinja2 via <tt>yum install python-jinja2</tt></li>
+
+<li>Squid client via <tt>yum install squid</tt></li>
+
+<li>ES Head plugin via <tt>/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head</tt></li>
+</ul>
+<p>Start Squid via <tt>service squid start</tt></p>
+<p>Now that we have flask and jinja, we can create a mock DGA service to deploy with MaaS:</p>
+
+<ul>
+
+<li>Download the files in <a class="externalLink" href="https://gist.github.com/cestella/cba10aff0f970078a4c2c8cade3a4d1a">this</a> gist into the <tt>/root/mock_dga</tt> directory</li>
+
+<li>Make <tt>rest.sh</tt> executable via <tt>chmod +x /root/mock_dga/rest.sh</tt></li>
+</ul>
+<p>This service will treat <tt>yahoo.com</tt> and <tt>amazon.com</tt> as legit and everything else as malicious. The contract is that the REST service exposes an endpoint <tt>/apply</tt> and returns back JSON maps with a single key <tt>is_malicious</tt> which can be <tt>malicious</tt> or <tt>legit</tt>.</p></div>
+<div class="section">
+<h2><a name="Deploy_Mock_DGA_Service_via_MaaS"></a>Deploy Mock DGA Service via MaaS</h2>
+<p>Now let’s start MaaS and deploy the Mock DGA Service:</p>
+
+<ul>
+
+<li>Start MaaS via <tt>$METRON_HOME/bin/maas_service.sh -zq node1:2181</tt></li>
+
+<li>Start one instance of the mock DGA model with 512M of memory via <tt>$METRON_HOME/bin/maas_deploy.sh -zq node1:2181 -lmp /root/mock_dga -hmp /user/root/models -mo ADD -m 512 -n dga -v 1.0 -ni 1</tt></li>
+
+<li>As a sanity check:
+
+<ul>
+
+<li>Ensure that the model is running via <tt>$METRON_HOME/bin/maas_deploy.sh -zq node1:2181 -mo LIST</tt>. You should see <tt>Model dga @ 1.0</tt> be displayed and under that a url such as (but not exactly) <tt>http://node1:36161</tt></li>
+
+<li>Try to hit the model via curl: <tt>curl 'http://localhost:36161/apply?host=caseystella.com'</tt> and ensure that it returns a JSON map indicating the domain is malicious.</li>
+ </ul></li>
+</ul></div>
+<div class="section">
+<h2><a name="Adjust_Configurations_for_Squid_to_Call_Model"></a>Adjust Configurations for Squid to Call Model</h2>
+<p>Now that we have a deployed model, let’s adjust the configurations for the Squid topology to annotate the messages with the output of the model.</p>
+
+<ul>
+
+<li>Edit the squid parser configuration at <tt>$METRON_HOME/config/zookeeper/parsers/squid.json</tt> in your favorite text editor and add a new FieldTransformation to indicate a threat alert based on the model (note the addition of <tt>is_malicious</tt> and <tt>is_alert</tt>):</li>
+</ul>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "parserClassName": "org.apache.metron.parsers.GrokParser",
+ "sensorTopic": "squid",
+ "parserConfig": {
+ "grokPath": "/patterns/squid",
+ "patternLabel": "SQUID_DELIMITED",
+ "timestampField": "timestamp"
+ },
+ "fieldTransformations" : [
+ {
+ "transformation" : "STELLAR"
+ ,"output" : [ "full_hostname", "domain_without_subdomains", "is_malicious", "is_alert" ]
+ ,"config" : {
+ "full_hostname" : "URL_TO_HOST(url)"
+ ,"domain_without_subdomains" : "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)"
+ ,"is_malicious" : "MAP_GET('is_malicious', MAAS_MODEL_APPLY(MAAS_GET_ENDPOINT('dga'), {'host' : domain_without_subdomains}))"
+ ,"is_alert" : "if is_malicious == 'malicious' then 'true' else null"
+ }
+ }
+ ]
+}
+</pre></div></div>
+
+<ul>
+
+<li>Edit the squid enrichment configuration at <tt>$METRON_HOME/config/zookeeper/enrichments/squid.json</tt> (this file will not exist, so create a new one) to make the threat triage adjust the level of risk based on the model output:</li>
+</ul>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "index": "squid",
+ "batchSize": 1,
+ "enrichment" : {
+ "fieldMap": {}
+ },
+ "threatIntel" : {
+ "fieldMap":{},
+ "triageConfig" : {
+ "riskLevelRules" : [
+ {
+ "rule" : "is_malicious == 'malicious'",
+ "score" : 100
+ }
+ ],
+ "aggregator" : "MAX"
+ }
+ }
+}
+</pre></div></div>
+
+<ul>
+
+<li>Upload new configs via <tt>$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -i $METRON_HOME/config/zookeeper -z node1:2181</tt></li>
+
+<li>Make the Squid topic in kafka via <tt>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper node1:2181 --create --topic squid --partitions 1 --replication-factor 1</tt></li>
+</ul></div>
+<div class="section">
+<h2><a name="Start_Topologies_and_Send_Data"></a>Start Topologies and Send Data</h2>
+<p>Now we need to start the topologies and send some data:</p>
+
+<ul>
+
+<li>Start the squid topology via <tt>$METRON_HOME/bin/start_parser_topology.sh -k node1:6667 -z node1:2181 -s squid</tt></li>
+
+<li>Generate some data via the squid client:
+
+<ul>
+
+<li>Generate a legit example: <tt>squidclient http://yahoo.com</tt></li>
+
+<li>Generate a malicious example: <tt>squidclient http://cnn.com</tt></li>
+ </ul></li>
+
+<li>Send the data to kafka via <tt>cat /var/log/squid/access.log | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic squid</tt></li>
+
+<li>Browse the data in elasticsearch via the ES Head plugin @ <a class="externalLink" href="http://node1:9200/_plugin/head/">http://node1:9200/_plugin/head/</a> and verify that in the squid index you have two documents
+
+<ul>
+
+<li>One from <tt>yahoo.com</tt> which does not have <tt>is_alert</tt> set and does have <tt>is_malicious</tt> set to <tt>legit</tt></li>
+
+<li>One from <tt>cnn.com</tt> which does have <tt>is_alert</tt> set to <tt>true</tt>, <tt>is_malicious</tt> set to <tt>malicious</tt> and <tt>threat:triage:level</tt> set to 100</li>
+ </ul></li>
+</ul></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-analytics/metron-profiler-client/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/metron-profiler-client/index.html b/site/current-book/metron-analytics/metron-profiler-client/index.html
new file mode 100644
index 0000000..54dd5d6
--- /dev/null
+++ b/site/current-book/metron-analytics/metron-profiler-client/index.html
@@ -0,0 +1,427 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Metron Profiler Client</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Metron Profiler Client</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-down"></i>
+ Analytics</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service">
+ <i class="none"></i>
+ Maas-service</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-profiler/index.html" title="Profiler">
+ <i class="none"></i>
+ Profiler</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Profiler-client</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-statistics/index.html" title="Statistics">
+ <i class="icon-chevron-right"></i>
+ Statistics</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Metron Profiler Client</h1>
+<p><a name="Metron_Profiler_Client"></a></p>
+<p>This project provides a client API for accessing the profiles generated by the <a href="../metron-profiler/index.html">Metron Profiler</a>. This includes both a Java API and Stellar API for accessing the profile data. The primary use case is to extract profile data for use during model scoring.</p>
+<div class="section">
+<h2><a name="Stellar_Client_API"></a>Stellar Client API</h2>
+<p>The following are usage examples that show how the Stellar API can be used to read profiles generated by the <a href="../metron-profiler/index.html">Metron Profiler</a>. This API would be used in conjunction with other Stellar functions like <a href="../../metron-platform/metron-common/index.html#MAAS_MODEL_APPLY"><tt>MAAS_MODEL_APPLY</tt></a> to perform model scoring on streaming data.</p>
+<p>These examples assume a profile has been defined called ‘snort-alerts’ that tracks the number of Snort alerts associated with an IP address over time. The profile definition might look similar to the following.</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "profiles": [
+ {
+ "profile": "snort-alerts",
+ "foreach": "ip_src_addr",
+ "onlyif": "source.type == 'snort'",
+ "update": { "s": "STATS_ADD(s, 1)" },
+ "result": "STATS_MEAN(s)"
+ }
+ ]
+}
+</pre></div></div>
+<p>During model scoring the entity being scored, in this case a particular IP address, will be known. The following examples shows how this profile data might be retrieved.</p>
+<p>The Stellar client consists of the <tt>PROFILE_GET</tt> command, which takes the following arguments:</p>
+
+<div class="source">
+<div class="source">
+<pre>REQUIRED:
+ profile - The name of the profile
+ entity - The name of the entity
+ periods - The list of profile periods to grab. These are ProfilePeriod objects.
+OPTIONAL:
+ groups_list - Optional, must correspond to the 'groupBy' list used in profile creation - List (in square brackets) of
+ groupBy values used to filter the profile. Default is the empty list, meaning groupBy was not used when
+ creating the profile.
+ config_overrides - Optional - Map (in curly braces) of name:value pairs, each overriding the global config parameter
+ of the same name. Default is the empty Map, meaning no overrides.
+</pre></div></div>
+<p>There is an older calling format where <tt>groups_list</tt> is specified as a sequence of group names, “varargs” style, instead of a List object. This format is still supported for backward compatibility, but it is deprecated, and it is disallowed if the optional <tt>config_overrides</tt> argument is used.</p>
+<p>The <tt>periods</tt> field is (likely) the output of another Stellar function which defines the times to include.</p>
+<p><tt>PROFILE_FIXED</tt>: The profiler periods associated with a fixed lookback starting from now. These are ProfilePeriod objects.</p>
+
+<div class="source">
+<div class="source">
+<pre>REQUIRED:
+ durationAgo - How long ago should values be retrieved from?
+ units - The units of 'durationAgo'.
+OPTIONAL:
+ config_overrides - Optional - Map (in curly braces) of name:value pairs, each overriding the global config parameter
+ of the same name. Default is the empty Map, meaning no overrides.
+
+e.g. To retrieve all the profiles for the last 5 hours. PROFILE_GET('profile', 'entity', PROFILE_FIXED(5, 'HOURS'))
+</pre></div></div>
+<div class="section">
+<h3><a name="Groups_list_argument"></a>Groups_list argument</h3>
+<p>The <tt>groups_list</tt> argument in the client must exactly correspond to the <a href="../metron-profiler/index.html#groupBy"><tt>groupBy</tt></a> configuration in the profile definition. If <tt>groupBy</tt> was not used in the profile, <tt>groups_list</tt> must be empty in the client. If <tt>groupBy</tt> was used in the profile, then the client <tt>groups_list</tt> is <b>not</b> optional; it must be the same length as the <tt>groupBy</tt> list, and specify exactly one selected group value for each <tt>groupBy</tt> criterion, in the same order. For example:</p>
+
+<div class="source">
+<div class="source">
+<pre>If in Profile, the groupBy criteria are: [ “DAY_OF_WEEK()”, “URL_TO_PORT()” ]
+Then in PROFILE_GET, an allowed groups value would be: [ “3”, “8080” ]
+which will select only records from Tuesdays with port number 8080.
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Configuration_and_the_config_overrides_argument"></a>Configuration and the config_overrides argument</h3>
+<p>By default, the Profiler creates profiles with a period duration of 15 minutes. This means that data is accumulated, summarized and flushed every 15 minutes. The Client API must also have knowledge of this duration to correctly retrieve the profile data. If the Client is expecting 15 minute periods, it will not be able to read data generated by a Profiler that was configured for 1 hour periods, and will return zero results. </p>
+<p>Similarly, all six Client configuration parameters listed in the table below must match the Profiler configuration parameter settings from the time the profile was created. The period duration and other configuration parameters from the Profiler topology are stored in local filesystem at <tt>$METRON_HOME/config/profiler.properties</tt>. The Stellar Client API can be configured correspondingly by setting the following properties in Metron’s global configuration, on local filesystem at <tt>$METRON_HOME/config/zookeeper/global.json</tt>, then uploaded to Zookeeper (at <tt>/metron/topology/global</tt>) by using <tt>zk_load_configs.sh</tt>: </p>
+
+<div class="source">
+<div class="source">
+<pre> $ cd $METRON_HOME
+ $ bin/zk_load_configs.sh -m PUSH -i config/zookeeper/ -z node1:2181
+</pre></div></div>
+<p>Any of these six Client configuration parameters may be overridden at run time using the <tt>config_overrides</tt> Map argument in PROFILE_GET. The primary use case is when historical profiles have been created with a different Profiler configuration than is currently configured, and the analyst needing to access them does not want to change the global Client configuration so as not to disrupt the work of other analysts working with current profiles.</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Key </th>
+
+<th>Description </th>
+
+<th>Required </th>
+
+<th>Default </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>profiler.client.period.duration </td>
+
+<td>The duration of each profile period. This value should be defined along with <tt>profiler.client.period.duration.units</tt>. </td>
+
+<td>Optional </td>
+
+<td>15 </td>
+ </tr>
+
+<tr class="a">
+
+<td>profiler.client.period.duration.units </td>
+
+<td>The units used to specify the profile period duration. This value should be defined along with <tt>profiler.client.period.duration</tt>. </td>
+
+<td>Optional </td>
+
+<td>MINUTES </td>
+ </tr>
+
+<tr class="b">
+
+<td>profiler.client.hbase.table </td>
+
+<td>The name of the HBase table used to store profile data. </td>
+
+<td>Optional </td>
+
+<td>profiler </td>
+ </tr>
+
+<tr class="a">
+
+<td>profiler.client.hbase.column.family </td>
+
+<td>The name of the HBase column family used to store profile data. </td>
+
+<td>Optional </td>
+
+<td>P </td>
+ </tr>
+
+<tr class="b">
+
+<td>profiler.client.salt.divisor </td>
+
+<td>The salt divisor used to store profile data. </td>
+
+<td>Optional </td>
+
+<td>1000 </td>
+ </tr>
+
+<tr class="a">
+
+<td>hbase.provider.impl </td>
+
+<td>The name of the HBaseTableProvider implementation class. </td>
+
+<td>Optional </td>
+
+<td> </td>
+ </tr>
+ </tbody>
+</table></div>
+<div class="section">
+<h3><a name="Errors"></a>Errors</h3>
+<p>The most common result of incorrect PROFILE_GET arguments or Client configuration parameters is an empty result set, rather than an error. The Client cannot effectively validate the arguments, because the Profiler configuration parameters may be changed and the profile itself does not store them. The person doing the querying must carry forward the knowledge of the Profiler configuration parameters from the time of profile creation, and use corresponding PROFILE_GET arguments and Client configuration parameters when querying the data.</p></div>
+<div class="section">
+<h3><a name="Examples"></a>Examples</h3>
+<p>Retrieve all values of ‘snort-alerts’ from ‘10.0.0.1’ over the past 4 hours.</p>
+
+<div class="source">
+<div class="source">
+<pre>PROFILE_GET('snort-alerts', '10.0.0.1', 4, 'HOURS')
+</pre></div></div>
+<p>Retrieve all values of ‘snort-alerts’ from ‘10.0.0.1’ over the past 2 days.</p>
+
+<div class="source">
+<div class="source">
+<pre>PROFILE_GET('snort-alerts', '10.0.0.1', 2, 'DAYS')
+</pre></div></div>
+<p>If the profile had been defined to group the data by weekday versus weekend, then the following example would apply:</p>
+<p>Retrieve all values of ‘snort-alerts’ from ‘10.0.0.1’ that occurred on ‘weekdays’ over the past month.</p>
+
+<div class="source">
+<div class="source">
+<pre>PROFILE_GET('snort-alerts', '10.0.0.1', 1, 'MONTHS', ['weekdays'] )
+</pre></div></div>
+<p>The client may need to use a configuration different from the current Client configuration settings. For example, perhaps you are on a cluster shared with other analysts, and need to access a profile that was constructed 2 months ago using different period duration, while they are accessing more recent profiles constructed with the currently configured period duration. For this situation, you may use the <tt>config_overrides</tt> argument:</p>
+<p>Retrieve all values of ‘snort-alerts’ from ‘10.0.0.1’ over the past 2 days, with no <tt>groupBy</tt>, and overriding the usual global client configuration parameters for window duration.</p>
+
+<div class="source">
+<div class="source">
+<pre>PROFILE_GET('profile1', 'entity1', 2, 'DAYS', [], {'profiler.client.period.duration' : '2', 'profiler.client.period.duration.units' : 'MINUTES'})
+</pre></div></div>
+<p>Retrieve all values of ‘snort-alerts’ from ‘10.0.0.1’ that occurred on ‘weekdays’ over the past month, overriding the usual global client configuration parameters for window duration.</p>
+
+<div class="source">
+<div class="source">
+<pre>PROFILE_GET('profile1', 'entity1', 1, 'MONTHS', ['weekdays'], {'profiler.client.period.duration' : '2', 'profiler.client.period.duration.units' : 'MINUTES'})
+</pre></div></div></div></div>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>These instructions step through the process of using the Stellar Client API on a live cluster. These instructions assume that the ‘Getting Started’ instructions included with the <a href="../metron-profiler/index.html">Metron Profiler</a> have been followed. This will create a Profile called ‘test’ whose data will be retrieved with the Stellar Client API.</p>
+<p>To validate that everything is working, login to the server hosting Metron. We will use the Stellar Shell to replicate the execution environment of Stellar running in a Storm topology, like Metron’s Parser or Enrichment topology. Replace ‘node1:2181’ with the URL to a Zookeeper Broker. </p>
+
+<div class="source">
+<div class="source">
+<pre>[root@node1 0.3.1]# bin/stellar -z node1:2181
+Stellar, Go!
+Please note that functions are loading lazily in the background and will be unavailable until loaded fully.
+{es.clustername=metron, es.ip=node1, es.port=9300, es.date.format=yyyy.MM.dd.HH}
+
+[Stellar]>>> ?PROFILE_GET
+Functions loaded, you may refer to functions now...
+PROFILE_GET
+Description: Retrieves a series of values from a stored profile.
+
+Arguments:
+ profile - The name of the profile.
+ entity - The name of the entity.
+ durationAgo - How long ago should values be retrieved from?
+ units - The units of 'durationAgo'.
+ groups_list - Optional, must correspond to the 'groupBy' list used in profile creation - List (in square brackets) of
+ groupBy values used to filter the profile. Default is the empty list, meaning groupBy was not used when
+ creating the profile.
+ config_overrides - Optional - Map (in curly braces) of name:value pairs, each overriding the global config parameter
+ of the same name. Default is the empty Map, meaning no overrides.
+
+Returns: The selected profile measurements.
+
+[Stellar]>>> PROFILE_GET('test','192.168.138.158', 1, 'HOURS')
+[12078.0, 8921.0, 12131.0]
+</pre></div></div>
+<p>The client API call above has retrieved the past hour of the ‘test’ profile for the entity ‘192.168.138.158’.</p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-analytics/metron-profiler/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/metron-profiler/index.html b/site/current-book/metron-analytics/metron-profiler/index.html
new file mode 100644
index 0000000..adb7ffb
--- /dev/null
+++ b/site/current-book/metron-analytics/metron-profiler/index.html
@@ -0,0 +1,779 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Metron Profiler</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Metron Profiler</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-down"></i>
+ Analytics</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service">
+ <i class="none"></i>
+ Maas-service</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Profiler</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client">
+ <i class="none"></i>
+ Profiler-client</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-statistics/index.html" title="Statistics">
+ <i class="icon-chevron-right"></i>
+ Statistics</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Metron Profiler</h1>
+<p><a name="Metron_Profiler"></a></p>
+<p>The Profiler is a feature extraction mechanism that can generate a profile describing the behavior of an entity. An entity might be a server, user, subnet or application. Once a profile has been generated defining what normal behavior looks-like, models can be built that identify anomalous behavior. </p>
+<p>This is achieved by summarizing the streaming telemetry data consumed by Metron over sliding windows. A summary statistic is applied to the data received within a given window. Collecting this summary across many windows results in a time series that is useful for analysis.</p>
+<p>Any field contained within a message can be used to generate a profile. A profile can even be produced by combining fields that originate in different data sources. A user has considerable power to transform the data used in a profile by leveraging the Stellar language. A user only need configure the desired profiles and ensure that the Profiler topology is running.</p>
+
+<ul>
+
+<li><a href="#Getting_Started">Getting Started</a></li>
+
+<li><a href="#Creating_Profiles">Creating Profiles</a></li>
+
+<li><a href="#Configuring_the_Profiler">Configuring the Profiler</a></li>
+
+<li><a href="#Examples">Examples</a></li>
+
+<li><a href="#Implementation">Implementation</a></li>
+</ul>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>This section will describe the steps required to get your first profile running.</p>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Stand-up a Metron environment. For this example, we will use the ‘Quick Dev’ environment. Follow the instructions included with <a href="../../metron-deployment/vagrant/quick-dev-platform/index.html">Quick Dev</a> or build your own.</p></li>
+
+<li>
+<p>Create a table within HBase that will store the profile data. The table name and column family must match the <a href="#configuring-the-profiler">Profiler’s configuration</a>.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ /usr/hdp/current/hbase-client/bin/hbase shell
+hbase(main):001:0> create 'profiler', 'P'
+</pre></div></div></li>
+
+<li>
+<p>Edit the configuration file located at <tt>$METRON_HOME/config/profiler.properties</tt>. Change the kafka.zk and kafka.broker values from “node1” to the appropriate host name. Keep the same port numbers:</p>
+
+<div class="source">
+<div class="source">
+<pre>kafka.zk=node1:2181
+kafka.broker=node1:6667
+</pre></div></div></li>
+
+<li>
+<p>Define the profile in a file located at <tt>$METRON_HOME/config/zookeeper/profiler.json</tt>. The following example JSON will create a profile that simply counts the number of messages per <tt>ip_src_addr</tt>, during each sampling interval.</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "profiles": [
+ {
+ "profile": "test",
+ "foreach": "ip_src_addr",
+ "init": { "count": "0" },
+ "update": { "count": "count + 1" },
+ "result": "count"
+ }
+ ]
+}
+</pre></div></div></li>
+
+<li>
+<p>Upload the profile definition to Zookeeper. (As always, change “node1” to the actual hostname.)</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_HOME
+$ bin/zk_load_configs.sh -m PUSH -i config/zookeeper/ -z node1:2181
+</pre></div></div></li>
+
+<li>
+<p>Start the Profiler topology.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ bin/start_profiler_topology.sh
+</pre></div></div></li>
+
+<li>
+<p>Ensure that test messages are being sent to the Profiler’s input topic in Kafka. The Profiler will consume messages from the <tt>inputTopic</tt> defined in the <a href="#configuring-the-profiler">Profiler’s configuration</a>.</p></li>
+
+<li>
+<p>Check the HBase table to validate that the Profiler is writing the profile. Remember that the Profiler is flushing the profile every 15 minutes. You will need to wait at least this long to start seeing profile data in HBase.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ /usr/hdp/current/hbase-client/bin/hbase shell
+hbase(main):001:0> count 'profiler'
+</pre></div></div></li>
+
+<li>
+<p>Use the Profiler Client to read the profile data. The below example <tt>PROFILE_GET</tt> command will read data written by the sample profile given above, if 10.0.0.1 is one of the input values for <tt>ip_src_addr</tt>. More information on configuring and using the client can be found <a href="../metron-profiler-client/index.html">here</a>. It is assumed that the <tt>PROFILE_GET</tt> client is correctly configured before using it.</p>
+<p>``` $ bin/stellar -z node1:2181</p>
+<p>[Stellar]>>> PROFILE_GET( “test”, “10.0.0.1”, PROFILE_FIXED(30, “MINUTES”)) [451, 448] ```</p></li>
+</ol></div>
+<div class="section">
+<h2><a name="Creating_Profiles"></a>Creating Profiles</h2>
+<p>The Profiler specification requires a JSON-formatted set of elements, many of which can contain Stellar code. The specification contains the following elements. (For the impatient, skip ahead to the <a href="#Examples">Examples</a>.) The specification for the Profiler topology is stored in Zookeeper at <tt>/metron/topology/profiler</tt>. These properties also exist in the local filesystem at <tt>$METRON_HOME/config/zookeeper/profiler.json</tt>. The values can be changed on disk and then uploaded to Zookeeper using <tt>$METRON_HOME/bin/zk_load_configs.sh</tt>.</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Name </th>
+
+<th>Required </th>
+
+<th>Description </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td><a href="#profile">profile</a> </td>
+
+<td>Required </td>
+
+<td>Unique name identifying the profile. </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#foreach">foreach</a> </td>
+
+<td>Required </td>
+
+<td>A separate profile is maintained “for each” of these. </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#onlyif">onlyif</a> </td>
+
+<td>Optional </td>
+
+<td>Boolean expression that determines if a message should be applied to the profile. </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#groupBy">groupBy</a> </td>
+
+<td>Optional </td>
+
+<td>One or more Stellar expressions used to group the profile measurements when persisted. </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#init">init</a> </td>
+
+<td>Optional </td>
+
+<td>One or more expressions executed at the start of a window period. </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#update">update</a> </td>
+
+<td>Required </td>
+
+<td>One or more expressions executed when a message is applied to the profile. </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#result">result</a> </td>
+
+<td>Required </td>
+
+<td>A Stellar expression that is executed when the window period expires. </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#expires">expires</a> </td>
+
+<td>Optional </td>
+
+<td>Profile data is purged after this period of time, specified in milliseconds. </td>
+ </tr>
+ </tbody>
+</table>
+<div class="section">
+<h3><a name="profile"></a><tt>profile</tt></h3>
+<p><i>Required</i></p>
+<p>A unique name identifying the profile. The field is treated as a string. </p></div>
+<div class="section">
+<h3><a name="foreach"></a><tt>foreach</tt></h3>
+<p><i>Required</i></p>
+<p>A separate profile is maintained ‘for each’ of these. This is effectively the entity that the profile is describing. The field is expected to contain a Stellar expression whose result is the entity name. </p>
+<p>For example, if <tt>ip_src_addr</tt> then a separate profile would be maintained for each unique IP source address in the data; 10.0.0.1, 10.0.0.2, etc.</p></div>
+<div class="section">
+<h3><a name="onlyif"></a><tt>onlyif</tt></h3>
+<p><i>Optional</i></p>
+<p>An expression that determines if a message should be applied to the profile. A Stellar expression that returns a Boolean is expected. A message is only applied to a profile if this expression is true. This allows a profile to filter the messages that get applied to it. </p></div>
+<div class="section">
+<h3><a name="groupBy"></a><tt>groupBy</tt></h3>
+<p><i>Optional</i></p>
+<p>One or more Stellar expressions used to group the profile measurements when persisted. This is intended to sort the Profile data to allow for a contiguous scan when accessing subsets of the data. </p>
+<p>The ‘groupBy’ expressions can refer to any field within a <tt>org.apache.metron.profiler.ProfileMeasurement</tt>. A common use case would be grouping by day of week. This allows a contiguous scan to access all profile data for Mondays only. Using the following definition would achieve this. </p>
+
+<div class="source">
+<div class="source">
+<pre>"groupBy": [ "DAY_OF_WEEK()" ]
+</pre></div></div></div>
+<div class="section">
+<h3><a name="init"></a><tt>init</tt></h3>
+<p><i>Optional</i></p>
+<p>One or more expressions executed at the start of a window period. A map is expected where the key is the variable name and the value is a Stellar expression. The map can contain zero or more variable:expression pairs. At the start of each window period, each expression is executed once and stored in the given variable. Note that constant init values such as “0” must be in quotes regardless of their type, as the init value must be a string to be executed by Stellar.</p>
+
+<div class="source">
+<div class="source">
+<pre>"init": {
+ "var1": "0",
+ "var2": "1"
+}
+</pre></div></div></div>
+<div class="section">
+<h3><a name="update"></a><tt>update</tt></h3>
+<p><i>Required</i></p>
+<p>One or more expressions executed when a message is applied to the profile. A map is expected where the key is the variable name and the value is a Stellar expression. The map can include 0 or more variables/expressions. When each message is applied to the profile, the expression is executed and stored in a variable with the given name.</p>
+
+<div class="source">
+<div class="source">
+<pre>"update": {
+ "var1": "var1 + 1",
+ "var2": "var2 + 1"
+}
+</pre></div></div></div>
+<div class="section">
+<h3><a name="result"></a><tt>result</tt></h3>
+<p><i>Required</i></p>
+<p>A Stellar expression that is executed when the window period expires. The expression is expected to summarize the messages that were applied to the profile over the window period, using the state accumulated by the updates. The result will typically be a single numeric value, but it may be any serializable object, as shown in Example 4 below. </p></div>
+<div class="section">
+<h3><a name="expires"></a><tt>expires</tt></h3>
+<p><i>Optional</i></p>
+<p>A numeric value that defines how many days the profile data is retained. After this time, the data expires and is no longer accessible. If no value is defined, the data does not expire.</p></div></div>
+<div class="section">
+<h2><a name="Configuring_the_Profiler"></a>Configuring the Profiler</h2>
+<p>The Profiler runs as an independent Storm topology. The configuration for the Profiler topology is stored in local filesystem at <tt>$METRON_HOME/config/profiler.properties</tt>. The values can be changed on disk and then the Profiler topology must be restarted.</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Setting </th>
+
+<th>Description </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>profiler.workers </td>
+
+<td>The number of worker processes to create for the topology. </td>
+ </tr>
+
+<tr class="a">
+
+<td>profiler.executors </td>
+
+<td>The number of executors to spawn per component. </td>
+ </tr>
+
+<tr class="b">
+
+<td>profiler.input.topic </td>
+
+<td>The name of the Kafka topic from which to consume data. </td>
+ </tr>
+
+<tr class="a">
+
+<td>profiler.period.duration </td>
+
+<td>The duration of each profile period. This value should be defined along with <tt>profiler.period.duration.units</tt>. </td>
+ </tr>
+
+<tr class="b">
+
+<td>profiler.period.duration.units </td>
+
+<td>The units used to specify the <tt>profiler.period.duration</tt>. </td>
+ </tr>
+
+<tr class="a">
+
+<td>profiler.ttl </td>
+
+<td>If a message has not been applied to a Profile in this period of time, the Profile will be forgotten and its resources will be cleaned up. This value should be defined along with <tt>profiler.ttl.units</tt>. </td>
+ </tr>
+
+<tr class="b">
+
+<td>profiler.ttl.units </td>
+
+<td>The units used to specify the <tt>profiler.ttl</tt>. </td>
+ </tr>
+
+<tr class="a">
+
+<td>profiler.hbase.salt.divisor </td>
+
+<td>A salt is prepended to the row key to help prevent hotspotting. This constant is used to generate the salt. Ideally, this constant should be roughly equal to the number of nodes in the Hbase cluster. </td>
+ </tr>
+
+<tr class="b">
+
+<td>profiler.hbase.table </td>
+
+<td>The name of the HBase table that profiles are written to. </td>
+ </tr>
+
+<tr class="a">
+
+<td>profiler.hbase.column.family </td>
+
+<td>The column family used to store profiles. </td>
+ </tr>
+
+<tr class="b">
+
+<td>profiler.hbase.batch </td>
+
+<td>The number of puts that are written in a single batch. </td>
+ </tr>
+
+<tr class="a">
+
+<td>profiler.hbase.flush.interval.seconds </td>
+
+<td>The maximum number of seconds between batch writes to HBase. </td>
+ </tr>
+ </tbody>
+</table>
+<p>After altering the configuration, start the Profiler.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ $METRON_HOME/start_profiler_topology.sh
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Examples"></a>Examples</h2>
+<p>The following examples are intended to highlight the functionality provided by the Profiler. Each shows the configuration that would be required to generate the profile. </p>
+<p>These examples assume a fictitious input message stream that looks something like the following.</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "ip_src_addr": "10.0.0.1",
+ "protocol": "HTTPS",
+ "length": "10",
+ "bytes_in": "234"
+},
+{
+ "ip_src_addr": "10.0.0.2",
+ "protocol": "HTTP",
+ "length": "20",
+ "bytes_in": "390"
+},
+{
+ "ip_src_addr": "10.0.0.3",
+ "protocol": "DNS",
+ "length": "30",
+ "bytes_in": "560"
+}
+</pre></div></div>
+<div class="section">
+<h3><a name="Example_1"></a>Example 1</h3>
+<p>The total number of bytes of HTTP data for each host. The following configuration would be used to generate this profile.</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "profiles": [
+ {
+ "profile": "example1",
+ "foreach": "ip_src_addr",
+ "onlyif": "protocol == 'HTTP'",
+ "init": {
+ "total_bytes": 0.0
+ },
+ "update": {
+ "total_bytes": "total_bytes + bytes_in"
+ },
+ "result": "total_bytes",
+ "expires": 30
+ }
+ ]
+}
+</pre></div></div>
+<p>This creates a profile…</p>
+
+<ul>
+
+<li>Named ‘example1’</li>
+
+<li>That for each IP source address</li>
+
+<li>Only if the ‘protocol’ field equals ‘HTTP’</li>
+
+<li>Initializes a counter ‘total_bytes’ to zero</li>
+
+<li>Adds to ‘total_bytes’ the value of the message’s ‘bytes_in’ field</li>
+
+<li>Returns ‘total_bytes’ as the result</li>
+
+<li>The profile data will expire in 30 days</li>
+</ul></div>
+<div class="section">
+<h3><a name="Example_2"></a>Example 2</h3>
+<p>The ratio of DNS traffic to HTTP traffic for each host. The following configuration would be used to generate this profile.</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "profiles": [
+ {
+ "profile": "example2",
+ "foreach": "ip_src_addr",
+ "onlyif": "protocol == 'DNS' or protocol == 'HTTP'",
+ "init": {
+ "num_dns": 1.0,
+ "num_http": 1.0
+ },
+ "update": {
+ "num_dns": "num_dns + (if protocol == 'DNS' then 1 else 0)",
+ "num_http": "num_http + (if protocol == 'HTTP' then 1 else 0)"
+ },
+ "result": "num_dns / num_http"
+ }
+ ]
+}
+</pre></div></div>
+<p>This creates a profile…</p>
+
+<ul>
+
+<li>Named ‘example2’</li>
+
+<li>That for each IP source address</li>
+
+<li>Only if the ‘protocol’ field equals ‘HTTP’ or ‘DNS’</li>
+
+<li>Accumulates the number of DNS requests</li>
+
+<li>Accumulates the number of HTTP requests</li>
+
+<li>Returns the ratio of these as the result</li>
+</ul></div>
+<div class="section">
+<h3><a name="Example_3"></a>Example 3</h3>
+<p>The average of the <tt>length</tt> field of HTTP traffic. The following configuration would be used to generate this profile.</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "profiles": [
+ {
+ "profile": "example3",
+ "foreach": "ip_src_addr",
+ "onlyif": "protocol == 'HTTP'",
+ "update": { "s": "STATS_ADD(s, length)" },
+ "result": "STATS_MEAN(s)"
+ }
+ ]
+}
+</pre></div></div>
+<p>This creates a profile…</p>
+
+<ul>
+
+<li>Named ‘example3’</li>
+
+<li>That for each IP source address</li>
+
+<li>Only if the ‘protocol’ field is ‘HTTP’</li>
+
+<li>Adds the <tt>length</tt> field from each message</li>
+
+<li>Calculates the average as the result</li>
+</ul></div>
+<div class="section">
+<h3><a name="Example_4"></a>Example 4</h3>
+<p>It is important to note that the Profiler can persist any serializable Object, not just numeric values. An alternative to the previous example could take advantage of this. </p>
+<p>Instead of storing the mean of the lengths, the profile could store a statistical summarization of the lengths. This summary can then be used at a later time to calculate the mean, min, max, percentiles, or any other sensible metric. This provides a much greater degree of flexibility.</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "profiles": [
+ {
+ "profile": "example4",
+ "foreach": "ip_src_addr",
+ "onlyif": "protocol == 'HTTP'",
+ "update": { "s": "STATS_ADD(s, length)" },
+ "result": "s"
+ }
+ ]
+}
+</pre></div></div>
+<p>The following Stellar REPL session shows how you might use this summary to calculate different metrics with the same underlying profile data. It is assumed that the PROFILE_GET client is configured as described <a href="../metron-profiler-client/index.html">here</a>.</p>
+<p>Retrieve the last 30 minutes of profile measurements for a specific host.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ bin/stellar -z node1:2181
+
+[Stellar]>>> stats := PROFILE_GET( "example4", "10.0.0.1", PROFILE_FIXED(30, "MINUTES"))
+[Stellar]>>> stats
+[org.apache.metron.common.math.stats.OnlineStatisticsProvider@79fe4ab9, ...]
+</pre></div></div>
+<p>Calculate different metrics with the same profile data.</p>
+
+<div class="source">
+<div class="source">
+<pre>[Stellar]>>> STATS_MEAN( GET_FIRST( stats))
+15979.0625
+
+[Stellar]>>> STATS_PERCENTILE( GET_FIRST(stats), 90)
+30310.958
+</pre></div></div>
+<p>Merge all of the profile measurements over the past 30 minutes into a single summary and calculate the 90th percentile.</p>
+
+<div class="source">
+<div class="source">
+<pre>[Stellar]>>> merged := STATS_MERGE( stats)
+[Stellar]>>> STATS_PERCENTILE(merged, 90)
+29810.992
+</pre></div></div>
+<p>More information on accessing profile data can be found in the <a href="../metron-profiler-client/index.html">Profiler Client</a>.</p>
+<p>More information on using the <a href="../../metron-platform/metron-common/index.html"><tt>STATS_*</tt> functions in Stellar can be found here</a>.</p></div></div>
+<div class="section">
+<h2><a name="Implementation"></a>Implementation</h2></div>
+<div class="section">
+<h2><a name="Key_Classes"></a>Key Classes</h2>
+
+<ul>
+
+<li>
+<p><tt>ProfileMeasurement</tt> - Represents a single data point within a Profile. A Profile is effectively a time series. To this end a Profile is composed of many ProfileMeasurement values which in aggregate form a time series.</p></li>
+
+<li>
+<p><tt>ProfilePeriod</tt> - The Profiler captures one <tt>ProfileMeasurement</tt> each <tt>ProfilePeriod</tt>. A <tt>ProfilePeriod</tt> will occur at fixed, deterministic points in time. This allows for efficient retrieval of profile data.</p></li>
+
+<li>
+<p><tt>RowKeyBuilder</tt> - Builds row keys that can be used to read or write profile data to HBase.</p></li>
+
+<li>
+<p><tt>ColumnBuilder</tt> - Defines the columns of data stored with a profile measurement.</p></li>
+
+<li>
+<p><tt>ProfileHBaseMapper</tt> - Defines for the <tt>HBaseBolt</tt> how profile measurements are stored in HBase. This class leverages a <tt>RowKeyBuilder</tt> and <tt>ColumnBuilder</tt>.</p></li>
+</ul></div>
+<div class="section">
+<h2><a name="Storm_Topology"></a>Storm Topology</h2>
+<p>The Profiler is implemented as a Storm topology using the following bolts and spouts.</p>
+
+<ul>
+
+<li>
+<p><tt>KafkaSpout</tt> - A spout that consumes messages from a single Kafka topic. In most cases, the Profiler topology will consume messages from the <tt>indexing</tt> topic. This topic contains fully enriched messages that are ready to be indexed. This ensures that profiles can take advantage of all the available data elements.</p></li>
+
+<li>
+<p><tt>ProfileSplitterBolt</tt> - The bolt responsible for filtering incoming messages and directing each to the one or more downstream bolts that are responsible for building a profile. Each message may be needed by 0, 1 or even many profiles. Each emitted tuple contains the ‘resolved’ entity name, the profile definition, and the input message.</p></li>
+
+<li>
+<p><tt>ProfileBuilderBolt</tt> - This bolt maintains all of the state required to build a profile. When the window period expires, the data is summarized as a <tt>ProfileMeasurement</tt>, all state is flushed, and the <tt>ProfileMeasurement</tt> is emitted. Each instance of this bolt is responsible for maintaining the state for a single Profile-Entity pair.</p></li>
+
+<li>
+<p><tt>HBaseBolt</tt> - A bolt that is responsible for writing to HBase. Most profiles will be flushed every 15 minutes or so. If each <tt>ProfileBuilderBolt</tt> were responsible for writing to HBase itself, there would be little to no opportunity to optimize these writes. By aggregating the writes from multiple Profile-Entity pairs these writes can be batched, for example.</p></li>
+</ul></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
[08/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-analytics/metron-statistics/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/metron-statistics/index.html b/site/current-book/metron-analytics/metron-statistics/index.html
new file mode 100644
index 0000000..c167eaf
--- /dev/null
+++ b/site/current-book/metron-analytics/metron-statistics/index.html
@@ -0,0 +1,916 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Statistics and Mathematical Functions</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Statistics and Mathematical Functions</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-down"></i>
+ Analytics</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service">
+ <i class="none"></i>
+ Maas-service</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-profiler/index.html" title="Profiler">
+ <i class="none"></i>
+ Profiler</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client">
+ <i class="none"></i>
+ Profiler-client</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="icon-chevron-down"></i>Statistics</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-analytics/metron-statistics/HLLP.html" title="HLLP">
+ <i class="none"></i>
+ HLLP</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Statistics and Mathematical Functions</h1>
+<p><a name="Statistics_and_Mathematical_Functions"></a></p>
+<p>A variety of non-trivial and advanced analytics make use of statistics and advanced mathematical functions. Particular, capturing the statistical snapshots in a scalable way can open up doors for more advanced analytics such as outlier analysis. As such, this project is aimed at capturing a robust set of statistical functions and statistical-based algorithms in the form of Stellar functions. These functions can be used from everywhere where Stellar is used.</p>
+<div class="section">
+<h2><a name="Stellar_Functions"></a>Stellar Functions</h2>
+<div class="section">
+<h3><a name="Approximation_Statistics"></a>Approximation Statistics</h3>
+<div class="section">
+<h4><a name="HLLP_ADD"></a><tt>HLLP_ADD</tt></h4>
+
+<ul>
+
+<li>Description: Add value to the HyperLogLogPlus estimator set. See <a href="HLLP.html">HLLP README</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>hyperLogLogPlus - the hllp estimator to add a value to</li>
+
+<li>value+ - value to add to the set. Takes a single item or a list.</li>
+ </ul></li>
+
+<li>Returns: The HyperLogLogPlus set with a new value added</li>
+</ul></div>
+<div class="section">
+<h4><a name="HLLP_CARDINALITY"></a><tt>HLLP_CARDINALITY</tt></h4>
+
+<ul>
+
+<li>Description: Returns HyperLogLogPlus-estimated cardinality for this set. See <a href="HLLP.html">HLLP README</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>hyperLogLogPlus - the hllp set</li>
+ </ul></li>
+
+<li>Returns: Long value representing the cardinality for this set</li>
+</ul></div>
+<div class="section">
+<h4><a name="HLLP_INIT"></a><tt>HLLP_INIT</tt></h4>
+
+<ul>
+
+<li>Description: Initializes the HyperLogLogPlus estimator set. p must be a value between 4 and sp and sp must be less than 32 and greater than 4. See <a href="HLLP.html">HLLP README</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>p - the precision value for the normal set</li>
+
+<li>sp - the precision value for the sparse set. If p is set, but sp is 0 or not specified, the sparse set will be disabled.</li>
+ </ul></li>
+
+<li>Returns: A new HyperLogLogPlus set</li>
+</ul></div>
+<div class="section">
+<h4><a name="HLLP_MERGE"></a><tt>HLLP_MERGE</tt></h4>
+
+<ul>
+
+<li>Description: Merge hllp sets together. The resulting estimator is initialized with p and sp precision values from the first provided hllp estimator set. See <a href="HLLP.html">HLLP README</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>hllp - List of hllp estimators to merge. Takes a single hllp set or a list.</li>
+ </ul></li>
+
+<li>Returns: A new merged HyperLogLogPlus estimator set</li>
+</ul></div></div>
+<div class="section">
+<h3><a name="Mathematical_Functions"></a>Mathematical Functions</h3>
+<div class="section">
+<h4><a name="ABS"></a><tt>ABS</tt></h4>
+
+<ul>
+
+<li>Description: Returns the absolute value of a number.</li>
+
+<li>Input:
+
+<ul>
+
+<li>number - The number to take the absolute value of</li>
+ </ul></li>
+
+<li>Returns: The absolute value of the number passed in.</li>
+</ul></div>
+<div class="section">
+<h4><a name="BIN"></a><tt>BIN</tt></h4>
+
+<ul>
+
+<li>Description: Computes the bin that the value is in given a set of bounds.</li>
+
+<li>Input:
+
+<ul>
+
+<li>value - The value to bin</li>
+
+<li>bounds - A list of value bounds (excluding min and max) in sorted order.</li>
+ </ul></li>
+
+<li>Returns: Which bin N the value falls in such that bound(N-1) < value <= bound(N). No min and max bounds are provided, so values smaller than the 0’th bound go in the 0’th bin, and values greater than the last bound go in the M’th bin.</li>
+</ul></div></div>
+<div class="section">
+<h3><a name="Distributional_Statistics"></a>Distributional Statistics</h3>
+<div class="section">
+<h4><a name="STATS_ADD"></a><tt>STATS_ADD</tt></h4>
+
+<ul>
+
+<li>Description: Adds one or more input values to those that are used to calculate the summary statistics.</li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object. If null, then a new one is initialized.</li>
+
+<li>value+ - One or more numbers to add</li>
+ </ul></li>
+
+<li>Returns: A Stellar statistics object</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_BIN"></a><tt>STATS_BIN</tt></h4>
+
+<ul>
+
+<li>Description: Computes the bin that the value is in based on the statistical distribution.</li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+
+<li>value - The value to bin</li>
+
+<li>bounds? - A list of percentile bin bounds (excluding min and max) or a string representing a known and common set of bins. For convenience, we have provided QUARTILE, QUINTILE, and DECILE which you can pass in as a string arg. If this argument is omitted, then we assume a Quartile bin split.</li>
+ </ul></li>
+
+<li>Returns: "Which bin N the value falls in such that bound(N-1) < value <= bound(N). No min and max bounds are provided, so values smaller than the 0’th bound go in the 0’th bin, and values greater than the last bound go in the M’th bin.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_COUNT"></a><tt>STATS_COUNT</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the count of the values accumulated (or in the window if a window is used).</li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The count of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_GEOMETRIC_MEAN"></a><tt>STATS_GEOMETRIC_MEAN</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the geometric mean of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The geometric mean of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_INIT"></a><tt>STATS_INIT</tt></h4>
+
+<ul>
+
+<li>Description: Initializes a statistics object</li>
+
+<li>Input:
+
+<ul>
+
+<li>window_size - The number of input data values to maintain in a rolling window in memory. If window_size is equal to 0, then no rolling window is maintained. Using no rolling window is less memory intensive, but cannot calculate certain statistics like percentiles and kurtosis.</li>
+ </ul></li>
+
+<li>Returns: A Stellar statistics object</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_KURTOSIS"></a><tt>STATS_KURTOSIS</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the kurtosis of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The kurtosis of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_MAX"></a><tt>STATS_MAX</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the maximum of the accumulated values (or in the window if a window is used).</li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The maximum of the accumulated values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_MEAN"></a><tt>STATS_MEAN</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the mean of the accumulated values (or in the window if a window is used).</li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The mean of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_MERGE"></a><tt>STATS_MERGE</tt></h4>
+
+<ul>
+
+<li>Description: Merges statistics objects.</li>
+
+<li>Input:
+
+<ul>
+
+<li>statistics - A list of statistics objects</li>
+ </ul></li>
+
+<li>Returns: A Stellar statistics object</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_MIN"></a><tt>STATS_MIN</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the minimum of the accumulated values (or in the window if a window is used).</li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The minimum of the accumulated values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_PERCENTILE"></a><tt>STATS_PERCENTILE</tt></h4>
+
+<ul>
+
+<li>Description: Computes the p’th percentile of the accumulated values (or in the window if a window is used).</li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+
+<li>p - a double where 0 <= p < 1 representing the percentile</li>
+ </ul></li>
+
+<li>Returns: The p’th percentile of the data or NaN if the statistics object is null</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_POPULATION_VARIANCE"></a><tt>STATS_POPULATION_VARIANCE</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the population variance of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The population variance of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_QUADRATIC_MEAN"></a><tt>STATS_QUADRATIC_MEAN</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the quadratic mean of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The quadratic mean of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_SD"></a><tt>STATS_SD</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the standard deviation of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The standard deviation of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_SKEWNESS"></a><tt>STATS_SKEWNESS</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the skewness of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The skewness of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_SUM"></a><tt>STATS_SUM</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the sum of the accumulated values (or in the window if a window is used).</li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The sum of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_SUM_LOGS"></a><tt>STATS_SUM_LOGS</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the sum of the (natural) log of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The sum of the (natural) log of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_SUM_SQUARES"></a><tt>STATS_SUM_SQUARES</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the sum of the squares of the accumulated values (or in the window if a window is used).</li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The sum of the squares of the values in the window or NaN if the statistics object is null.</li>
+</ul></div>
+<div class="section">
+<h4><a name="STATS_VARIANCE"></a><tt>STATS_VARIANCE</tt></h4>
+
+<ul>
+
+<li>Description: Calculates the variance of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>stats - The Stellar statistics object</li>
+ </ul></li>
+
+<li>Returns: The variance of the values in the window or NaN if the statistics object is null.</li>
+</ul></div></div>
+<div class="section">
+<h3><a name="Statistical_Outlier_Detection"></a>Statistical Outlier Detection</h3>
+<div class="section">
+<h4><a name="OUTLIER_MAD_STATE_MERGE"></a><tt>OUTLIER_MAD_STATE_MERGE</tt></h4>
+
+<ul>
+
+<li>Description: Update the statistical state required to compute the Median Absolute Deviation.</li>
+
+<li>Input:
+
+<ul>
+
+<li>[state] - A list of Median Absolute Deviation States to merge. Generally these are states across time.</li>
+
+<li>currentState? - The current state (optional)</li>
+ </ul></li>
+
+<li>Returns: The Median Absolute Deviation state</li>
+</ul></div>
+<div class="section">
+<h4><a name="OUTLIER_MAD_ADD"></a><tt>OUTLIER_MAD_ADD</tt></h4>
+
+<ul>
+
+<li>Description: Add a piece of data to the state.</li>
+
+<li>Input:
+
+<ul>
+
+<li>state - The MAD state</li>
+
+<li>value - The numeric value to add</li>
+ </ul></li>
+
+<li>Returns: The MAD state</li>
+</ul></div>
+<div class="section">
+<h4><a name="OUTLIER_MAD_SCORE"></a><tt>OUTLIER_MAD_SCORE</tt></h4>
+
+<ul>
+
+<li>Description: Get the modified z-score normalized by the MAD: scale * | x_i - median(X) | / MAD. See the first page of <a class="externalLink" href="http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf">http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf</a></li>
+
+<li>Input:
+
+<ul>
+
+<li>state - The MAD state</li>
+
+<li>value - The numeric value to score</li>
+
+<li>scale? - Optionally the scale to use when computing the modified z-score. Default is <tt>0.6745</tt>, see the first page of <a class="externalLink" href="http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf">http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf</a></li>
+ </ul></li>
+
+<li>Returns: The modified z-score</li>
+</ul>
+<p><a name="Outlier_Analysis"></a></p>
+<h1>Outlier Analysis</h1>
+<p>A common desire is to find anomalies in numerical data. To that end, we have some simple statistical anomaly detectors.</p></div></div></div>
+<div class="section">
+<h2><a name="Median_Absolute_Deviation"></a>Median Absolute Deviation</h2>
+<p>Much has been written about this robust estimator. See the first page of <a class="externalLink" href="http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf">http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf</a> for a good coverage of the good and the bad of MAD. The usage, however is fairly straightforward:</p>
+
+<ul>
+
+<li>Gather the statistical state required to compute the MAD
+
+<ul>
+
+<li>The distribution of the values of a univariate random variable over time.</li>
+
+<li>The distribution of the absolute deviations of the values from the median.</li>
+ </ul></li>
+
+<li>Use this statistical state to score unseen values. The higher the score, the more unlike the previously seen data the value is.</li>
+</ul>
+<p>There are a couple of issues which make MAD a bit hard to compute. First, the statistical state requires computing median, which can be computationally expensive to compute exactly. To get around this, we use the OnlineStatisticalProvider to compute a sketch rather than the exact median. Secondly, the statistical state for seasonal data should be limited to a fixed, trailing window. We do this by ensuring that the MAD state is mergeable and able to be queried from within the Profiler.</p>
+<div class="section">
+<h3><a name="Example"></a>Example</h3>
+<p>We will create a dummy data stream of gaussian noise to illustrate how to use the MAD functionality along with the profiler to tag messages as outliers or not.</p>
+<p>To do this, we will create a </p>
+
+<ul>
+
+<li>data generator</li>
+
+<li>parser</li>
+
+<li>profiler profile</li>
+
+<li>enrichment and threat triage</li>
+</ul>
+<div class="section">
+<h4><a name="Data_Generator"></a>Data Generator</h4>
+<p>We can create a simple python script to generate a stream of gaussian noise at the frequency of one message per second as a python script which should be saved at <tt>~/rand_gen.py</tt>:</p>
+
+<div class="source">
+<div class="source">
+<pre>#!/usr/bin/python
+import random
+import sys
+import time
+def main():
+ mu = float(sys.argv[1])
+ sigma = float(sys.argv[2])
+ freq_s = int(sys.argv[3])
+ while True:
+ print str(random.gauss(mu, sigma))
+ sys.stdout.flush()
+ time.sleep(freq_s)
+
+if __name__ == '__main__':
+ main()
+</pre></div></div>
+<p>This script will take the following as arguments:</p>
+
+<ul>
+
+<li>The mean of the data generated</li>
+
+<li>The standard deviation of the data generated</li>
+
+<li>The frequency (in seconds) of the data generated</li>
+</ul>
+<p>If, however, you’d like to test a longer tailed distribution, like the student t-distribution and have numpy installed, you can use the following as <tt>~/rand_gen.py</tt>:</p>
+
+<div class="source">
+<div class="source">
+<pre>#!/usr/bin/python
+import random
+import sys
+import time
+import numpy as np
+
+def main():
+ df = float(sys.argv[1])
+ freq_s = int(sys.argv[2])
+ while True:
+ print str(np.random.standard_t(df))
+ sys.stdout.flush()
+ time.sleep(freq_s)
+
+if __name__ == '__main__':
+ main()
+</pre></div></div>
+<p>This script will take the following as arguments:</p>
+
+<ul>
+
+<li>The degrees of freedom for the distribution</li>
+
+<li>The frequency (in seconds) of the data generated</li>
+</ul></div>
+<div class="section">
+<h4><a name="The_Parser"></a>The Parser</h4>
+<p>We will create a parser that will take the single numbers in and create a message with a field called <tt>value</tt> in them using the <tt>CSVParser</tt>.</p>
+<p>Add the following file to <tt>$METRON_HOME/config/zookeeper/parsers/mad.json</tt>:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "parserClassName" : "org.apache.metron.parsers.csv.CSVParser"
+ ,"sensorTopic" : "mad"
+ ,"parserConfig" : {
+ "columns" : {
+ "value_str" : 0
+ }
+ }
+ ,"fieldTransformations" : [
+ {
+ "transformation" : "STELLAR"
+ ,"output" : [ "value" ]
+ ,"config" : {
+ "value" : "TO_DOUBLE(value_str)"
+ }
+ }
+ ]
+}
+</pre></div></div></div>
+<div class="section">
+<h4><a name="Enrichment_and_Threat_Intel"></a>Enrichment and Threat Intel</h4>
+<p>We will set a threat triage level of <tt>10</tt> if a message generates a outlier score of more than 3.5. This cutoff will depend on your data and should be adjusted based on the assumed underlying distribution. Note that under the assumptions of normality, MAD will act as a robust estimator of the standard deviation, so the cutoff should be considered the number of standard deviations away. For other distributions, there are other interpretations which will make sense in the context of measuring the “degree different”. See <a class="externalLink" href="http://eurekastatistics.com/using-the-median-absolute-deviation-to-find-outliers/">http://eurekastatistics.com/using-the-median-absolute-deviation-to-find-outliers/</a> for a brief discussion of this.</p>
+<p>Create the following in <tt>$METRON_HOME/config/zookeeper/enrichments/mad.json</tt>:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "index": "mad",
+ "batchSize": 1,
+ "enrichment": {
+ "fieldMap": {
+ "stellar" : {
+ "config" : {
+ "parser_score" : "OUTLIER_MAD_SCORE(OUTLIER_MAD_STATE_MERGE(
+PROFILE_GET( 'sketchy_mad', 'global', PROFILE_FIXED(10, 'MINUTES')) ), value)"
+ ,"is_alert" : "if parser_score > 3.5 then true else is_alert"
+ }
+ }
+ }
+ ,"fieldToTypeMap": { }
+ },
+ "threatIntel": {
+ "fieldMap": { },
+ "fieldToTypeMap": { },
+ "triageConfig" : {
+ "riskLevelRules" : [
+ {
+ "rule" : "parser_score > 3.5",
+ "score" : 10
+ }
+ ],
+ "aggregator" : "MAX"
+ }
+ }
+}
+</pre></div></div></div>
+<div class="section">
+<h4><a name="The_Profiler"></a>The Profiler</h4>
+<p>We can set up the profiler to track the MAD statistical state required to compute MAD. For the purposes of this demonstration, we will configure the profiler to capture statistics on the minute mark. We will capture a global statistical state for the <tt>value</tt> field and we will look back for a 5 minute window when computing the median.</p>
+<p>Create the following file at <tt>$METRON_HOME/config/zookeeper/profiler.json</tt>:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "profiles": [
+ {
+ "profile": "sketchy_mad",
+ "foreach": "'global'",
+ "onlyif": "true",
+ "init" : {
+ "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
+'global', PROFILE_FIXED(5, 'MINUTES')))"
+ },
+ "update": {
+ "s": "OUTLIER_MAD_ADD(s, value)"
+ },
+ "result": "s"
+ }
+ ]
+}
+</pre></div></div>
+<p>Adjust <tt>$METRON_HOME/config/zookeeper/global.json</tt> to adjust the capture duration:</p>
+
+<div class="source">
+<div class="source">
+<pre> "profiler.client.period.duration" : "1",
+ "profiler.client.period.duration.units" : "MINUTES"
+</pre></div></div>
+<p>Adjust <tt>$METRON_HOME/config/profiler.properties</tt> to adjust the capture duration by changing <tt>profiler.period.duration=15</tt> to <tt>profiler.period.duration=1</tt></p></div>
+<div class="section">
+<h4><a name="Execute_the_Flow"></a>Execute the Flow</h4>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Install the elasticsearch head plugin by executing: <tt>/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head</tt></p></li>
+
+<li>
+<p>Stopping all other parser topologies via monit</p></li>
+
+<li>
+<p>Create the <tt>mad</tt> kafka topic by executing: <tt>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper node1:2181 --create --topic mad --partitions 1 --replication-factor 1</tt></p></li>
+
+<li>
+<p>Push the modified configs by executing: <tt>$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -z node1:2181 -i $METRON_HOME/config/zookeeper/</tt></p></li>
+
+<li>
+<p>Start the profiler by executing: <tt>$METRON_HOME/bin/start_profiler_topology.sh</tt></p></li>
+
+<li>
+<p>Start the parser topology by executing: <tt>$METRON_HOME/bin/start_parser_topology.sh -k node1:6667 -z node1:2181 -s mad</tt></p></li>
+
+<li>
+<p>Ensure that the enrichment and indexing topologies are started. If not, then start those via monit or by hand.</p></li>
+
+<li>
+<p>Generate data into kafka by executing the following for at least 10 minutes: <tt>~/rand_gen.py 0 1 1 | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic mad</tt> Note: if you chose the use the t-distribution script above, you would adjust the parameters of the <tt>rand_gen.py</tt> script accordingly.</p></li>
+
+<li>
+<p>Stop the above with ctrl-c and send in an obvious outlier into kafka: <tt>echo "1000" | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic mad</tt></p></li>
+</ol>
+<p>You should be able to find the outlier via the elasticsearch head plugin by searching for the messages where <tt>is_alert</tt> is <tt>true</tt>.</p></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/amazon-ec2/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/amazon-ec2/index.html b/site/current-book/metron-deployment/amazon-ec2/index.html
new file mode 100644
index 0000000..8eab79a
--- /dev/null
+++ b/site/current-book/metron-deployment/amazon-ec2/index.html
@@ -0,0 +1,521 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Apache Metron on Amazon EC2</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Apache Metron on Amazon EC2</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-right"></i>
+ Roles</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Apache Metron on Amazon EC2</h1>
+<p>This project fully automates the provisioning of Apache Metron on Amazon EC2 infrastructure. Starting with only your Amazon EC2 credentials, this project will create a fully-functioning, end-to-end, multi-node cluster running Apache Metron.</p>
+<p>Warning: Amazon will charge for the use of their resources when running Apache Metron. The amount will vary based on the number and size of hosts, along with current Amazon pricing structure. Be sure to stop or terminate all of the hosts instantiated by Apache Metron when not in use to avoid unnecessary charges.</p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<p>The host used to deploy Apache Metron will need the following software tools installed. The following versions are known to work as of the time of this writing, but by no means are these the only working versions.</p>
+
+<ul>
+
+<li>Ansible 2.0.0.2</li>
+
+<li>Python 2.7.11</li>
+
+<li>Maven 3.3.9</li>
+</ul>
+<p>Any platform that supports these tools is suitable, but the following instructions cover only macOS. The easiest means of installing these tools on a Mac is to use the excellent <a class="externalLink" href="http://brew.sh/">Homebrew</a> project.</p>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Install Homebrew by running the following command in a terminal. Refer to the <a class="externalLink" href="http://brew.sh/">Homebrew</a> home page for the latest installation instructions.</p>
+
+<div class="source">
+<div class="source">
+<pre> /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
+</pre></div></div></li>
+
+<li>
+<p>With Homebrew installed, run the following command in a terminal to install all of the required tools.</p>
+
+<div class="source">
+<div class="source">
+<pre> brew cask install java
+ brew install maven git
+ brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/ee1273bf919a5e4e50838513a9e55ea423e1d7ce/Formula/ansible.rb
+ brew switch ansible 2.0.0.2
+</pre></div></div></li>
+
+<li>
+<p>Ensure that a public SSH key is located at <tt>~/.ssh/id_rsa.pub</tt>. </p>
+
+<div class="source">
+<div class="source">
+<pre> $ cat ~/.ssh/id_rsa.pub
+ ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQChv5GJxPjR39UJV7VY17ivbLVlxFrH7UHwh1Jsjem4d1eYiAtde5N2y65/HRNxWbhYli9ED8k0/MRP92ejewucEbrPNq5mytPqdC4IvZ98Ln2GbqTDwvlP3T7xa/wYFOpFsOmXXql8216wSrnrS4f3XK7ze34S6/VmY+lsBYnr3dzyj8sG/mexpJgFS/w83mWJV0e/ryf4Hd7P6DZ5fO+nmTXfKNK22ga4ctcnbZ+toYcPL+ODCh8598XCKVo97XjwF5OxN3vl1p1HHguo3cHB4H1OIaqX5mUt59gFIZcAXUME89PO6NUiZDd3RTstpf125nQVkQAHu2fvW96/f037 nick@localhost
+</pre></div></div>
+<p>If this file does not exist, run the following command at a terminal and accept all defaults. Only the public key, not the private key, will be uploaded to Amazon and configured on each host to enable SSH connectivity. While it is possible to create and use an alternative key those details will not be covered. </p>
+
+<div class="source">
+<div class="source">
+<pre> ssh-keygen -t rsa
+</pre></div></div></li>
+</ol></div>
+<div class="section">
+<h3><a name="Amazon_Web_Services"></a>Amazon Web Services</h3>
+<p>If you already have an Amazon Web Services account that you have used to deploy EC2 hosts, then you should be able to skip the next few steps.</p>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Head over to <a class="externalLink" href="http://aws.amazon.com/">Amazon Web Services</a> and create an account. As part of the account creation process you will need to provide a credit card to cover any charges that may apply.</p></li>
+
+<li>
+<p>Create a set of user credentials through <a class="externalLink" href="https://console.aws.amazon.com/iam/">Amazon’s Identity and Access Management (IAM) </a> dashboard. On the IAM dashboard menu click “Users” and then “Create New User”. Provide a name and ensure that “Generate an access key for each user” remains checked. Download the credentials and keep them for later use.</p></li>
+
+<li>
+<p>While still in <a class="externalLink" href="https://console.aws.amazon.com/iam/">Amazon’s Identity and Access Management (IAM) </a> dashboard, click on the user that was previously created. Click the “Permissions” tab and then the “Attach Policy” button. Attach the following policies to the user.</p>
+
+<ul>
+
+<li>AmazonEC2FullAccess</li>
+
+<li>AmazonVPCFullAccess</li>
+ </ul></li>
+
+<li>
+<p>Apache Metron uses the <a class="externalLink" href="https://aws.amazon.com/marketplace/pp/B00NQAYLWO">official, open source CentOS 6</a> Amazon Machine Image (AMI). If you have never used this AMI before then you will need to accept Amazon’s terms and conditions. Navigate to the <a class="externalLink" href="https://aws.amazon.com/marketplace/pp/B00NQAYLWO">web page for this AMI</a> and click the “Continue” button. Choose the “Manual Launch” tab then click the “Accept Software Terms” button.</p></li>
+</ol>
+<p>Having successfully created your Amazon Web Services account, hopefully you will find that the most difficult tasks are behind us. </p></div>
+<div class="section">
+<h3><a name="Deploy_Metron"></a>Deploy Metron</h3>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Use the Amazon access key by exporting its values via the shell’s environment. This allows Ansible to authenticate with Amazon EC2. For example:</p>
+
+<div class="source">
+<div class="source">
+<pre> export AWS_ACCESS_KEY_ID="AKIAI6NRFEO27E5FFELQ"
+ export AWS_SECRET_ACCESS_KEY="vTDydWJQnAer7OWauUS150i+9Np7hfCXrrVVP6ed"
+</pre></div></div>
+<p>Notice: You must replace the access key values above with values from your own access key.</p></li>
+
+<li>
+<p>Start the Apache Metron deployment process. When prompted provide a unique name for your Metron environment or accept the default. </p>
+
+<div class="source">
+<div class="source">
+<pre> $ ./run.sh
+ Metron Environment [metron-test]: my-metron-env
+ ...
+</pre></div></div>
+<p>The process is likely to take between 70-90 minutes. Fortunately, everything is fully automated and you should feel free to grab a coffee.</p></li>
+</ol></div>
+<div class="section">
+<h3><a name="Explore_Metron"></a>Explore Metron</h3>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>After the deployment has completed successfully, a message like the following will be displayed. Navigate to the specified resources to explore your newly minted Apache Metron environment.</p>
+
+<div class="source">
+<div class="source">
+<pre> TASK [debug] *******************************************************************
+ ok: [localhost] => {
+ "Success": [
+ "Apache Metron deployed successfully",
+ " Metron @ http://ec2-52-37-255-142.us-west-2.compute.amazonaws.com:5000",
+ " Ambari @ http://ec2-52-37-225-202.us-west-2.compute.amazonaws.com:8080",
+ " Sensors @ ec2-52-37-225-202.us-west-2.compute.amazonaws.com on tap0",
+ "For additional information, see https://metron.incubator.apache.org/'"
+ ]
+ }
+</pre></div></div></li>
+
+<li>
+<p>Each of the provisioned hosts will be accessible from the internet. Connecting to one over SSH as the user <tt>centos</tt> will not require a password as it will authenticate with the pre-defined SSH key. </p>
+
+<div class="source">
+<div class="source">
+<pre> ssh centos@ec2-52-91-215-174.compute-1.amazonaws.com
+</pre></div></div></li>
+</ol></div></div>
+<div class="section">
+<h2><a name="Advanced_Usage"></a>Advanced Usage</h2>
+<div class="section">
+<h3><a name="Multiple_Environments"></a>Multiple Environments</h3>
+<p>This process can support provisioning of multiple, isolated environments. Simply change the <tt>env</tt> settings in <tt>conf/defaults.yml</tt>. For example, you might provision separate development, test, and production environments.</p>
+
+<div class="source">
+<div class="source">
+<pre>env: metron-test
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Selective_Provisioning"></a>Selective Provisioning</h3>
+<p>To provision only subsets of the entire Metron deployment, Ansible tags can be specified. For example, to only deploy the sensors on an Amazon EC2 environment, run the following command.</p>
+
+<div class="source">
+<div class="source">
+<pre>ansible-playbook -i ec2.py playbook.yml --tags "ec2,sensors"
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Custom_SSH_Key"></a>Custom SSH Key</h3>
+<p>By default, the playbook will attempt to register your public SSH key <tt>~/.ssh/id_rsa.pub</tt> with each provisioned host. This enables Ansible to communicate with each host using an SSH connection. If would prefer to use another key simply add the path to the public key file to the <tt>key_file</tt> property in <tt>conf/defaults.yml</tt>.</p>
+<p>For example, generate a new SSH key for Metron that will be stored at <tt>~/.ssh/my-metron-key</tt>.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ ssh-keygen -q -f ~/.ssh/my-metron-key
+Enter passphrase (empty for no passphrase):
+Enter same passphrase again:
+</pre></div></div>
+<p>Add the path to the newly created SSH public key to <tt>conf/defaults.yml</tt>.</p>
+
+<div class="source">
+<div class="source">
+<pre>key_file: ~/.ssh/metron-private-key.pub
+</pre></div></div></div></div>
+<div class="section">
+<h2><a name="Common_Errors"></a>Common Errors</h2>
+<div class="section">
+<h3><a name="Error:_unsupported_operation_exception_custom_format_isnt_supported"></a>Error: [unsupported_operation_exception] custom format isn’t supported</h3>
+<p>This error might be seen within Metron’s default dashboard in Kibana 4. This occurs when the index templates do not exist for the Snort, Bro or YAF indices in Elasticsearch. </p>
+<p>The dashboard expects fields to be of a certain type. If the index templates have not been loaded correctly, the data types for the fields in these indices will be incorrect and the dashboard will display this error.</p>
+<div class="section">
+<h4><a name="Solution"></a>Solution</h4>
+<p>If you see this error, please report your findings by creating a JIRA or dropping an email to the Metron Users mailing list. Follow these steps to work around the problem.</p>
+<p>(1) Define which Elasticsearch host to interact with. Any Elasticsearch host should work.</p>
+
+<div class="source">
+<div class="source">
+<pre>export ES_HOST="http://ec2-52-25-237-20.us-west-2.compute.amazonaws.com:9200"
+</pre></div></div>
+<p>(2) Confirm the index templates are in fact missing. </p>
+
+<div class="source">
+<div class="source">
+<pre>curl -s -XGET $ES_HOST/_template
+</pre></div></div>
+<p>(3) Manually load the index templates.</p>
+
+<div class="source">
+<div class="source">
+<pre>cd metron-deployment
+curl -s -XPOST $ES_HOST/_template/bro_index -d @roles/metron_elasticsearch_templates/files/es_templates/bro_index.template
+curl -s -XPOST $ES_HOST/_template/snort_index -d @roles/metron_elasticsearch_templates/files/es_templates/snort_index.template
+curl -s -XPOST $ES_HOST/_template/yaf_index -d @roles/metron_elasticsearch_templates/files/es_templates/yaf_index.template
+</pre></div></div>
+<p>(4) Delete the existing indexes. Only a new index will use the templates defined in the previous step.</p>
+
+<div class="source">
+<div class="source">
+<pre>curl -s -XDELETE "$ES_HOST/yaf_index*"
+curl -s -XDELETE "$ES_HOST/bro_index*"
+curl -s -XDELETE "$ES_HOST/snort_index*"
+</pre></div></div>
+<p>(5) Open up Kibana and wait for the new indexes to be created. The dashboard should now work.</p></div></div>
+<div class="section">
+<h3><a name="Error:_No_handler_was_ready_to_authenticateCheck_your_credentials"></a>Error: ‘No handler was ready to authenticate…Check your credentials’</h3>
+
+<div class="source">
+<div class="source">
+<pre>TASK [Define keypair] **********************************************************
+failed: [localhost] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXbcb1AlWsEPP
+ r9jEFrn0yun3PYNidJ/...david@hasselhoff.com) => {"failed": true, "item": "ssh-r
+ sa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXbcb1AlWsEPPr9jEFr... david@hasselhoff.com",
+ "msg": "No handler was ready to authenticate. 1 handlers were checked.
+ ['HmacAuthV4Handler'] Check your credentials"}
+</pre></div></div>
+<div class="section">
+<h4><a name="Solution_1"></a>Solution 1</h4>
+<p>This occurs when Ansible does not have the correct AWS access keys. The following commands must return a valid access key that is defined within Amazon’s <a class="externalLink" href="https://console.aws.amazon.com/iam/">Identity and Access Management</a> console. </p>
+
+<div class="source">
+<div class="source">
+<pre>$ echo $AWS_ACCESS_KEY_ID
+AKIAI6NRFEO27E5FFELQ
+
+$ echo $AWS_SECRET_ACCESS_KEY
+vTDydWJQnAer7OWauUS150i+9Np7hfCXrrVVP6ed
+</pre></div></div></div>
+<div class="section">
+<h4><a name="Solution_2"></a>Solution 2</h4>
+<p>This error can occur if you have exported the correct AWS access key, but you are using <tt>sudo</tt> to run the Ansible playbook. Do not use the <tt>sudo</tt> command when running the Ansible playbook.</p></div></div>
+<div class="section">
+<h3><a name="Error:_OptInRequired:__you_need_to_accept_terms_and_subscribe"></a>Error: ‘OptInRequired: … you need to accept terms and subscribe’</h3>
+
+<div class="source">
+<div class="source">
+<pre>TASK [metron-test: Instantiate 1 host(s) as sensors,ambari_master,metron,ec2] **
+fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg":
+"Instance creation failed => OptInRequired: In order to use this AWS Marketplace
+product you need to accept terms and subscribe. To do so please visit
+http://aws.amazon.com/marketplace/pp?sku=6x5jmcajty9edm3f211pqjfn2"}
+to retry, use: --limit @playbook.retry
+</pre></div></div>
+<div class="section">
+<h4><a name="Solution"></a>Solution</h4>
+<p>Apache Metron uses the <a class="externalLink" href="https://aws.amazon.com/marketplace/pp?sku=6x5jmcajty9edm3f211pqjfn2">official CentOS 6 Amazon Machine Image</a> when provisioning hosts. Amazon requires that you accept certain terms and conditions when using any Amazon Machine Image (AMI). Follow the link provided in the error message to accept the terms and conditions then re-run the playbook. </p></div></div>
+<div class="section">
+<h3><a name="Error:_PendingVerification:_Your_account_is_currently_being_verified"></a>Error: ‘PendingVerification: Your account is currently being verified’</h3>
+
+<div class="source">
+<div class="source">
+<pre>TASK [metron-test: Instantiate 1 host(s) as sensors,ambari_master,metron,ec2] **
+fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg":
+"Instance creation failed => PendingVerification: Your account is currently
+being verified. Verification normally takes less than 2 hours. Until your
+account is verified, you may not be able to launch additional instances or
+create additional volumes. If you are still receiving this message after more
+than 2 hours, please let us know by writing to aws-verification@amazon.com. We
+appreciate your patience."}
+to retry, use: --limit @playbook.retry
+</pre></div></div>
+<div class="section">
+<h4><a name="Solution"></a>Solution</h4>
+<p>This will occur if you are attempting to deploy Apache Metron using a newly created Amazon Web Services account. Follow the advice of the message and wait until Amazon’s verification process is complete. Amazon has some additional <a class="externalLink" href="http://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html">advice for dealing with this error and more</a>.</p>
+
+<blockquote>
+<p>Your account is pending verification. Until the verification process is complete, you may not be able to carry out requests with this account. If you have questions, contact <a class="externalLink" href="http://console.aws.amazon.com/support/home#/">AWS Support</a>.</p>
+</blockquote></div></div>
+<div class="section">
+<h3><a name="Error:_Instance_creation_failed__InstanceLimitExceeded"></a>Error: ‘Instance creation failed => InstanceLimitExceeded’</h3>
+
+<div class="source">
+<div class="source">
+<pre>TASK [metron-test: Instantiate 3 host(s) as search,metron,ec2] *****************
+fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg":
+"Instance creation failed => InstanceLimitExceeded: You have requested more
+instances (11) than your current instance limit of 10 allows for the specified
+instance type. Please visit http://aws.amazon.com/contact-us/ec2-request to
+request an adjustment to this limit."}
+to retry, use: --limit @playbook.retry
+</pre></div></div>
+<div class="section">
+<h4><a name="Solution"></a>Solution</h4>
+<p>This will occur if Apache Metron attempts to deploy more host instances than allowed by your account. The total number of instances required for Apache Metron can be reduced by editing <tt>deployment/amazon-ec/playbook.yml</tt>. Perhaps a better alternative is to request of Amazon that this limit be increased. Amazon has some additional <a class="externalLink" href="http://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html">advice for dealing with this error and more</a>.</p>
+
+<blockquote>
+<p>You’ve reached the limit on the number of instances you can run concurrently. The limit depends on the instance type. For more information, see <a class="externalLink" href="http://aws.amazon.com/ec2/faqs/#How_many_instances_can_I_run_in_Amazon_EC2">How many instances can I run in Amazon EC2</a>. If you need additional instances, complete the <a class="externalLink" href="https://console.aws.amazon.com/support/home#/case/create?issueType=service-limit-increase&limitType=service-code-ec2-instances">Amazon EC2 Instance Request Form</a>.</p>
+</blockquote></div></div>
+<div class="section">
+<h3><a name="Error:_SSH_encountered_an_unknown_error_during_the_connection"></a>Error: ‘SSH encountered an unknown error during the connection’</h3>
+
+<div class="source">
+<div class="source">
+<pre>TASK [setup] *******************************************************************
+fatal: [ec2-52-26-113-221.us-west-2.compute.amazonaws.com]: UNREACHABLE! => {
+ "changed": false, "msg": "SSH encountered an unknown error during the
+ connection. We recommend you re-run the command using -vvvv, which will enable
+ SSH debugging output to help diagnose the issue", "unreachable": true}
+</pre></div></div>
+<div class="section">
+<h4><a name="Solution"></a>Solution</h4>
+<p>This most often indicates that Ansible cannot connect to the host with the SSH key that it has access to. This could occur if hosts are provisioned with one SSH key, but the playbook is executed subsequently with a different SSH key. The issue can be addressed by either altering the <tt>key_file</tt> variable to point to the key that was used to provision the hosts or by simply terminating all hosts and re-running the playbook.</p></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/index.html b/site/current-book/metron-deployment/index.html
new file mode 100644
index 0000000..43e6678
--- /dev/null
+++ b/site/current-book/metron-deployment/index.html
@@ -0,0 +1,434 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Overview</title>
+ <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../css/site.css" />
+ <link rel="stylesheet" href="../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Overview</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="icon-chevron-down"></i>Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-right"></i>
+ Roles</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Overview</h1>
+<p><a name="Overview"></a></p>
+<p>This set of playbooks can be used to deploy an Ambari-managed Hadoop cluster, Metron services, or both using ansible playbooks. These playbooks currently only target RHEL/CentOS 6.x operating systems.</p>
+<p>In addition, an Ambari Management Pack can be built which can be deployed in conjuction with RPMs detailed in this README.</p>
+<div class="section">
+<h2><a name="Prerequisites"></a>Prerequisites</h2>
+<p>The following tools are required to run these scripts:</p>
+
+<ul>
+
+<li><a class="externalLink" href="https://maven.apache.org/">Maven</a></li>
+
+<li><a class="externalLink" href="https://git-scm.com/">Git</a></li>
+
+<li><a class="externalLink" href="http://www.ansible.com/">Ansible</a> (version 2.0 or greater)</li>
+</ul>
+<p>Currently Metron must be built from source. Before running these scripts perform the following steps:</p>
+
+<ol style="list-style-type: decimal">
+
+<li>Clone the Metron git repository with <tt>git clone git@github.com:apache/incubator-metron.git</tt></li>
+
+<li>Navigate to <tt>incubator-metron</tt> and run <tt>mvn clean package</tt></li>
+</ol>
+<p>These scripts depend on two files for configuration:</p>
+
+<ul>
+
+<li>hosts - declares which Ansible roles will be run on which hosts</li>
+
+<li>group_vars/all - various configuration settings needed to install Metron</li>
+</ul>
+<p>Examples can be found in the <tt>incubator-metron/metron-deployment/inventory/metron_example</tt> directory and are a good starting point. Copy this directory into <tt>incubator-metron/metron-deployment/inventory/</tt> and rename it to your <tt>project_name</tt>. More information about Ansible files and directory structure can be found at <a class="externalLink" href="http://docs.ansible.com/ansible/playbooks_best_practices.html">http://docs.ansible.com/ansible/playbooks_best_practices.html</a>.</p></div>
+<div class="section">
+<h2><a name="Ambari"></a>Ambari</h2>
+<p>The Ambari playbook will install a Hadoop cluster with all the services and configuration required by Metron. This section can be skipped if installing Metron on a pre-existing cluster.</p>
+<p>Currently, this playbook supports building a local development cluster running on one node but options for other types of clusters will be added in the future.</p>
+<div class="section">
+<h3><a name="Setting_up_your_inventory"></a>Setting up your inventory</h3>
+<p>Make sure to update the hosts file in <tt>incubator-metron/metron-deployment/inventory/project_name/hosts</tt> or provide an alternate inventory file when you launch the playbooks, including the ssh user(s) and ssh keyfile location(s). These playbooks expect two host groups:</p>
+
+<ul>
+
+<li>ambari_master</li>
+
+<li>ambari_slaves</li>
+</ul></div>
+<div class="section">
+<h3><a name="Running_the_playbook"></a>Running the playbook</h3>
+<p>This playbook will install the Ambari server on the ambari_master, install the ambari agents on the ambari_slaves, and create a cluster in Ambari with a blueprint for the required Metron components.</p>
+<p>Navigate to <tt>incubator-metron/metron-deployment/playbooks</tt> and run: <tt>ansible-playbook -i ../inventory/project_name ambari_install.yml</tt></p></div></div>
+<div class="section">
+<h2><a name="Metron"></a>Metron</h2>
+<p>The Metron playbook will gather the necessary cluster settings from Ambari and install the Metron services.</p>
+<div class="section">
+<h3><a name="Setting_up_your_inventory"></a>Setting up your inventory</h3>
+<p>Edit the hosts file at <tt>incubator-metron/metron-deployment/inventory/project_name/hosts</tt>. Declare where which hosts the Metron services will be installed on by updating these groups:</p>
+
+<ul>
+
+<li>enrichment - submits the topology code to Storm and requires a storm client</li>
+
+<li>search - host where Elasticsearch will be run</li>
+
+<li>web - host where the Metron UI and underlying services will run</li>
+
+<li>sensors - host where network data will be collected and published to Kafka</li>
+</ul>
+<p>The Metron topologies depend on Kafka topics and HBase tables being created beforehand. Declare a host that has Kafka and HBase clients installed by updating these groups:</p>
+
+<ul>
+
+<li>metron_kafka_topics</li>
+
+<li>metron_hbase_tables</li>
+</ul>
+<p>If only installing Metron, these groups can be ignored:</p>
+
+<ul>
+
+<li>ambari_master</li>
+
+<li>ambari_slaves</li>
+</ul></div>
+<div class="section">
+<h3><a name="Configuring_group_variables"></a>Configuring group variables</h3>
+<p>The Metron Ansible scripts depend on a set of variables. These variables can be found in the file at <tt>incubator-metron/metron-deployment/inventory/project_name/group_vars/all</tt>. Edit the ambari* variables to match your Ambari instance and update the java_home variable to match the java path on your hosts.</p></div>
+<div class="section">
+<h3><a name="Running_the_playbook"></a>Running the playbook</h3>
+<p>Navigate to <tt>incubator-metron/metron-deployment/playbooks</tt> and run: <tt>ansible-playbook -i ../inventory/project_name metron_install.yml</tt></p></div></div>
+<div class="section">
+<h2><a name="Vagrant"></a>Vagrant</h2>
+<p>A VagrantFile is included and will install a working version of the entire Metron stack. The following is required to run this:</p>
+
+<ul>
+
+<li><a class="externalLink" href="https://www.vagrantup.com/">Vagrant</a></li>
+
+<li>Hostmanager plugin for vagrant - Run <tt>vagrant plugin install vagrant-hostmanager</tt> on the machine where Vagrant is installed</li>
+</ul>
+<p>Navigate to <tt>incubator-metron/metron-deployment/vagrant/full-dev-platform</tt> and run <tt>vagrant up</tt>. This also provides a good example of how to run a full end-to-end Metron install.</p></div>
+<div class="section">
+<h2><a name="Ambari_Management_Pack"></a>Ambari Management Pack</h2>
+<p>An Ambari Management Pack can be built in order to make the Metron service available on top of an existing stack, rather than needing a direct stack update.</p>
+<p>This will set up</p>
+
+<ul>
+
+<li>Metron Parsers</li>
+
+<li>Enrichment</li>
+
+<li>Indexing</li>
+
+<li>GeoIP data</li>
+
+<li>Optional Elasticsearch</li>
+
+<li>Optional Kibana</li>
+</ul>
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+
+<ul>
+
+<li>A cluster managed by Ambari 2.4</li>
+
+<li>Metron RPMs available on the cluster in the /localrepo directory. See <a href="#RPM">RPM</a> for further information.</li>
+</ul></div>
+<div class="section">
+<h3><a name="Building_Management_Pack"></a>Building Management Pack</h3>
+<p>From <tt>metron-deployment</tt> run</p>
+
+<div class="source">
+<div class="source">
+<pre>mvn clean package
+</pre></div></div>
+<p>A tar.gz that can be used with Ambari can be found at <tt>metron-deployment/packaging/ambari/metron-mpack/target/</tt></p></div>
+<div class="section">
+<h3><a name="Installing_Management_Pack"></a>Installing Management Pack</h3>
+<p>Before installing the mpack, update Storm’s topology.classpath in Ambari to include ‘/etc/hbase/conf:/etc/hadoop/conf’. Restart Storm service.</p>
+<p>Place the mpack’s tar.gz onto the node running Ambari Server. From the command line on this node, run</p>
+
+<div class="source">
+<div class="source">
+<pre>ambari-server install-mpack --mpack=<mpack_location> --verbose
+</pre></div></div>
+<p>This will make the services available in Ambari in the same manner as any services in a stack, e.g. through Add Services or during cluster install. The Indexing / Parsers/ Enrichment masters should be colocated with a Kafka Broker (to create topics) and HBase client (to create the enrichment and theatintel tables). This colocation is currently not enforced by Ambari, and should be managed by either a Service or Stack advisor as an enhancement.</p>
+<p>Several configuration parameters will need to be filled in, and should be pretty self explanatory (primarily a couple of Elasticsearch configs, and the Storm REST URL). Examples are provided in the descriptions on Ambari. Notably, the URL for the GeoIP database that is preloaded (and is prefilled by default) can be set to use a <tt>file:///</tt> location</p>
+<p>After installation, a custom action is available in Ambari (where stop / start services are) to install Elasticsearch templates. Similar to this, a custom Kibana action to Load Template is available.</p>
+<p>Another custom action is available in Ambari to import Zeppelin dashboards. See the <a href="../metron-platform/metron-indexing/index.html">metron-indexing documentation</a></p>
+<div class="section">
+<h4><a name="Offline_installation"></a>Offline installation</h4>
+<p>Currently there is only one point that would reach out to the internet during an install. This is the URL for the GeoIP database information.</p>
+<p>The RPMs DO NOT reach out to the internet (because there is currently no hosting for them). They look on the local filesystem in <tt>/localrepo</tt>.</p></div></div>
+<div class="section">
+<h3><a name="Current_Limitations"></a>Current Limitations</h3>
+<p>There are a set of limitations that should be addressed based to improve the current state of the mpacks.</p>
+
+<ul>
+
+<li>There is currently no hosting for RPMs remotely. They will have to be built locally.</li>
+
+<li>Colocation of appropriate services should be enforced by Ambari. See [#Installing Management Pack] for more details.</li>
+
+<li>Storm’s topology.classpath is not updated with the Metron service install and needs to be updated separately.</li>
+
+<li>Several configuration parameters used when installing the Metron service could (and should) be grabbed from Ambari. Install will require them to be manually entered.</li>
+
+<li>Need to handle upgrading Metron</li>
+</ul></div></div>
+<div class="section">
+<h2><a name="RPM"></a>RPM</h2>
+<p>RPMs can be built to install the components in metron-platform. These RPMs are built in a Docker container and placed into <tt>target</tt>.</p>
+<p>Components in the RPMs:</p>
+
+<ul>
+
+<li>metron-common</li>
+
+<li>metron-data-management</li>
+
+<li>metron-elasticsearch</li>
+
+<li>metron-enrichment</li>
+
+<li>metron-parsers</li>
+
+<li>metron-pcap</li>
+
+<li>metron-solr</li>
+</ul>
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+
+<ul>
+
+<li>Docker. The image detailed in: <tt>metron-deployment/packaging/docker/rpm-docker/README.md</tt> will automatically be built (or rebuilt if necessary).</li>
+
+<li>Artifacts for metron-platform have been produced. E.g. <tt>mvn clean package -DskipTests</tt> in <tt>metron-platform</tt></li>
+</ul></div>
+<div class="section">
+<h3><a name="Building_RPMs"></a>Building RPMs</h3>
+<p>From <tt>metron-deployment</tt> run</p>
+
+<div class="source">
+<div class="source">
+<pre>mvn clean package -Pbuild-rpms
+</pre></div></div>
+<p>The output RPM files will land in <tt>target/RPMS/noarch</tt>. They can be installed with the standard</p>
+
+<div class="source">
+<div class="source">
+<pre>rpm -i <package>
+</pre></div></div></div></div>
+<div class="section">
+<h2><a name="TODO"></a>TODO</h2>
+
+<ul>
+
+<li>Support Ubuntu deployments</li>
+</ul></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
[13/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/css/apache-maven-fluido-1.3.0.min.css
----------------------------------------------------------------------
diff --git a/site/current-book/css/apache-maven-fluido-1.3.0.min.css b/site/current-book/css/apache-maven-fluido-1.3.0.min.css
new file mode 100644
index 0000000..7db4133
--- /dev/null
+++ b/site/current-book/css/apache-maven-fluido-1.3.0.min.css
@@ -0,0 +1,9 @@
+/*!
+ * Bootstrap v2.1.0
+ *
+ * Copyright 2012 Twitter, Inc
+ * Licensed under the Apache License v2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Designed and built with all the love in the world @twitter by @mdo and @fat.
+ */article,aside,details,figcaption,figure,footer,header,hgroup,nav,section{display:block}audio,canvas,video{display:inline-block;*display:inline;*zoom:1}audio:not([controls]){display:none}html{font-size:100%;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}a:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}a:hover,a:active{outline:0}sub,sup{position:relative;font-size:75%;line-height:0;vertical-align:baseline}sup{top:-0.5em}sub{bottom:-0.25em}img{height:auto;max-width:100%;vertical-align:middle;border:0;-ms-interpolation-mode:bicubic}#map_canvas img{max-width:none}button,input,select,textarea{margin:0;font-size:100%;vertical-align:middle}button,input{*overflow:visible;line-height:normal}button::-moz-focus-inner,input::-moz-focus-inner{padding:0;border:0}button,input[type="button"],input[type="reset"],input[type="submit"]{cursor:pointer;-webkit-appearance:button}input[type="search"]{-webkit-box-sizing:content-box;-moz-box-sizing:con
tent-box;box-sizing:content-box;-webkit-appearance:textfield}input[type="search"]::-webkit-search-decoration,input[type="search"]::-webkit-search-cancel-button{-webkit-appearance:none}textarea{overflow:auto;vertical-align:top}.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;line-height:0;content:""}.clearfix:after{clear:both}.hide-text{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.input-block-level{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}body{margin:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:14px;line-height:20px;color:#333;background-color:#fff}a{color:#08c;text-decoration:none}a:hover{color:#005580;text-decoration:underline}.img-rounded{-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.img-polaroid{padding:4px;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);-webkit-box-shadow:0
1px 3px rgba(0,0,0,0.1);-moz-box-shadow:0 1px 3px rgba(0,0,0,0.1);box-shadow:0 1px 3px rgba(0,0,0,0.1)}.img-circle{-webkit-border-radius:500px;-moz-border-radius:500px;border-radius:500px}.row{margin-left:-20px;*zoom:1}.row:before,.row:after{display:table;line-height:0;content:""}.row:after{clear:both}[class*="span"]{float:left;margin-left:20px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:940px}.span12{width:940px}.span11{width:860px}.span10{width:780px}.span9{width:700px}.span8{width:620px}.span7{width:540px}.span6{width:460px}.span5{width:380px}.span4{width:300px}.span3{width:220px}.span2{width:140px}.span1{width:60px}.offset12{margin-left:980px}.offset11{margin-left:900px}.offset10{margin-left:820px}.offset9{margin-left:740px}.offset8{margin-left:660px}.offset7{margin-left:580px}.offset6{margin-left:500px}.offset5{margin-left:420px}.offset4{margin-left:340px}.offset3{margin-left:260px}.offset2{margin-left:180px}.offs
et1{margin-left:100px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid [class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.127659574468085%;*margin-left:2.074468085106383%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.48936170212765%;*width:91.43617021276594%}.row-fluid .span10{width:82.97872340425532%;*width:82.92553191489361%}.row-fluid .span9{width:74.46808510638297%;*width:74.41489361702126%}.row-fluid .span8{width:65.95744680851064%;*width:65.90425531914893%}.row-fluid .span7{width:57.44680851063829%;*width:57.39361702127659%}.row-fluid .span6{width:48.93617021276595%;*width:48.88297872340425%}.row-fluid .span5{width:40.42553191489362%;*width:40.37234042553192%}.row-fluid .span4{width:31.9
14893617021278%;*width:31.861702127659576%}.row-fluid .span3{width:23.404255319148934%;*width:23.351063829787233%}.row-fluid .span2{width:14.893617021276595%;*width:14.840425531914894%}.row-fluid .span1{width:6.382978723404255%;*width:6.329787234042553%}.row-fluid .offset12{margin-left:104.25531914893617%;*margin-left:104.14893617021275%}.row-fluid .offset12:first-child{margin-left:102.12765957446808%;*margin-left:102.02127659574467%}.row-fluid .offset11{margin-left:95.74468085106382%;*margin-left:95.6382978723404%}.row-fluid .offset11:first-child{margin-left:93.61702127659574%;*margin-left:93.51063829787232%}.row-fluid .offset10{margin-left:87.23404255319149%;*margin-left:87.12765957446807%}.row-fluid .offset10:first-child{margin-left:85.1063829787234%;*margin-left:84.99999999999999%}.row-fluid .offset9{margin-left:78.72340425531914%;*margin-left:78.61702127659572%}.row-fluid .offset9:first-child{margin-left:76.59574468085106%;*margin-left:76.48936170212764%}.row-fluid .offset8{mar
gin-left:70.2127659574468%;*margin-left:70.10638297872339%}.row-fluid .offset8:first-child{margin-left:68.08510638297872%;*margin-left:67.9787234042553%}.row-fluid .offset7{margin-left:61.70212765957446%;*margin-left:61.59574468085106%}.row-fluid .offset7:first-child{margin-left:59.574468085106375%;*margin-left:59.46808510638297%}.row-fluid .offset6{margin-left:53.191489361702125%;*margin-left:53.085106382978715%}.row-fluid .offset6:first-child{margin-left:51.063829787234035%;*margin-left:50.95744680851063%}.row-fluid .offset5{margin-left:44.68085106382979%;*margin-left:44.57446808510638%}.row-fluid .offset5:first-child{margin-left:42.5531914893617%;*margin-left:42.4468085106383%}.row-fluid .offset4{margin-left:36.170212765957444%;*margin-left:36.06382978723405%}.row-fluid .offset4:first-child{margin-left:34.04255319148936%;*margin-left:33.93617021276596%}.row-fluid .offset3{margin-left:27.659574468085104%;*margin-left:27.5531914893617%}.row-fluid .offset3:first-child{margin-left:25
.53191489361702%;*margin-left:25.425531914893618%}.row-fluid .offset2{margin-left:19.148936170212764%;*margin-left:19.04255319148936%}.row-fluid .offset2:first-child{margin-left:17.02127659574468%;*margin-left:16.914893617021278%}.row-fluid .offset1{margin-left:10.638297872340425%;*margin-left:10.53191489361702%}.row-fluid .offset1:first-child{margin-left:8.51063829787234%;*margin-left:8.404255319148938%}[class*="span"].hide,.row-fluid [class*="span"].hide{display:none}[class*="span"].pull-right,.row-fluid [class*="span"].pull-right{float:right}.container{margin-right:auto;margin-left:auto;*zoom:1}.container:before,.container:after{display:table;line-height:0;content:""}.container:after{clear:both}.container-fluid{padding-right:20px;padding-left:20px;*zoom:1}.container-fluid:before,.container-fluid:after{display:table;line-height:0;content:""}.container-fluid:after{clear:both}p{margin:0 0 10px}.lead{margin-bottom:20px;font-size:20px;font-weight:200;line-height:30px}small{font-size:8
5%}strong{font-weight:bold}em{font-style:italic}cite{font-style:normal}.muted{color:#999}h1,h2,h3,h4,h5,h6{margin:10px 0;font-family:inherit;font-weight:bold;line-height:1;color:inherit;text-rendering:optimizelegibility}h1 small,h2 small,h3 small,h4 small,h5 small,h6 small{font-weight:normal;line-height:1;color:#999}h1{font-size:36px;line-height:40px}h2{font-size:30px;line-height:40px}h3{font-size:24px;line-height:40px}h4{font-size:18px;line-height:20px}h5{font-size:14px;line-height:20px}h6{font-size:12px;line-height:20px}h1 small{font-size:24px}h2 small{font-size:18px}h3 small{font-size:14px}h4 small{font-size:14px}.page-header{padding-bottom:9px;margin:20px 0 30px;border-bottom:1px solid #eee}ul,ol{padding:0;margin:0 0 10px 25px}ul ul,ul ol,ol ol,ol ul{margin-bottom:0}li{line-height:20px}ul.unstyled,ol.unstyled{margin-left:0;list-style:none}dl{margin-bottom:20px}dt,dd{line-height:20px}dt{font-weight:bold}dd{margin-left:10px}.dl-horizontal dt{float:left;width:120px;overflow:hidden;
clear:left;text-align:right;text-overflow:ellipsis;white-space:nowrap}.dl-horizontal dd{margin-left:130px}hr{margin:20px 0;border:0;border-top:1px solid #eee;border-bottom:1px solid #fff}abbr[title]{cursor:help;border-bottom:1px dotted #999}abbr.initialism{font-size:90%;text-transform:uppercase}blockquote{padding:0 0 0 15px;margin:0 0 20px;border-left:5px solid #eee}blockquote p{margin-bottom:0;font-size:16px;font-weight:300;line-height:25px}blockquote small{display:block;line-height:20px;color:#999}blockquote small:before{content:'\2014 \00A0'}blockquote.pull-right{float:right;padding-right:15px;padding-left:0;border-right:5px solid #eee;border-left:0}blockquote.pull-right p,blockquote.pull-right small{text-align:right}blockquote.pull-right small:before{content:''}blockquote.pull-right small:after{content:'\00A0 \2014'}q:before,q:after,blockquote:before,blockquote:after{content:""}address{display:block;margin-bottom:20px;font-style:normal;line-height:20px}code,pre{padding:0 3px 2px
;font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:12px;color:#333;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}code{padding:2px 4px;color:#d14;background-color:#f7f7f9;border:1px solid #e1e1e8}pre{display:block;padding:9.5px;margin:0 0 10px;font-size:13px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre;white-space:pre-wrap;background-color:#f5f5f5;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.15);-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}pre.prettyprint{margin-bottom:20px}pre code{padding:0;color:inherit;background-color:transparent;border:0}.pre-scrollable{max-height:340px;overflow-y:scroll}form{margin:0 0 20px}fieldset{padding:0;margin:0;border:0}legend{display:block;width:100%;padding:0;margin-bottom:20px;font-size:21px;line-height:40px;color:#333;border:0;border-bottom:1px solid #e5e5e5}legend small{font-size:15px;color:#999}label,input,button,select,textarea{font-size:14px;font-we
ight:normal;line-height:20px}input,button,select,textarea{font-family:"Helvetica Neue",Helvetica,Arial,sans-serif}label{display:block;margin-bottom:5px}select,textarea,input[type="text"],input[type="password"],input[type="datetime"],input[type="datetime-local"],input[type="date"],input[type="month"],input[type="time"],input[type="week"],input[type="number"],input[type="email"],input[type="url"],input[type="search"],input[type="tel"],input[type="color"],.uneditable-input{display:inline-block;height:20px;padding:4px 6px;margin-bottom:9px;font-size:14px;line-height:20px;color:#555;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}input,textarea{width:210px}textarea{height:auto}textarea,input[type="text"],input[type="password"],input[type="datetime"],input[type="datetime-local"],input[type="date"],input[type="month"],input[type="time"],input[type="week"],input[type="number"],input[type="email"],input[type="url"],input[type="search"],input[type="tel"],input[type="color"]
,.uneditable-input{background-color:#fff;border:1px solid #ccc;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-webkit-transition:border linear .2s,box-shadow linear .2s;-moz-transition:border linear .2s,box-shadow linear .2s;-o-transition:border linear .2s,box-shadow linear .2s;transition:border linear .2s,box-shadow linear .2s}textarea:focus,input[type="text"]:focus,input[type="password"]:focus,input[type="datetime"]:focus,input[type="datetime-local"]:focus,input[type="date"]:focus,input[type="month"]:focus,input[type="time"]:focus,input[type="week"]:focus,input[type="number"]:focus,input[type="email"]:focus,input[type="url"]:focus,input[type="search"]:focus,input[type="tel"]:focus,input[type="color"]:focus,.uneditable-input:focus{border-color:rgba(82,168,236,0.8);outline:0;outline:thin dotted \9;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6);-moz-
box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6)}input[type="radio"],input[type="checkbox"]{margin:4px 0 0;margin-top:1px \9;*margin-top:0;line-height:normal;cursor:pointer}input[type="file"],input[type="image"],input[type="submit"],input[type="reset"],input[type="button"],input[type="radio"],input[type="checkbox"]{width:auto}select,input[type="file"]{height:30px;*margin-top:4px;line-height:30px}select{width:220px;background-color:#fff;border:1px solid #bbb}select[multiple],select[size]{height:auto}select:focus,input[type="file"]:focus,input[type="radio"]:focus,input[type="checkbox"]:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}.uneditable-input,.uneditable-textarea{color:#999;cursor:not-allowed;background-color:#fcfcfc;border-color:#ccc;-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.025);-moz-box-shadow:inset 0 1px 2px rgba(0,0,0,0.025)
;box-shadow:inset 0 1px 2px rgba(0,0,0,0.025)}.uneditable-input{overflow:hidden;white-space:nowrap}.uneditable-textarea{width:auto;height:auto}input:-moz-placeholder,textarea:-moz-placeholder{color:#999}input:-ms-input-placeholder,textarea:-ms-input-placeholder{color:#999}input::-webkit-input-placeholder,textarea::-webkit-input-placeholder{color:#999}.radio,.checkbox{min-height:18px;padding-left:18px}.radio input[type="radio"],.checkbox input[type="checkbox"]{float:left;margin-left:-18px}.controls>.radio:first-child,.controls>.checkbox:first-child{padding-top:5px}.radio.inline,.checkbox.inline{display:inline-block;padding-top:5px;margin-bottom:0;vertical-align:middle}.radio.inline+.radio.inline,.checkbox.inline+.checkbox.inline{margin-left:10px}.input-mini{width:60px}.input-small{width:90px}.input-medium{width:150px}.input-large{width:210px}.input-xlarge{width:270px}.input-xxlarge{width:530px}input[class*="span"],select[class*="span"],textarea[class*="span"],.uneditable-input[class*
="span"],.row-fluid input[class*="span"],.row-fluid select[class*="span"],.row-fluid textarea[class*="span"],.row-fluid .uneditable-input[class*="span"]{float:none;margin-left:0}.input-append input[class*="span"],.input-append .uneditable-input[class*="span"],.input-prepend input[class*="span"],.input-prepend .uneditable-input[class*="span"],.row-fluid input[class*="span"],.row-fluid select[class*="span"],.row-fluid textarea[class*="span"],.row-fluid .uneditable-input[class*="span"],.row-fluid .input-prepend [class*="span"],.row-fluid .input-append [class*="span"]{display:inline-block}input,textarea,.uneditable-input{margin-left:0}.controls-row [class*="span"]+[class*="span"]{margin-left:20px}input.span12,textarea.span12,.uneditable-input.span12{width:926px}input.span11,textarea.span11,.uneditable-input.span11{width:846px}input.span10,textarea.span10,.uneditable-input.span10{width:766px}input.span9,textarea.span9,.uneditable-input.span9{width:686px}input.span8,textarea.span8,.unedit
able-input.span8{width:606px}input.span7,textarea.span7,.uneditable-input.span7{width:526px}input.span6,textarea.span6,.uneditable-input.span6{width:446px}input.span5,textarea.span5,.uneditable-input.span5{width:366px}input.span4,textarea.span4,.uneditable-input.span4{width:286px}input.span3,textarea.span3,.uneditable-input.span3{width:206px}input.span2,textarea.span2,.uneditable-input.span2{width:126px}input.span1,textarea.span1,.uneditable-input.span1{width:46px}.controls-row{*zoom:1}.controls-row:before,.controls-row:after{display:table;line-height:0;content:""}.controls-row:after{clear:both}.controls-row [class*="span"]{float:left}input[disabled],select[disabled],textarea[disabled],input[readonly],select[readonly],textarea[readonly]{cursor:not-allowed;background-color:#eee}input[type="radio"][disabled],input[type="checkbox"][disabled],input[type="radio"][readonly],input[type="checkbox"][readonly]{background-color:transparent}.control-group.warning>label,.control-group.warning .h
elp-block,.control-group.warning .help-inline{color:#c09853}.control-group.warning .checkbox,.control-group.warning .radio,.control-group.warning input,.control-group.warning select,.control-group.warning textarea{color:#c09853;border-color:#c09853;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.warning .checkbox:focus,.control-group.warning .radio:focus,.control-group.warning input:focus,.control-group.warning select:focus,.control-group.warning textarea:focus{border-color:#a47e3c;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e}.control-group.warning .input-prepend .add-on,.control-group.warning .input-append .add-on{color:#c09853;background-color:#fcf8e3;border-color:#c09853}.control-group.error>label,.control-group.error
.help-block,.control-group.error .help-inline{color:#b94a48}.control-group.error .checkbox,.control-group.error .radio,.control-group.error input,.control-group.error select,.control-group.error textarea{color:#b94a48;border-color:#b94a48;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.error .checkbox:focus,.control-group.error .radio:focus,.control-group.error input:focus,.control-group.error select:focus,.control-group.error textarea:focus{border-color:#953b39;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392}.control-group.error .input-prepend .add-on,.control-group.error .input-append .add-on{color:#b94a48;background-color:#f2dede;border-color:#b94a48}.control-group.success>label,.control-group.success .help-block,.control
-group.success .help-inline{color:#468847}.control-group.success .checkbox,.control-group.success .radio,.control-group.success input,.control-group.success select,.control-group.success textarea{color:#468847;border-color:#468847;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.success .checkbox:focus,.control-group.success .radio:focus,.control-group.success input:focus,.control-group.success select:focus,.control-group.success textarea:focus{border-color:#356635;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b}.control-group.success .input-prepend .add-on,.control-group.success .input-append .add-on{color:#468847;background-color:#dff0d8;border-color:#468847}input:focus:required:invalid,textarea:focus:required:invalid,selec
t:focus:required:invalid{color:#b94a48;border-color:#ee5f5b}input:focus:required:invalid:focus,textarea:focus:required:invalid:focus,select:focus:required:invalid:focus{border-color:#e9322d;-webkit-box-shadow:0 0 6px #f8b9b7;-moz-box-shadow:0 0 6px #f8b9b7;box-shadow:0 0 6px #f8b9b7}.form-actions{padding:19px 20px 20px;margin-top:20px;margin-bottom:20px;background-color:#f5f5f5;border-top:1px solid #e5e5e5;*zoom:1}.form-actions:before,.form-actions:after{display:table;line-height:0;content:""}.form-actions:after{clear:both}.help-block,.help-inline{color:#595959}.help-block{display:block;margin-bottom:10px}.help-inline{display:inline-block;*display:inline;padding-left:5px;vertical-align:middle;*zoom:1}.input-append,.input-prepend{margin-bottom:5px;font-size:0;white-space:nowrap}.input-append input,.input-prepend input,.input-append select,.input-prepend select,.input-append .uneditable-input,.input-prepend .uneditable-input{position:relative;margin-bottom:0;*margin-left:0;font-size:1
4px;vertical-align:top;-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}.input-append input:focus,.input-prepend input:focus,.input-append select:focus,.input-prepend select:focus,.input-append .uneditable-input:focus,.input-prepend .uneditable-input:focus{z-index:2}.input-append .add-on,.input-prepend .add-on{display:inline-block;width:auto;height:20px;min-width:16px;padding:4px 5px;font-size:14px;font-weight:normal;line-height:20px;text-align:center;text-shadow:0 1px 0 #fff;background-color:#eee;border:1px solid #ccc}.input-append .add-on,.input-prepend .add-on,.input-append .btn,.input-prepend .btn{margin-left:-1px;vertical-align:top;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.input-append .active,.input-prepend .active{background-color:#a9dba9;border-color:#46a546}.input-prepend .add-on,.input-prepend .btn{margin-right:-1px}.input-prepend .add-on:first-child,.input-prepend .btn:first-child{-webkit-border-radius:3px 0 0
3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.input-append input,.input-append select,.input-append .uneditable-input{-webkit-border-radius:3px 0 0 3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.input-append .add-on:last-child,.input-append .btn:last-child{-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}.input-prepend.input-append input,.input-prepend.input-append select,.input-prepend.input-append .uneditable-input{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.input-prepend.input-append .add-on:first-child,.input-prepend.input-append .btn:first-child{margin-right:-1px;-webkit-border-radius:3px 0 0 3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.input-prepend.input-append .add-on:last-child,.input-prepend.input-append .btn:last-child{margin-left:-1px;-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}input.search-query{padding-right:14px;pad
ding-right:4px \9;padding-left:14px;padding-left:4px \9;margin-bottom:0;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.form-search .input-append .search-query,.form-search .input-prepend .search-query{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.form-search .input-append .search-query{-webkit-border-radius:14px 0 0 14px;-moz-border-radius:14px 0 0 14px;border-radius:14px 0 0 14px}.form-search .input-append .btn{-webkit-border-radius:0 14px 14px 0;-moz-border-radius:0 14px 14px 0;border-radius:0 14px 14px 0}.form-search .input-prepend .search-query{-webkit-border-radius:0 14px 14px 0;-moz-border-radius:0 14px 14px 0;border-radius:0 14px 14px 0}.form-search .input-prepend .btn{-webkit-border-radius:14px 0 0 14px;-moz-border-radius:14px 0 0 14px;border-radius:14px 0 0 14px}.form-search input,.form-inline input,.form-horizontal input,.form-search textarea,.form-inline textarea,.form-horizontal textarea,.form-search select,.form-inline select,.for
m-horizontal select,.form-search .help-inline,.form-inline .help-inline,.form-horizontal .help-inline,.form-search .uneditable-input,.form-inline .uneditable-input,.form-horizontal .uneditable-input,.form-search .input-prepend,.form-inline .input-prepend,.form-horizontal .input-prepend,.form-search .input-append,.form-inline .input-append,.form-horizontal .input-append{display:inline-block;*display:inline;margin-bottom:0;vertical-align:middle;*zoom:1}.form-search .hide,.form-inline .hide,.form-horizontal .hide{display:none}.form-search label,.form-inline label,.form-search .btn-group,.form-inline .btn-group{display:inline-block}.form-search .input-append,.form-inline .input-append,.form-search .input-prepend,.form-inline .input-prepend{margin-bottom:0}.form-search .radio,.form-search .checkbox,.form-inline .radio,.form-inline .checkbox{padding-left:0;margin-bottom:0;vertical-align:middle}.form-search .radio input[type="radio"],.form-search .checkbox input[type="checkbox"],.form-inli
ne .radio input[type="radio"],.form-inline .checkbox input[type="checkbox"]{float:left;margin-right:3px;margin-left:0}.control-group{margin-bottom:10px}legend+.control-group{margin-top:20px;-webkit-margin-top-collapse:separate}.form-horizontal .control-group{margin-bottom:20px;*zoom:1}.form-horizontal .control-group:before,.form-horizontal .control-group:after{display:table;line-height:0;content:""}.form-horizontal .control-group:after{clear:both}.form-horizontal .control-label{float:left;width:140px;padding-top:5px;text-align:right}.form-horizontal .controls{*display:inline-block;*padding-left:20px;margin-left:160px;*margin-left:0}.form-horizontal .controls:first-child{*padding-left:160px}.form-horizontal .help-block{margin-top:10px;margin-bottom:0}.form-horizontal .form-actions{padding-left:160px}table{max-width:100%;background-color:transparent;border-collapse:collapse;border-spacing:0}.table{width:100%;margin-bottom:20px}.table th,.table td{padding:8px;line-height:20px;text-alig
n:left;vertical-align:top;border-top:1px solid #ddd}.table th{font-weight:bold}.table thead th{vertical-align:bottom}.table caption+thead tr:first-child th,.table caption+thead tr:first-child td,.table colgroup+thead tr:first-child th,.table colgroup+thead tr:first-child td,.table thead:first-child tr:first-child th,.table thead:first-child tr:first-child td{border-top:0}.table tbody+tbody{border-top:2px solid #ddd}.table-condensed th,.table-condensed td{padding:4px 5px}.table-bordered{border:1px solid #ddd;border-collapse:separate;*border-collapse:collapse;border-left:0;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.table-bordered th,.table-bordered td{border-left:1px solid #ddd}.table-bordered caption+thead tr:first-child th,.table-bordered caption+tbody tr:first-child th,.table-bordered caption+tbody tr:first-child td,.table-bordered colgroup+thead tr:first-child th,.table-bordered colgroup+tbody tr:first-child th,.table-bordered colgroup+tbody tr:first-child
td,.table-bordered thead:first-child tr:first-child th,.table-bordered tbody:first-child tr:first-child th,.table-bordered tbody:first-child tr:first-child td{border-top:0}.table-bordered thead:first-child tr:first-child th:first-child,.table-bordered tbody:first-child tr:first-child td:first-child{-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-topleft:4px}.table-bordered thead:first-child tr:first-child th:last-child,.table-bordered tbody:first-child tr:first-child td:last-child{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-moz-border-radius-topright:4px}.table-bordered thead:last-child tr:last-child th:first-child,.table-bordered tbody:last-child tr:last-child td:first-child,.table-bordered tfoot:last-child tr:last-child td:first-child{-webkit-border-radius:0 0 0 4px;-moz-border-radius:0 0 0 4px;border-radius:0 0 0 4px;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-moz-border-radius-bottomleft:4px}.table-b
ordered thead:last-child tr:last-child th:last-child,.table-bordered tbody:last-child tr:last-child td:last-child,.table-bordered tfoot:last-child tr:last-child td:last-child{-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-moz-border-radius-bottomright:4px}.table-bordered caption+thead tr:first-child th:first-child,.table-bordered caption+tbody tr:first-child td:first-child,.table-bordered colgroup+thead tr:first-child th:first-child,.table-bordered colgroup+tbody tr:first-child td:first-child{-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-topleft:4px}.table-bordered caption+thead tr:first-child th:last-child,.table-bordered caption+tbody tr:first-child td:last-child,.table-bordered colgroup+thead tr:first-child th:last-child,.table-bordered colgroup+tbody tr:first-child td:last-child{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-moz-border-right-topleft:4px}.table-striped tbody tr:nth-child(odd) td,.table-s
triped tbody tr:nth-child(odd) th{background-color:#f9f9f9}.table-hover tbody tr:hover td,.table-hover tbody tr:hover th{background-color:#f5f5f5}table [class*=span],.row-fluid table [class*=span]{display:table-cell;float:none;margin-left:0}table .span1{float:none;width:44px;margin-left:0}table .span2{float:none;width:124px;margin-left:0}table .span3{float:none;width:204px;margin-left:0}table .span4{float:none;width:284px;margin-left:0}table .span5{float:none;width:364px;margin-left:0}table .span6{float:none;width:444px;margin-left:0}table .span7{float:none;width:524px;margin-left:0}table .span8{float:none;width:604px;margin-left:0}table .span9{float:none;width:684px;margin-left:0}table .span10{float:none;width:764px;margin-left:0}table .span11{float:none;width:844px;margin-left:0}table .span12{float:none;width:924px;margin-left:0}table .span13{float:none;width:1004px;margin-left:0}table .span14{float:none;width:1084px;margin-left:0}table .span15{float:none;width:1164px;margin-left:
0}table .span16{float:none;width:1244px;margin-left:0}table .span17{float:none;width:1324px;margin-left:0}table .span18{float:none;width:1404px;margin-left:0}table .span19{float:none;width:1484px;margin-left:0}table .span20{float:none;width:1564px;margin-left:0}table .span21{float:none;width:1644px;margin-left:0}table .span22{float:none;width:1724px;margin-left:0}table .span23{float:none;width:1804px;margin-left:0}table .span24{float:none;width:1884px;margin-left:0}.table tbody tr.success td{background-color:#dff0d8}.table tbody tr.error td{background-color:#f2dede}.table tbody tr.info td{background-color:#d9edf7}[class^="icon-"],[class*=" icon-"]{display:inline-block;width:14px;height:14px;margin-top:1px;*margin-right:.3em;line-height:14px;vertical-align:text-top;background-image:url("../img/glyphicons-halflings.png");background-position:14px 14px;background-repeat:no-repeat}.icon-white,.nav>.active>a>[class^="icon-"],.nav>.active>a>[class*=" icon-"],.dropdown-menu>li>a:hover>[clas
s^="icon-"],.dropdown-menu>li>a:hover>[class*=" icon-"],.dropdown-menu>.active>a>[class^="icon-"],.dropdown-menu>.active>a>[class*=" icon-"]{background-image:url("../img/glyphicons-halflings-white.png")}.icon-glass{background-position:0 0}.icon-music{background-position:-24px 0}.icon-search{background-position:-48px 0}.icon-envelope{background-position:-72px 0}.icon-heart{background-position:-96px 0}.icon-star{background-position:-120px 0}.icon-star-empty{background-position:-144px 0}.icon-user{background-position:-168px 0}.icon-film{background-position:-192px 0}.icon-th-large{background-position:-216px 0}.icon-th{background-position:-240px 0}.icon-th-list{background-position:-264px 0}.icon-ok{background-position:-288px 0}.icon-remove{background-position:-312px 0}.icon-zoom-in{background-position:-336px 0}.icon-zoom-out{background-position:-360px 0}.icon-off{background-position:-384px 0}.icon-signal{background-position:-408px 0}.icon-cog{background-position:-432px 0}.icon-trash{back
ground-position:-456px 0}.icon-home{background-position:0 -24px}.icon-file{background-position:-24px -24px}.icon-time{background-position:-48px -24px}.icon-road{background-position:-72px -24px}.icon-download-alt{background-position:-96px -24px}.icon-download{background-position:-120px -24px}.icon-upload{background-position:-144px -24px}.icon-inbox{background-position:-168px -24px}.icon-play-circle{background-position:-192px -24px}.icon-repeat{background-position:-216px -24px}.icon-refresh{background-position:-240px -24px}.icon-list-alt{background-position:-264px -24px}.icon-lock{background-position:-287px -24px}.icon-flag{background-position:-312px -24px}.icon-headphones{background-position:-336px -24px}.icon-volume-off{background-position:-360px -24px}.icon-volume-down{background-position:-384px -24px}.icon-volume-up{background-position:-408px -24px}.icon-qrcode{background-position:-432px -24px}.icon-barcode{background-position:-456px -24px}.icon-tag{background-position:0 -48px}.ic
on-tags{background-position:-25px -48px}.icon-book{background-position:-48px -48px}.icon-bookmark{background-position:-72px -48px}.icon-print{background-position:-96px -48px}.icon-camera{background-position:-120px -48px}.icon-font{background-position:-144px -48px}.icon-bold{background-position:-167px -48px}.icon-italic{background-position:-192px -48px}.icon-text-height{background-position:-216px -48px}.icon-text-width{background-position:-240px -48px}.icon-align-left{background-position:-264px -48px}.icon-align-center{background-position:-288px -48px}.icon-align-right{background-position:-312px -48px}.icon-align-justify{background-position:-336px -48px}.icon-list{background-position:-360px -48px}.icon-indent-left{background-position:-384px -48px}.icon-indent-right{background-position:-408px -48px}.icon-facetime-video{background-position:-432px -48px}.icon-picture{background-position:-456px -48px}.icon-pencil{background-position:0 -72px}.icon-map-marker{background-position:-24px -72p
x}.icon-adjust{background-position:-48px -72px}.icon-tint{background-position:-72px -72px}.icon-edit{background-position:-96px -72px}.icon-share{background-position:-120px -72px}.icon-check{background-position:-144px -72px}.icon-move{background-position:-168px -72px}.icon-step-backward{background-position:-192px -72px}.icon-fast-backward{background-position:-216px -72px}.icon-backward{background-position:-240px -72px}.icon-play{background-position:-264px -72px}.icon-pause{background-position:-288px -72px}.icon-stop{background-position:-312px -72px}.icon-forward{background-position:-336px -72px}.icon-fast-forward{background-position:-360px -72px}.icon-step-forward{background-position:-384px -72px}.icon-eject{background-position:-408px -72px}.icon-chevron-left{background-position:-432px -72px}.icon-chevron-right{background-position:-456px -72px}.icon-plus-sign{background-position:0 -96px}.icon-minus-sign{background-position:-24px -96px}.icon-remove-sign{background-position:-48px -96px
}.icon-ok-sign{background-position:-72px -96px}.icon-question-sign{background-position:-96px -96px}.icon-info-sign{background-position:-120px -96px}.icon-screenshot{background-position:-144px -96px}.icon-remove-circle{background-position:-168px -96px}.icon-ok-circle{background-position:-192px -96px}.icon-ban-circle{background-position:-216px -96px}.icon-arrow-left{background-position:-240px -96px}.icon-arrow-right{background-position:-264px -96px}.icon-arrow-up{background-position:-289px -96px}.icon-arrow-down{background-position:-312px -96px}.icon-share-alt{background-position:-336px -96px}.icon-resize-full{background-position:-360px -96px}.icon-resize-small{background-position:-384px -96px}.icon-plus{background-position:-408px -96px}.icon-minus{background-position:-433px -96px}.icon-asterisk{background-position:-456px -96px}.icon-exclamation-sign{background-position:0 -120px}.icon-gift{background-position:-24px -120px}.icon-leaf{background-position:-48px -120px}.icon-fire{backgrou
nd-position:-72px -120px}.icon-eye-open{background-position:-96px -120px}.icon-eye-close{background-position:-120px -120px}.icon-warning-sign{background-position:-144px -120px}.icon-plane{background-position:-168px -120px}.icon-calendar{background-position:-192px -120px}.icon-random{width:16px;background-position:-216px -120px}.icon-comment{background-position:-240px -120px}.icon-magnet{background-position:-264px -120px}.icon-chevron-up{background-position:-288px -120px}.icon-chevron-down{background-position:-313px -119px}.icon-retweet{background-position:-336px -120px}.icon-shopping-cart{background-position:-360px -120px}.icon-folder-close{background-position:-384px -120px}.icon-folder-open{width:16px;background-position:-408px -120px}.icon-resize-vertical{background-position:-432px -119px}.icon-resize-horizontal{background-position:-456px -118px}.icon-hdd{background-position:0 -144px}.icon-bullhorn{background-position:-24px -144px}.icon-bell{background-position:-48px -144px}.icon-
certificate{background-position:-72px -144px}.icon-thumbs-up{background-position:-96px -144px}.icon-thumbs-down{background-position:-120px -144px}.icon-hand-right{background-position:-144px -144px}.icon-hand-left{background-position:-168px -144px}.icon-hand-up{background-position:-192px -144px}.icon-hand-down{background-position:-216px -144px}.icon-circle-arrow-right{background-position:-240px -144px}.icon-circle-arrow-left{background-position:-264px -144px}.icon-circle-arrow-up{background-position:-288px -144px}.icon-circle-arrow-down{background-position:-312px -144px}.icon-globe{background-position:-336px -144px}.icon-wrench{background-position:-360px -144px}.icon-tasks{background-position:-384px -144px}.icon-filter{background-position:-408px -144px}.icon-briefcase{background-position:-432px -144px}.icon-fullscreen{background-position:-456px -144px}.dropup,.dropdown{position:relative}.dropdown-toggle{*margin-bottom:-3px}.dropdown-toggle:active,.open .dropdown-toggle{outline:0}.car
et{display:inline-block;width:0;height:0;vertical-align:top;border-top:4px solid #000;border-right:4px solid transparent;border-left:4px solid transparent;content:""}.dropdown .caret{margin-top:8px;margin-left:2px}.dropdown-menu{position:absolute;top:100%;left:0;z-index:1000;display:none;float:left;min-width:160px;padding:5px 0;margin:2px 0 0;list-style:none;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);*border-right-width:2px;*border-bottom-width:2px;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 5px 10px rgba(0,0,0,0.2);-moz-box-shadow:0 5px 10px rgba(0,0,0,0.2);box-shadow:0 5px 10px rgba(0,0,0,0.2);-webkit-background-clip:padding-box;-moz-background-clip:padding;background-clip:padding-box}.dropdown-menu.pull-right{right:0;left:auto}.dropdown-menu .divider{*width:100%;height:1px;margin:9px 1px;*margin:-5px 0 5px;overflow:hidden;background-color:#e5e5e5;border-bottom:1px solid #fff}.dropdown-menu a{display:blo
ck;padding:3px 20px;clear:both;font-weight:normal;line-height:20px;color:#333;white-space:nowrap}.dropdown-menu li>a:hover,.dropdown-menu li>a:focus,.dropdown-submenu:hover>a{color:#fff;text-decoration:none;background-color:#08c;background-color:#0081c2;background-image:-moz-linear-gradient(top,#08c,#0077b3);background-image:-webkit-gradient(linear,0 0,0 100%,from(#08c),to(#0077b3));background-image:-webkit-linear-gradient(top,#08c,#0077b3);background-image:-o-linear-gradient(top,#08c,#0077b3);background-image:linear-gradient(to bottom,#08c,#0077b3);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0077b3',GradientType=0)}.dropdown-menu .active>a,.dropdown-menu .active>a:hover{color:#fff;text-decoration:none;background-color:#08c;background-color:#0081c2;background-image:linear-gradient(to bottom,#08c,#0077b3);background-image:-moz-linear-gradient(top,#08c,#0077b3);background-image:-webkit-gradient(linear,0 0,0 100
%,from(#08c),to(#0077b3));background-image:-webkit-linear-gradient(top,#08c,#0077b3);background-image:-o-linear-gradient(top,#08c,#0077b3);background-repeat:repeat-x;outline:0;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0077b3',GradientType=0)}.dropdown-menu .disabled>a,.dropdown-menu .disabled>a:hover{color:#999}.dropdown-menu .disabled>a:hover{text-decoration:none;cursor:default;background-color:transparent}.open{*z-index:1000}.open>.dropdown-menu{display:block}.pull-right>.dropdown-menu{right:0;left:auto}.dropup .caret,.navbar-fixed-bottom .dropdown .caret{border-top:0;border-bottom:4px solid #000;content:"\2191"}.dropup .dropdown-menu,.navbar-fixed-bottom .dropdown .dropdown-menu{top:auto;bottom:100%;margin-bottom:1px}.dropdown-submenu{position:relative}.dropdown-submenu>.dropdown-menu{top:0;left:100%;margin-top:-6px;margin-left:-1px;-webkit-border-radius:0 6px 6px 6px;-moz-border-radius:0 6px 6px 6px;border-radius:0 6px 6px 6px}.d
ropdown-submenu:hover .dropdown-menu{display:block}.dropdown-submenu>a:after{display:block;float:right;width:0;height:0;margin-top:5px;margin-right:-10px;border-color:transparent;border-left-color:#ccc;border-style:solid;border-width:5px 0 5px 5px;content:" "}.dropdown-submenu:hover>a:after{border-left-color:#fff}.dropdown .dropdown-menu .nav-header{padding-right:20px;padding-left:20px}.typeahead{margin-top:2px;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.well{min-height:20px;padding:19px;margin-bottom:20px;background-color:#f5f5f5;border:1px solid #e3e3e3;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.05);box-shadow:inset 0 1px 1px rgba(0,0,0,0.05)}.well blockquote{border-color:#ddd;border-color:rgba(0,0,0,0.15)}.well-large{padding:24px;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.well-small{padding:9px;-webkit-border-rad
ius:3px;-moz-border-radius:3px;border-radius:3px}.fade{opacity:0;-webkit-transition:opacity .15s linear;-moz-transition:opacity .15s linear;-o-transition:opacity .15s linear;transition:opacity .15s linear}.fade.in{opacity:1}.collapse{position:relative;height:0;overflow:hidden;overflow:visible \9;-webkit-transition:height .35s ease;-moz-transition:height .35s ease;-o-transition:height .35s ease;transition:height .35s ease}.collapse.in{height:auto}.close{float:right;font-size:20px;font-weight:bold;line-height:20px;color:#000;text-shadow:0 1px 0 #fff;opacity:.2;filter:alpha(opacity=20)}.close:hover{color:#000;text-decoration:none;cursor:pointer;opacity:.4;filter:alpha(opacity=40)}button.close{padding:0;cursor:pointer;background:transparent;border:0;-webkit-appearance:none}.btn{display:inline-block;*display:inline;padding:4px 14px;margin-bottom:0;*margin-left:.3em;font-size:14px;line-height:20px;*line-height:20px;color:#333;text-align:center;text-shadow:0 1px 1px rgba(255,255,255,0.75);
vertical-align:middle;cursor:pointer;background-color:#f5f5f5;*background-color:#e6e6e6;background-image:-webkit-gradient(linear,0 0,0 100%,from(#fff),to(#e6e6e6));background-image:-webkit-linear-gradient(top,#fff,#e6e6e6);background-image:-o-linear-gradient(top,#fff,#e6e6e6);background-image:linear-gradient(to bottom,#fff,#e6e6e6);background-image:-moz-linear-gradient(top,#fff,#e6e6e6);background-repeat:repeat-x;border:1px solid #bbb;*border:0;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);border-color:#e6e6e6 #e6e6e6 #bfbfbf;border-bottom-color:#a2a2a2;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffffffff',endColorstr='#ffe6e6e6',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false);*zoom:1;-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);box-
shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05)}.btn:hover,.btn:active,.btn.active,.btn.disabled,.btn[disabled]{color:#333;background-color:#e6e6e6;*background-color:#d9d9d9}.btn:active,.btn.active{background-color:#ccc \9}.btn:first-child{*margin-left:0}.btn:hover{color:#333;text-decoration:none;background-color:#e6e6e6;*background-color:#d9d9d9;background-position:0 -15px;-webkit-transition:background-position .1s linear;-moz-transition:background-position .1s linear;-o-transition:background-position .1s linear;transition:background-position .1s linear}.btn:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}.btn.active,.btn:active{background-color:#e6e6e6;background-color:#d9d9d9 \9;background-image:none;outline:0;-webkit-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2p
x rgba(0,0,0,0.05)}.btn.disabled,.btn[disabled]{cursor:default;background-color:#e6e6e6;background-image:none;opacity:.65;filter:alpha(opacity=65);-webkit-box-shadow:none;-moz-box-shadow:none;box-shadow:none}.btn-large{padding:9px 14px;font-size:16px;line-height:normal;-webkit-border-radius:5px;-moz-border-radius:5px;border-radius:5px}.btn-large [class^="icon-"]{margin-top:2px}.btn-small{padding:3px 9px;font-size:12px;line-height:18px}.btn-small [class^="icon-"]{margin-top:0}.btn-mini{padding:2px 6px;font-size:11px;line-height:16px}.btn-block{display:block;width:100%;padding-right:0;padding-left:0;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.btn-block+.btn-block{margin-top:5px}.btn-primary.active,.btn-warning.active,.btn-danger.active,.btn-success.active,.btn-info.active,.btn-inverse.active{color:rgba(255,255,255,0.75)}.btn{border-color:#c5c5c5;border-color:rgba(0,0,0,0.15) rgba(0,0,0,0.15) rgba(0,0,0,0.25)}.btn-primary{color:#fff;text-shadow:0 -1p
x 0 rgba(0,0,0,0.25);background-color:#006dcc;*background-color:#04c;background-image:-webkit-gradient(linear,0 0,0 100%,from(#08c),to(#04c));background-image:-webkit-linear-gradient(top,#08c,#04c);background-image:-o-linear-gradient(top,#08c,#04c);background-image:linear-gradient(to bottom,#08c,#04c);background-image:-moz-linear-gradient(top,#08c,#04c);background-repeat:repeat-x;border-color:#04c #0044cc #002a80;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0044cc',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-primary:hover,.btn-primary:active,.btn-primary.active,.btn-primary.disabled,.btn-primary[disabled]{color:#fff;background-color:#04c;*background-color:#003bb3}.btn-primary:active,.btn-primary.active{background-color:#039 \9}.btn-warning{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#faa732;*background-color:#f89
406;background-image:-webkit-gradient(linear,0 0,0 100%,from(#fbb450),to(#f89406));background-image:-webkit-linear-gradient(top,#fbb450,#f89406);background-image:-o-linear-gradient(top,#fbb450,#f89406);background-image:linear-gradient(to bottom,#fbb450,#f89406);background-image:-moz-linear-gradient(top,#fbb450,#f89406);background-repeat:repeat-x;border-color:#f89406 #f89406 #ad6704;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#fffbb450',endColorstr='#fff89406',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-warning:hover,.btn-warning:active,.btn-warning.active,.btn-warning.disabled,.btn-warning[disabled]{color:#fff;background-color:#f89406;*background-color:#df8505}.btn-warning:active,.btn-warning.active{background-color:#c67605 \9}.btn-danger{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#da4f49;*background-color:#bd362f;background-image:-webki
t-gradient(linear,0 0,0 100%,from(#ee5f5b),to(#bd362f));background-image:-webkit-linear-gradient(top,#ee5f5b,#bd362f);background-image:-o-linear-gradient(top,#ee5f5b,#bd362f);background-image:linear-gradient(to bottom,#ee5f5b,#bd362f);background-image:-moz-linear-gradient(top,#ee5f5b,#bd362f);background-repeat:repeat-x;border-color:#bd362f #bd362f #802420;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffee5f5b',endColorstr='#ffbd362f',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-danger:hover,.btn-danger:active,.btn-danger.active,.btn-danger.disabled,.btn-danger[disabled]{color:#fff;background-color:#bd362f;*background-color:#a9302a}.btn-danger:active,.btn-danger.active{background-color:#942a25 \9}.btn-success{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#5bb75b;*background-color:#51a351;background-image:-webkit-gradient(linear,0 0,0 100%,from
(#62c462),to(#51a351));background-image:-webkit-linear-gradient(top,#62c462,#51a351);background-image:-o-linear-gradient(top,#62c462,#51a351);background-image:linear-gradient(to bottom,#62c462,#51a351);background-image:-moz-linear-gradient(top,#62c462,#51a351);background-repeat:repeat-x;border-color:#51a351 #51a351 #387038;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff62c462',endColorstr='#ff51a351',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-success:hover,.btn-success:active,.btn-success.active,.btn-success.disabled,.btn-success[disabled]{color:#fff;background-color:#51a351;*background-color:#499249}.btn-success:active,.btn-success.active{background-color:#408140 \9}.btn-info{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#49afcd;*background-color:#2f96b4;background-image:-webkit-gradient(linear,0 0,0 100%,from(#5bc0de),to(#2f96b4));backgr
ound-image:-webkit-linear-gradient(top,#5bc0de,#2f96b4);background-image:-o-linear-gradient(top,#5bc0de,#2f96b4);background-image:linear-gradient(to bottom,#5bc0de,#2f96b4);background-image:-moz-linear-gradient(top,#5bc0de,#2f96b4);background-repeat:repeat-x;border-color:#2f96b4 #2f96b4 #1f6377;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff5bc0de',endColorstr='#ff2f96b4',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-info:hover,.btn-info:active,.btn-info.active,.btn-info.disabled,.btn-info[disabled]{color:#fff;background-color:#2f96b4;*background-color:#2a85a0}.btn-info:active,.btn-info.active{background-color:#24748c \9}.btn-inverse{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#363636;*background-color:#222;background-image:-webkit-gradient(linear,0 0,0 100%,from(#444),to(#222));background-image:-webkit-linear-gradient(top,#444,#222);backgr
ound-image:-o-linear-gradient(top,#444,#222);background-image:linear-gradient(to bottom,#444,#222);background-image:-moz-linear-gradient(top,#444,#222);background-repeat:repeat-x;border-color:#222 #222222 #000;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff444444',endColorstr='#ff222222',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-inverse:hover,.btn-inverse:active,.btn-inverse.active,.btn-inverse.disabled,.btn-inverse[disabled]{color:#fff;background-color:#222;*background-color:#151515}.btn-inverse:active,.btn-inverse.active{background-color:#080808 \9}button.btn,input[type="submit"].btn{*padding-top:3px;*padding-bottom:3px}button.btn::-moz-focus-inner,input[type="submit"].btn::-moz-focus-inner{padding:0;border:0}button.btn.btn-large,input[type="submit"].btn.btn-large{*padding-top:7px;*padding-bottom:7px}button.btn.btn-small,input[type="submit"].btn.btn-small{
*padding-top:3px;*padding-bottom:3px}button.btn.btn-mini,input[type="submit"].btn.btn-mini{*padding-top:1px;*padding-bottom:1px}.btn-link,.btn-link:active{background-color:transparent;background-image:none;-webkit-box-shadow:none;-moz-box-shadow:none;box-shadow:none}.btn-link{color:#08c;cursor:pointer;border-color:transparent;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-link:hover{color:#005580;text-decoration:underline;background-color:transparent}.btn-group{position:relative;*margin-left:.3em;font-size:0;white-space:nowrap}.btn-group:first-child{*margin-left:0}.btn-group+.btn-group{margin-left:5px}.btn-toolbar{margin-top:10px;margin-bottom:10px;font-size:0}.btn-toolbar .btn-group{display:inline-block;*display:inline;*zoom:1}.btn-toolbar .btn+.btn,.btn-toolbar .btn-group+.btn,.btn-toolbar .btn+.btn-group{margin-left:5px}.btn-group>.btn{position:relative;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-group>.btn+.btn{margin-left:-1px}.btn-group
>.btn,.btn-group>.dropdown-menu{font-size:14px}.btn-group>.btn-mini{font-size:11px}.btn-group>.btn-small{font-size:12px}.btn-group>.btn-large{font-size:16px}.btn-group>.btn:first-child{margin-left:0;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-bottomleft:4px;-moz-border-radius-topleft:4px}.btn-group>.btn:last-child,.btn-group>.dropdown-toggle{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-moz-border-radius-topright:4px;-moz-border-radius-bottomright:4px}.btn-group>.btn.large:first-child{margin-left:0;-webkit-border-bottom-left-radius:6px;border-bottom-left-radius:6px;-webkit-border-top-left-radius:6px;border-top-left-radius:6px;-moz-border-radius-bottomleft:6px;-moz-border-radius-topleft:6px}.btn-group>.btn.large:last-child,.btn-group>.large.dropdown-toggle{-webkit-border-top-right-radius:6px;b
order-top-right-radius:6px;-webkit-border-bottom-right-radius:6px;border-bottom-right-radius:6px;-moz-border-radius-topright:6px;-moz-border-radius-bottomright:6px}.btn-group>.btn:hover,.btn-group>.btn:focus,.btn-group>.btn:active,.btn-group>.btn.active{z-index:2}.btn-group .dropdown-toggle:active,.btn-group.open .dropdown-toggle{outline:0}.btn-group>.btn+.dropdown-toggle{*padding-top:5px;padding-right:8px;*padding-bottom:5px;padding-left:8px;-webkit-box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05)}.btn-group>.btn-mini+.dropdown-toggle{*padding-top:2px;padding-right:5px;*padding-bottom:2px;padding-left:5px}.btn-group>.btn-small+.dropdown-toggle{*padding-top:5px;*padding-bottom:4px}.btn-group>.btn
-large+.dropdown-toggle{*padding-top:7px;padding-right:12px;*padding-bottom:7px;padding-left:12px}.btn-group.open .dropdown-toggle{background-image:none;-webkit-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05)}.btn-group.open .btn.dropdown-toggle{background-color:#e6e6e6}.btn-group.open .btn-primary.dropdown-toggle{background-color:#04c}.btn-group.open .btn-warning.dropdown-toggle{background-color:#f89406}.btn-group.open .btn-danger.dropdown-toggle{background-color:#bd362f}.btn-group.open .btn-success.dropdown-toggle{background-color:#51a351}.btn-group.open .btn-info.dropdown-toggle{background-color:#2f96b4}.btn-group.open .btn-inverse.dropdown-toggle{background-color:#222}.btn .caret{margin-top:8px;margin-left:0}.btn-mini .caret,.btn-small .caret,.btn-large .caret{margin-top:6px}.btn-large .caret{border-top-width:5p
x;border-right-width:5px;border-left-width:5px}.dropup .btn-large .caret{border-top:0;border-bottom:5px solid #000}.btn-primary .caret,.btn-warning .caret,.btn-danger .caret,.btn-info .caret,.btn-success .caret,.btn-inverse .caret{border-top-color:#fff;border-bottom-color:#fff}.btn-group-vertical{display:inline-block;*display:inline;*zoom:1}.btn-group-vertical .btn{display:block;float:none;width:100%;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-group-vertical .btn+.btn{margin-top:-1px;margin-left:0}.btn-group-vertical .btn:first-child{-webkit-border-radius:4px 4px 0 0;-moz-border-radius:4px 4px 0 0;border-radius:4px 4px 0 0}.btn-group-vertical .btn:last-child{-webkit-border-radius:0 0 4px 4px;-moz-border-radius:0 0 4px 4px;border-radius:0 0 4px 4px}.btn-group-vertical .btn-large:first-child{-webkit-border-radius:6px 6px 0 0;-moz-border-radius:6px 6px 0 0;border-radius:6px 6px 0 0}.btn-group-vertical .btn-large:last-child{-webkit-border-radius:0 0 6px 6px;-moz-bo
rder-radius:0 0 6px 6px;border-radius:0 0 6px 6px}.alert{padding:8px 35px 8px 14px;margin-bottom:20px;color:#c09853;text-shadow:0 1px 0 rgba(255,255,255,0.5);background-color:#fcf8e3;border:1px solid #fbeed5;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.alert h4{margin:0}.alert .close{position:relative;top:-2px;right:-21px;line-height:20px}.alert-success{color:#468847;background-color:#dff0d8;border-color:#d6e9c6}.alert-danger,.alert-error{color:#b94a48;background-color:#f2dede;border-color:#eed3d7}.alert-info{color:#3a87ad;background-color:#d9edf7;border-color:#bce8f1}.alert-block{padding-top:14px;padding-bottom:14px}.alert-block>p,.alert-block>ul{margin-bottom:0}.alert-block p+p{margin-top:5px}.nav{margin-bottom:20px;margin-left:0;list-style:none}.nav>li>a{display:block}.nav>li>a:hover{text-decoration:none;background-color:#eee}.nav>.pull-right{float:right}.nav-header{display:block;padding:3px 15px;font-size:11px;font-weight:bold;line-height:20px;color:#999;t
ext-shadow:0 1px 0 rgba(255,255,255,0.5);text-transform:uppercase}.nav li+.nav-header{margin-top:9px}.nav-list{padding-right:15px;padding-left:15px;margin-bottom:0}.nav-list>li>a,.nav-list .nav-header{margin-right:-15px;margin-left:-15px;text-shadow:0 1px 0 rgba(255,255,255,0.5)}.nav-list>li>a{padding:3px 15px}.nav-list>.active>a,.nav-list>.active>a:hover{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.2);background-color:#08c}.nav-list [class^="icon-"]{margin-right:2px}.nav-list .divider{*width:100%;height:1px;margin:9px 1px;*margin:-5px 0 5px;overflow:hidden;background-color:#e5e5e5;border-bottom:1px solid #fff}.nav-tabs,.nav-pills{*zoom:1}.nav-tabs:before,.nav-pills:before,.nav-tabs:after,.nav-pills:after{display:table;line-height:0;content:""}.nav-tabs:after,.nav-pills:after{clear:both}.nav-tabs>li,.nav-pills>li{float:left}.nav-tabs>li>a,.nav-pills>li>a{padding-right:12px;padding-left:12px;margin-right:2px;line-height:14px}.nav-tabs{border-bottom:1px solid #ddd}.nav-tabs>li{margin-
bottom:-1px}.nav-tabs>li>a{padding-top:8px;padding-bottom:8px;line-height:20px;border:1px solid transparent;-webkit-border-radius:4px 4px 0 0;-moz-border-radius:4px 4px 0 0;border-radius:4px 4px 0 0}.nav-tabs>li>a:hover{border-color:#eee #eeeeee #ddd}.nav-tabs>.active>a,.nav-tabs>.active>a:hover{color:#555;cursor:default;background-color:#fff;border:1px solid #ddd;border-bottom-color:transparent}.nav-pills>li>a{padding-top:8px;padding-bottom:8px;margin-top:2px;margin-bottom:2px;-webkit-border-radius:5px;-moz-border-radius:5px;border-radius:5px}.nav-pills>.active>a,.nav-pills>.active>a:hover{color:#fff;background-color:#08c}.nav-stacked>li{float:none}.nav-stacked>li>a{margin-right:0}.nav-tabs.nav-stacked{border-bottom:0}.nav-tabs.nav-stacked>li>a{border:1px solid #ddd;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.nav-tabs.nav-stacked>li:first-child>a{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-webkit-border-top-left-radius:4px;border-top-left-radi
us:4px;-moz-border-radius-topright:4px;-moz-border-radius-topleft:4px}.nav-tabs.nav-stacked>li:last-child>a{-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-moz-border-radius-bottomright:4px;-moz-border-radius-bottomleft:4px}.nav-tabs.nav-stacked>li>a:hover{z-index:2;border-color:#ddd}.nav-pills.nav-stacked>li>a{margin-bottom:3px}.nav-pills.nav-stacked>li:last-child>a{margin-bottom:1px}.nav-tabs .dropdown-menu{-webkit-border-radius:0 0 6px 6px;-moz-border-radius:0 0 6px 6px;border-radius:0 0 6px 6px}.nav-pills .dropdown-menu{-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.nav .dropdown-toggle .caret{margin-top:6px;border-top-color:#08c;border-bottom-color:#08c}.nav .dropdown-toggle:hover .caret{border-top-color:#005580;border-bottom-color:#005580}.nav-tabs .dropdown-toggle .caret{margin-top:8px}.nav .active .dropdown-toggle .caret{border-top-color:#fff;border-bottom-color:#f
ff}.nav-tabs .active .dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.nav>.dropdown.active>a:hover{cursor:pointer}.nav-tabs .open .dropdown-toggle,.nav-pills .open .dropdown-toggle,.nav>li.dropdown.open.active>a:hover{color:#fff;background-color:#999;border-color:#999}.nav li.dropdown.open .caret,.nav li.dropdown.open.active .caret,.nav li.dropdown.open a:hover .caret{border-top-color:#fff;border-bottom-color:#fff;opacity:1;filter:alpha(opacity=100)}.tabs-stacked .open>a:hover{border-color:#999}.tabbable{*zoom:1}.tabbable:before,.tabbable:after{display:table;line-height:0;content:""}.tabbable:after{clear:both}.tab-content{overflow:auto}.tabs-below>.nav-tabs,.tabs-right>.nav-tabs,.tabs-left>.nav-tabs{border-bottom:0}.tab-content>.tab-pane,.pill-content>.pill-pane{display:none}.tab-content>.active,.pill-content>.active{display:block}.tabs-below>.nav-tabs{border-top:1px solid #ddd}.tabs-below>.nav-tabs>li{margin-top:-1px;margin-bottom:0}.tabs-below>.nav-tabs>li>a
{-webkit-border-radius:0 0 4px 4px;-moz-border-radius:0 0 4px 4px;border-radius:0 0 4px 4px}.tabs-below>.nav-tabs>li>a:hover{border-top-color:#ddd;border-bottom-color:transparent}.tabs-below>.nav-tabs>.active>a,.tabs-below>.nav-tabs>.active>a:hover{border-color:transparent #ddd #ddd #ddd}.tabs-left>.nav-tabs>li,.tabs-right>.nav-tabs>li{float:none}.tabs-left>.nav-tabs>li>a,.tabs-right>.nav-tabs>li>a{min-width:74px;margin-right:0;margin-bottom:3px}.tabs-left>.nav-tabs{float:left;margin-right:19px;border-right:1px solid #ddd}.tabs-left>.nav-tabs>li>a{margin-right:-1px;-webkit-border-radius:4px 0 0 4px;-moz-border-radius:4px 0 0 4px;border-radius:4px 0 0 4px}.tabs-left>.nav-tabs>li>a:hover{border-color:#eee #dddddd #eee #eeeeee}.tabs-left>.nav-tabs .active>a,.tabs-left>.nav-tabs .active>a:hover{border-color:#ddd transparent #ddd #ddd;*border-right-color:#fff}.tabs-right>.nav-tabs{float:right;margin-left:19px;border-left:1px solid #ddd}.tabs-right>.nav-tabs>li>a{margin-left:-1px;-webkit-
border-radius:0 4px 4px 0;-moz-border-radius:0 4px 4px 0;border-radius:0 4px 4px 0}.tabs-right>.nav-tabs>li>a:hover{border-color:#eee #eeeeee #eee #dddddd}.tabs-right>.nav-tabs .active>a,.tabs-right>.nav-tabs .active>a:hover{border-color:#ddd #ddd #ddd transparent;*border-left-color:#fff}.nav>.disabled>a{color:#999}.nav>.disabled>a:hover{text-decoration:none;cursor:default;background-color:transparent}.navbar{*position:relative;*z-index:2;margin-bottom:20px;overflow:visible;color:#555}.navbar-inner{min-height:40px;padding-right:20px;padding-left:20px;background-color:#fafafa;background-image:-moz-linear-gradient(top,#fff,#f2f2f2);background-image:-webkit-gradient(linear,0 0,0 100%,from(#fff),to(#f2f2f2));background-image:-webkit-linear-gradient(top,#fff,#f2f2f2);background-image:-o-linear-gradient(top,#fff,#f2f2f2);background-image:linear-gradient(to bottom,#fff,#f2f2f2);background-repeat:repeat-x;border:1px solid #d4d4d4;-webkit-border-radius:4px;-moz-border-radius:4px;border-radiu
s:4px;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffffffff',endColorstr='#fff2f2f2',GradientType=0);-webkit-box-shadow:0 1px 4px rgba(0,0,0,0.065);-moz-box-shadow:0 1px 4px rgba(0,0,0,0.065);box-shadow:0 1px 4px rgba(0,0,0,0.065)}.navbar .container{width:auto}.nav-collapse.collapse{height:auto}.navbar .brand{display:block;float:left;padding:10px 20px 10px;margin-left:-20px;font-size:20px;font-weight:200;color:#555;text-shadow:0 1px 0 #fff}.navbar .brand:hover{text-decoration:none}.navbar-text{margin-bottom:0;line-height:40px}.navbar-link{color:#555}.navbar-link:hover{color:#333}.navbar .divider-vertical{height:40px;margin:0 9px;border-right:1px solid #fff;border-left:1px solid #f2f2f2}.navbar .btn,.navbar .btn-group{margin-top:6px}.navbar .btn-group .btn{margin:0}.navbar-form{margin-bottom:0;*zoom:1}.navbar-form:before,.navbar-form:after{display:table;line-height:0;content:""}.navbar-form:after{clear:both}.navbar-form input,.navbar-form select,.navbar-form .rad
io,.navbar-form .checkbox{margin-top:5px}.navbar-form input,.navbar-form select,.navbar-form .btn{display:inline-block;margin-bottom:0}.navbar-form input[type="image"],.navbar-form input[type="checkbox"],.navbar-form input[type="radio"]{margin-top:3px}.navbar-form .input-append,.navbar-form .input-prepend{margin-top:6px;white-space:nowrap}.navbar-form .input-append input,.navbar-form .input-prepend input{margin-top:0}.navbar-search{position:relative;float:left;margin-top:5px;margin-bottom:0}.navbar-search .search-query{padding:4px 14px;margin-bottom:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:13px;font-weight:normal;line-height:1;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.navbar-static-top{position:static;width:100%;margin-bottom:0}.navbar-static-top .navbar-inner{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.navbar-fixed-top,.navbar-fixed-bottom{position:fixed;right:0;left:0;z-index:1030;margin-bottom:0}.navbar-fix
ed-top .navbar-inner,.navbar-fixed-bottom .navbar-inner,.navbar-static-top .navbar-inner{border:0}.navbar-fixed-top .navbar-inner,.navbar-fixed-bottom .navbar-inner{padding-right:0;padding-left:0;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:940px}.navbar-fixed-top{top:0}.navbar-fixed-top .navbar-inner,.navbar-static-top .navbar-inner{-webkit-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.1),0 1px 10px rgba(0,0,0,0.1);-moz-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.1),0 1px 10px rgba(0,0,0,0.1);box-shadow:inset 0 -1px 0 rgba(0,0,0,0.1),0 1px 10px rgba(0,0,0,0.1)}.navbar-fixed-bottom{bottom:0}.navbar-fixed-bottom .navbar-inner{-webkit-box-shadow:inset 0 1px 0 rgba(0,0,0,0.1),0 -1px 10px rgba(0,0,0,0.1);-moz-box-shadow:inset 0 1px 0 rgba(0,0,0,0.1),0 -1px 10px rgba(0,0,0,0.1);box-shadow:inset 0 1px 0 rgba(0,0,0,0.1),0 -1px 10px rgba(0,0,0,0.1)}.navbar .nav{position:relative;left:0;displ
ay:block;float:left;margin:0 10px 0 0}.navbar .nav.pull-right{float:right}.navbar .nav>li{float:left}.navbar .nav>li>a{float:none;padding:10px 15px 10px;color:#555;text-decoration:none;text-shadow:0 1px 0 #fff}.navbar .nav .dropdown-toggle .caret{margin-top:8px}.navbar .nav>li>a:focus,.navbar .nav>li>a:hover{color:#333;text-decoration:none;background-color:transparent}.navbar .nav>.active>a,.navbar .nav>.active>a:hover,.navbar .nav>.active>a:focus{color:#555;text-decoration:none;background-color:#e5e5e5;-webkit-box-shadow:inset 0 3px 8px rgba(0,0,0,0.125);-moz-box-shadow:inset 0 3px 8px rgba(0,0,0,0.125);box-shadow:inset 0 3px 8px rgba(0,0,0,0.125)}.navbar .btn-navbar{display:none;float:right;padding:7px 10px;margin-right:5px;margin-left:5px;color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#ededed;*background-color:#e5e5e5;background-image:-webkit-gradient(linear,0 0,0 100%,from(#f2f2f2),to(#e5e5e5));background-image:-webkit-linear-gradient(top,#f2f2f2,#e5e5e5);back
ground-image:-o-linear-gradient(top,#f2f2f2,#e5e5e5);background-image:linear-gradient(to bottom,#f2f2f2,#e5e5e5);background-image:-moz-linear-gradient(top,#f2f2f2,#e5e5e5);background-repeat:repeat-x;border-color:#e5e5e5 #e5e5e5 #bfbfbf;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#fff2f2f2',endColorstr='#ffe5e5e5',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false);-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075);-moz-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075);box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075)}.navbar .btn-navbar:hover,.navbar .btn-navbar:active,.navbar .btn-navbar.active,.navbar .btn-navbar.disabled,.navbar .btn-navbar[disabled]{color:#fff;background-color:#e5e5e5;*background-color:#d9d9d9}.navbar .btn-navbar:active,.navbar .btn-navbar.active{background-co
lor:#ccc \9}.navbar .btn-navbar .icon-bar{display:block;width:18px;height:2px;background-color:#f5f5f5;-webkit-border-radius:1px;-moz-border-radius:1px;border-radius:1px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,0.25);-moz-box-shadow:0 1px 0 rgba(0,0,0,0.25);box-shadow:0 1px 0 rgba(0,0,0,0.25)}.btn-navbar .icon-bar+.icon-bar{margin-top:3px}.navbar .nav>li>.dropdown-menu:before{position:absolute;top:-7px;left:9px;display:inline-block;border-right:7px solid transparent;border-bottom:7px solid #ccc;border-left:7px solid transparent;border-bottom-color:rgba(0,0,0,0.2);content:''}.navbar .nav>li>.dropdown-menu:after{position:absolute;top:-6px;left:10px;display:inline-block;border-right:6px solid transparent;border-bottom:6px solid #fff;border-left:6px solid transparent;content:''}.navbar-fixed-bottom .nav>li>.dropdown-menu:before{top:auto;bottom:-7px;border-top:7px solid #ccc;border-bottom:0;border-top-color:rgba(0,0,0,0.2)}.navbar-fixed-bottom .nav>li>.dropdown-menu:after{top:auto;bottom:-6
px;border-top:6px solid #fff;border-bottom:0}.navbar .nav li.dropdown.open>.dropdown-toggle,.navbar .nav li.dropdown.active>.dropdown-toggle,.navbar .nav li.dropdown.open.active>.dropdown-toggle{color:#555;background-color:#e5e5e5}.navbar .nav li.dropdown>.dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.navbar .nav li.dropdown.open>.dropdown-toggle .caret,.navbar .nav li.dropdown.active>.dropdown-toggle .caret,.navbar .nav li.dropdown.open.active>.dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.navbar .pull-right>li>.dropdown-menu,.navbar .nav>li>.dropdown-menu.pull-right{right:0;left:auto}.navbar .pull-right>li>.dropdown-menu:before,.navbar .nav>li>.dropdown-menu.pull-right:before{right:12px;left:auto}.navbar .pull-right>li>.dropdown-menu:after,.navbar .nav>li>.dropdown-menu.pull-right:after{right:13px;left:auto}.navbar .pull-right>li>.dropdown-menu .dropdown-menu,.navbar .nav>li>.dropdown-menu.pull-right .dropdown-menu{right:100%;left:
auto;margin-right:-1px;margin-left:0;-webkit-border-radius:6px 0 6px 6px;-moz-border-radius:6px 0 6px 6px;border-radius:6px 0 6px 6px}.navbar-inverse{color:#999}.navbar-inverse .navbar-inner{background-color:#1b1b1b;background-image:-moz-linear-gradient(top,#222,#111);background-image:-webkit-gradient(linear,0 0,0 100%,from(#222),to(#111));background-image:-webkit-linear-gradient(top,#222,#111);background-image:-o-linear-gradient(top,#222,#111);background-image:linear-gradient(to bottom,#222,#111);background-repeat:repeat-x;border-color:#252525;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff222222',endColorstr='#ff111111',GradientType=0)}.navbar-inverse .brand,.navbar-inverse .nav>li>a{color:#999;text-shadow:0 -1px 0 rgba(0,0,0,0.25)}.navbar-inverse .brand:hover,.navbar-inverse .nav>li>a:hover{color:#fff}.navbar-inverse .nav>li>a:focus,.navbar-inverse .nav>li>a:hover{color:#fff;background-color:transparent}.navbar-inverse .nav .active>a,.navbar-inverse .nav .act
ive>a:hover,.navbar-inverse .nav .active>a:focus{color:#fff;background-color:#111}.navbar-inverse .navbar-link{color:#999}.navbar-inverse .navbar-link:hover{color:#fff}.navbar-inverse .divider-vertical{border-right-color:#222;border-left-color:#111}.navbar-inverse .nav li.dropdown.open>.dropdown-toggle,.navbar-inverse .nav li.dropdown.active>.dropdown-toggle,.navbar-inverse .nav li.dropdown.open.active>.dropdown-toggle{color:#fff;background-color:#111}.navbar-inverse .nav li.dropdown>.dropdown-toggle .caret{border-top-color:#999;border-bottom-color:#999}.navbar-inverse .nav li.dropdown.open>.dropdown-toggle .caret,.navbar-inverse .nav li.dropdown.active>.dropdown-toggle .caret,.navbar-inverse .nav li.dropdown.open.active>.dropdown-toggle .caret{border-top-color:#fff;border-bottom-color:#fff}.navbar-inverse .navbar-search .search-query{color:#fff;background-color:#515151;border-color:#111;-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);-moz-box-shado
w:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);box-shadow:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);-webkit-transition:none;-moz-transition:none;-o-transition:none;transition:none}.navbar-inverse .navbar-search .search-query:-moz-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query:-ms-input-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query::-webkit-input-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query:focus,.navbar-inverse .navbar-search .search-query.focused{padding:5px 15px;color:#333;text-shadow:0 1px 0 #fff;background-color:#fff;border:0;outline:0;-webkit-box-shadow:0 0 3px rgba(0,0,0,0.15);-moz-box-shadow:0 0 3px rgba(0,0,0,0.15);box-shadow:0 0 3px rgba(0,0,0,0.15)}.navbar-inverse .btn-navbar{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#0e0e0e;*background-color:#040404;background-image:-webkit-gradient(linear,0 0,0 100%,from(#151515),to(#040404));background-image:-
webkit-linear-gradient(top,#151515,#040404);background-image:-o-linear-gradient(top,#151515,#040404);background-image:linear-gradient(to bottom,#151515,#040404);background-image:-moz-linear-gradient(top,#151515,#040404);background-repeat:repeat-x;border-color:#040404 #040404 #000;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff151515',endColorstr='#ff040404',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.navbar-inverse .btn-navbar:hover,.navbar-inverse .btn-navbar:active,.navbar-inverse .btn-navbar.active,.navbar-inverse .btn-navbar.disabled,.navbar-inverse .btn-navbar[disabled]{color:#fff;background-color:#040404;*background-color:#000}.navbar-inverse .btn-navbar:active,.navbar-inverse .btn-navbar.active{background-color:#000 \9}.breadcrumb{padding:8px 15px;margin:0 0 20px;list-style:none;background-color:#f5f5f5;-webkit-border-radius:4px;-moz-border-radius:4px;borde
r-radius:4px}.breadcrumb li{display:inline-block;*display:inline;text-shadow:0 1px 0 #fff;*zoom:1}.breadcrumb .divider{padding:0 5px;color:#ccc}.breadcrumb .active{color:#999}.pagination{height:40px;margin:20px 0}.pagination ul{display:inline-block;*display:inline;margin-bottom:0;margin-left:0;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px;*zoom:1;-webkit-box-shadow:0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:0 1px 2px rgba(0,0,0,0.05);box-shadow:0 1px 2px rgba(0,0,0,0.05)}.pagination li{display:inline}.pagination a,.pagination span{float:left;padding:0 14px;line-height:38px;text-decoration:none;background-color:#fff;border:1px solid #ddd;border-left-width:0}.pagination a:hover,.pagination .active a,.pagination .active span{background-color:#f5f5f5}.pagination .active a,.pagination .active span{color:#999;cursor:default}.pagination .disabled span,.pagination .disabled a,.pagination .disabled a:hover{color:#999;cursor:default;background-color:transparent}.paginatio
n li:first-child a,.pagination li:first-child span{border-left-width:1px;-webkit-border-radius:3px 0 0 3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.pagination li:last-child a,.pagination li:last-child span{-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}.pagination-centered{text-align:center}.pagination-right{text-align:right}.pager{margin:20px 0;text-align:center;list-style:none;*zoom:1}.pager:before,.pager:after{display:table;line-height:0;content:""}.pager:after{clear:both}.pager li{display:inline}.pager a{display:inline-block;padding:5px 14px;background-color:#fff;border:1px solid #ddd;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.pager a:hover{text-decoration:none;background-color:#f5f5f5}.pager .next a{float:right}.pager .previous a{float:left}.pager .disabled a,.pager .disabled a:hover{color:#999;cursor:default;background-color:#fff}.modal-open .dropdown-menu{z-index:2050}.modal-open .dropdow
n.open{*z-index:2050}.modal-open .popover{z-index:2060}.modal-open .tooltip{z-index:2080}.modal-backdrop{position:fixed;top:0;right:0;bottom:0;left:0;z-index:1040;background-color:#000}.modal-backdrop.fade{opacity:0}.modal-backdrop,.modal-backdrop.fade.in{opacity:.8;filter:alpha(opacity=80)}.modal{position:fixed;top:50%;left:50%;z-index:1050;width:560px;margin:-250px 0 0 -280px;overflow:auto;background-color:#fff;border:1px solid #999;border:1px solid rgba(0,0,0,0.3);*border:1px solid #999;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 3px 7px rgba(0,0,0,0.3);-moz-box-shadow:0 3px 7px rgba(0,0,0,0.3);box-shadow:0 3px 7px rgba(0,0,0,0.3);-webkit-background-clip:padding-box;-moz-background-clip:padding-box;background-clip:padding-box}.modal.fade{top:-25%;-webkit-transition:opacity .3s linear,top .3s ease-out;-moz-transition:opacity .3s linear,top .3s ease-out;-o-transition:opacity .3s linear,top .3s ease-out;transition:opacity .3s linear,top .3
s ease-out}.modal.fade.in{top:50%}.modal-header{padding:9px 15px;border-bottom:1px solid #eee}.modal-header .close{margin-top:2px}.modal-header h3{margin:0;line-height:30px}.modal-body{max-height:400px;padding:15px;overflow-y:auto}.modal-form{margin-bottom:0}.modal-footer{padding:14px 15px 15px;margin-bottom:0;text-align:right;background-color:#f5f5f5;border-top:1px solid #ddd;-webkit-border-radius:0 0 6px 6px;-moz-border-radius:0 0 6px 6px;border-radius:0 0 6px 6px;*zoom:1;-webkit-box-shadow:inset 0 1px 0 #fff;-moz-box-shadow:inset 0 1px 0 #fff;box-shadow:inset 0 1px 0 #fff}.modal-footer:before,.modal-footer:after{display:table;line-height:0;content:""}.modal-footer:after{clear:both}.modal-footer .btn+.btn{margin-bottom:0;margin-left:5px}.modal-footer .btn-group .btn+.btn{margin-left:-1px}.tooltip{position:absolute;z-index:1030;display:block;padding:5px;font-size:11px;opacity:0;filter:alpha(opacity=0);visibility:visible}.tooltip.in{opacity:.8;filter:alpha(opacity=80)}.tooltip.top{m
argin-top:-3px}.tooltip.right{margin-left:3px}.tooltip.bottom{margin-top:3px}.tooltip.left{margin-left:-3px}.tooltip-inner{max-width:200px;padding:3px 8px;color:#fff;text-align:center;text-decoration:none;background-color:#000;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.tooltip-arrow{position:absolute;width:0;height:0;border-color:transparent;border-style:solid}.tooltip.top .tooltip-arrow{bottom:0;left:50%;margin-left:-5px;border-top-color:#000;border-width:5px 5px 0}.tooltip.right .tooltip-arrow{top:50%;left:0;margin-top:-5px;border-right-color:#000;border-width:5px 5px 5px 0}.tooltip.left .tooltip-arrow{top:50%;right:0;margin-top:-5px;border-left-color:#000;border-width:5px 0 5px 5px}.tooltip.bottom .tooltip-arrow{top:0;left:50%;margin-left:-5px;border-bottom-color:#000;border-width:0 5px 5px}.popover{position:absolute;top:0;left:0;z-index:1010;display:none;width:236px;padding:1px;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);
-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 5px 10px rgba(0,0,0,0.2);-moz-box-shadow:0 5px 10px rgba(0,0,0,0.2);box-shadow:0 5px 10px rgba(0,0,0,0.2);-webkit-background-clip:padding-box;-moz-background-clip:padding;background-clip:padding-box}.popover.top{margin-bottom:10px}.popover.right{margin-left:10px}.popover.bottom{margin-top:10px}.popover.left{margin-right:10px}.popover-title{padding:8px 14px;margin:0;font-size:14px;font-weight:normal;line-height:18px;background-color:#f7f7f7;border-bottom:1px solid #ebebeb;-webkit-border-radius:5px 5px 0 0;-moz-border-radius:5px 5px 0 0;border-radius:5px 5px 0 0}.popover-content{padding:9px 14px}.popover-content p,.popover-content ul,.popover-content ol{margin-bottom:0}.popover .arrow,.popover .arrow:after{position:absolute;display:inline-block;width:0;height:0;border-color:transparent;border-style:solid}.popover .arrow:after{z-index:-1;content:""}.popover.top .arrow{bottom:-10px;left:50%;margin-le
ft:-10px;border-top-color:#fff;border-width:10px 10px 0}.popover.top .arrow:after{bottom:-1px;left:-11px;border-top-color:rgba(0,0,0,0.25);border-width:11px 11px 0}.popover.right .arrow{top:50%;left:-10px;margin-top:-10px;border-right-color:#fff;border-width:10px 10px 10px 0}.popover.right .arrow:after{bottom:-11px;left:-1px;border-right-color:rgba(0,0,0,0.25);border-width:11px 11px 11px 0}.popover.bottom .arrow{top:-10px;left:50%;margin-left:-10px;border-bottom-color:#fff;border-width:0 10px 10px}.popover.bottom .arrow:after{top:-1px;left:-11px;border-bottom-color:rgba(0,0,0,0.25);border-width:0 11px 11px}.popover.left .arrow{top:50%;right:-10px;margin-top:-10px;border-left-color:#fff;border-width:10px 0 10px 10px}.popover.left .arrow:after{right:-1px;bottom:-11px;border-left-color:rgba(0,0,0,0.25);border-width:11px 0 11px 11px}.thumbnails{margin-left:-20px;list-style:none;*zoom:1}.thumbnails:before,.thumbnails:after{display:table;line-height:0;content:""}.thumbnails:after{clear:bo
th}.row-fluid .thumbnails{margin-left:0}.thumbnails>li{float:left;margin-bottom:20px;margin-left:20px}.thumbnail{display:block;padding:4px;line-height:20px;border:1px solid #ddd;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;-webkit-box-shadow:0 1px 3px rgba(0,0,0,0.055);-moz-box-shadow:0 1px 3px rgba(0,0,0,0.055);box-shadow:0 1px 3px rgba(0,0,0,0.055);-webkit-transition:all .2s ease-in-out;-moz-transition:all .2s ease-in-out;-o-transition:all .2s ease-in-out;transition:all .2s ease-in-out}a.thumbnail:hover{border-color:#08c;-webkit-box-shadow:0 1px 4px rgba(0,105,214,0.25);-moz-box-shadow:0 1px 4px rgba(0,105,214,0.25);box-shadow:0 1px 4px rgba(0,105,214,0.25)}.thumbnail>img{display:block;max-width:100%;margin-right:auto;margin-left:auto}.thumbnail .caption{padding:9px;color:#555}.label,.badge{font-size:11.844px;font-weight:bold;line-height:14px;color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);white-space:nowrap;vertical-align:baseline;background-color:#999}.lab
el{padding:1px 4px 2px;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}.badge{padding:1px 9px 2px;-webkit-border-radius:9px;-moz-border-radius:9px;border-radius:9px}a.label:hover,a.badge:hover{color:#fff;text-decoration:none;cursor:pointer}.label-important,.badge-important{background-color:#b94a48}.label-important[href],.badge-important[href]{background-color:#953b39}.label-warning,.badge-warning{background-color:#f89406}.label-warning[href],.badge-warning[href]{background-color:#c67605}.label-success,.badge-success{background-color:#468847}.label-success[href],.badge-success[href]{background-color:#356635}.label-info,.badge-info{background-color:#3a87ad}.label-info[href],.badge-info[href]{background-color:#2d6987}.label-inverse,.badge-inverse{background-color:#333}.label-inverse[href],.badge-inverse[href]{background-color:#1a1a1a}.btn .label,.btn .badge{position:relative;top:-1px}.btn-mini .label,.btn-mini .badge{top:0}@-webkit-keyframes progress-bar-stripes{from
{background-position:40px 0}to{background-position:0 0}}@-moz-keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}@-ms-keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}@-o-keyframes progress-bar-stripes{from{background-position:0 0}to{background-position:40px 0}}@keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}.progress{height:20px;margin-bottom:20px;overflow:hidden;background-color:#f7f7f7;background-image:-moz-linear-gradient(top,#f5f5f5,#f9f9f9);background-image:-webkit-gradient(linear,0 0,0 100%,from(#f5f5f5),to(#f9f9f9));background-image:-webkit-linear-gradient(top,#f5f5f5,#f9f9f9);background-image:-o-linear-gradient(top,#f5f5f5,#f9f9f9);background-image:linear-gradient(to bottom,#f5f5f5,#f9f9f9);background-repeat:repeat-x;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff
f5f5f5',endColorstr='#fff9f9f9',GradientType=0);-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1);-moz-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1);box-shadow:inset 0 1px 2px rgba(0,0,0,0.1)}.progress .bar{float:left;width:0;height:100%;font-size:12px;color:#fff;text-align:center;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#0e90d2;background-image:-moz-linear-gradient(top,#149bdf,#0480be);background-image:-webkit-gradient(linear,0 0,0 100%,from(#149bdf),to(#0480be));background-image:-webkit-linear-gradient(top,#149bdf,#0480be);background-image:-o-linear-gradient(top,#149bdf,#0480be);background-image:linear-gradient(to bottom,#149bdf,#0480be);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff149bdf',endColorstr='#ff0480be',GradientType=0);-webkit-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.15);-moz-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.15);box-shadow:inset 0 -1px 0 rgba(0,0,0,0.15);-webkit-box-sizing:border-box;-moz-box-sizing:border
-box;box-sizing:border-box;-webkit-transition:width .6s ease;-moz-transition:width .6s ease;-o-transition:width .6s ease;transition:width .6s ease}.progress .bar+.bar{-webkit-box-shadow:inset 1px 0 0 rgba(0,0,0,0.15),inset 0 -1px 0 rgba(0,0,0,0.15);-moz-box-shadow:inset 1px 0 0 rgba(0,0,0,0.15),inset 0 -1px 0 rgba(0,0,0,0.15);box-shadow:inset 1px 0 0 rgba(0,0,0,0.15),inset 0 -1px 0 rgba(0,0,0,0.15)}.progress-striped .bar{background-color:#149bdf;background-image:-webkit-gradient(linear,0 100%,100% 0,color-stop(0.25,rgba(255,255,255,0.15)),color-stop(0.25,transparent),color-stop(0.5,transparent),color-stop(0.5,rgba(255,255,255,0.15)),color-stop(0.75,rgba(255,255,255,0.15)),color-stop(0.75,transparent),to(transparent));background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-moz-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparen
t 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);-webkit-background-size:40px 40px;-moz-background-size:40px 40px;-o-background-size:40px 40px;background-size:40px 40px}.progress.active .bar{-webkit-animation:progress-bar-stripes 2s linear infinite;-moz-animation:progress-bar-stripes 2s linear infinite;-ms-animation:progress-bar-stripes 2s linear infinite;-o-animation:progress-bar-stripes 2s linear infinite;animation:progress-bar-stripes 2s linear infinite}.progress-danger .bar,.progress .bar-danger{background-color:#dd514c;background-image:-moz-linear-gradient(top,#ee5
f5b,#c43c35);background-image:-webkit-gradient(linear,0 0,0 100%,from(#ee5f5b),to(#c43c35));background-image:-webkit-linear-gradient(top,#ee5f5b,#c43c35);background-image:-o-linear-gradient(top,#ee5f5b,#c43c35);background-image:linear-gradient(to bottom,#ee5f5b,#c43c35);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffee5f5b',endColorstr='#ffc43c35',GradientType=0)}.progress-danger.progress-striped .bar,.progress-striped .bar-danger{background-color:#ee5f5b;background-image:-webkit-gradient(linear,0 100%,100% 0,color-stop(0.25,rgba(255,255,255,0.15)),color-stop(0.25,transparent),color-stop(0.5,transparent),color-stop(0.5,rgba(255,255,255,0.15)),color-stop(0.75,rgba(255,255,255,0.15)),color-stop(0.75,transparent),to(transparent));background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-moz-l
inear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent)}.progress-success .bar,.progress .bar-success{background-color:#5eb95e;background-image:-moz-linear-gradient(top,#62c462,#57a957);background-image:-webkit-gradient(linear,0 0,0 100%,from(#62c462),to(#57a957));background-image:-webkit-linear-gradient(top,#62c462,#57a957);background-image:-o-linear-gradient(top,#62c462,#57a957);background-image:linear-gradient(to bottom,#62c462,#57a957);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(star
tColorstr='#ff62c462',endColorstr='#ff57a957',GradientTyp
<TRUNCATED>
[12/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/css/print.css
----------------------------------------------------------------------
diff --git a/site/current-book/css/print.css b/site/current-book/css/print.css
new file mode 100644
index 0000000..1cd02d9
--- /dev/null
+++ b/site/current-book/css/print.css
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/* $Id: print.css 1201871 2011-11-14 20:18:24Z simonetripodi $ */
+
+#banner, #footer, #leftcol, #breadcrumbs, .docs #toc, .docs .courtesylinks, #leftColumn, #navColumn {display: none !important;}
+#bodyColumn, body.docs div.docs {margin: 0 !important;border: none !important}
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/css/site.css
----------------------------------------------------------------------
diff --git a/site/current-book/css/site.css b/site/current-book/css/site.css
new file mode 100644
index 0000000..055e7e2
--- /dev/null
+++ b/site/current-book/css/site.css
@@ -0,0 +1 @@
+/* You can override this file with your own styles */
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/ApacheIncubating_Logo.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/ApacheIncubating_Logo.png b/site/current-book/images/ApacheIncubating_Logo.png
new file mode 100644
index 0000000..83f096c
Binary files /dev/null and b/site/current-book/images/ApacheIncubating_Logo.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/accessories-text-editor.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/accessories-text-editor.png b/site/current-book/images/accessories-text-editor.png
new file mode 100644
index 0000000..abc3366
Binary files /dev/null and b/site/current-book/images/accessories-text-editor.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/add.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/add.gif b/site/current-book/images/add.gif
new file mode 100644
index 0000000..1cb3dbf
Binary files /dev/null and b/site/current-book/images/add.gif differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/apache-maven-project-2.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/apache-maven-project-2.png b/site/current-book/images/apache-maven-project-2.png
new file mode 100644
index 0000000..6c096ec
Binary files /dev/null and b/site/current-book/images/apache-maven-project-2.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/application-certificate.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/application-certificate.png b/site/current-book/images/application-certificate.png
new file mode 100644
index 0000000..cc6aff6
Binary files /dev/null and b/site/current-book/images/application-certificate.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/contact-new.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/contact-new.png b/site/current-book/images/contact-new.png
new file mode 100644
index 0000000..ebc4316
Binary files /dev/null and b/site/current-book/images/contact-new.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/document-properties.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/document-properties.png b/site/current-book/images/document-properties.png
new file mode 100644
index 0000000..34c2409
Binary files /dev/null and b/site/current-book/images/document-properties.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/drive-harddisk.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/drive-harddisk.png b/site/current-book/images/drive-harddisk.png
new file mode 100644
index 0000000..d7ce475
Binary files /dev/null and b/site/current-book/images/drive-harddisk.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/enrichment_arch.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/enrichment_arch.png b/site/current-book/images/enrichment_arch.png
new file mode 100644
index 0000000..3b8bcdb
Binary files /dev/null and b/site/current-book/images/enrichment_arch.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/fix.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/fix.gif b/site/current-book/images/fix.gif
new file mode 100644
index 0000000..b7eb3dc
Binary files /dev/null and b/site/current-book/images/fix.gif differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/icon_error_sml.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/icon_error_sml.gif b/site/current-book/images/icon_error_sml.gif
new file mode 100644
index 0000000..12e9a01
Binary files /dev/null and b/site/current-book/images/icon_error_sml.gif differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/icon_help_sml.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/icon_help_sml.gif b/site/current-book/images/icon_help_sml.gif
new file mode 100644
index 0000000..aaf20e6
Binary files /dev/null and b/site/current-book/images/icon_help_sml.gif differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/icon_info_sml.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/icon_info_sml.gif b/site/current-book/images/icon_info_sml.gif
new file mode 100644
index 0000000..b776326
Binary files /dev/null and b/site/current-book/images/icon_info_sml.gif differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/icon_success_sml.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/icon_success_sml.gif b/site/current-book/images/icon_success_sml.gif
new file mode 100644
index 0000000..0a19527
Binary files /dev/null and b/site/current-book/images/icon_success_sml.gif differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/icon_warning_sml.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/icon_warning_sml.gif b/site/current-book/images/icon_warning_sml.gif
new file mode 100644
index 0000000..ac6ad6a
Binary files /dev/null and b/site/current-book/images/icon_warning_sml.gif differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/image-x-generic.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/image-x-generic.png b/site/current-book/images/image-x-generic.png
new file mode 100644
index 0000000..ab49efb
Binary files /dev/null and b/site/current-book/images/image-x-generic.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/indexing_arch.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/indexing_arch.png b/site/current-book/images/indexing_arch.png
new file mode 100644
index 0000000..6c97206
Binary files /dev/null and b/site/current-book/images/indexing_arch.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/internet-web-browser.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/internet-web-browser.png b/site/current-book/images/internet-web-browser.png
new file mode 100644
index 0000000..307d6ac
Binary files /dev/null and b/site/current-book/images/internet-web-browser.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/logos/build-by-maven-black.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/logos/build-by-maven-black.png b/site/current-book/images/logos/build-by-maven-black.png
new file mode 100644
index 0000000..919fd0f
Binary files /dev/null and b/site/current-book/images/logos/build-by-maven-black.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/logos/build-by-maven-white.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/logos/build-by-maven-white.png b/site/current-book/images/logos/build-by-maven-white.png
new file mode 100644
index 0000000..7d44c9c
Binary files /dev/null and b/site/current-book/images/logos/build-by-maven-white.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/logos/maven-feather.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/logos/maven-feather.png b/site/current-book/images/logos/maven-feather.png
new file mode 100644
index 0000000..b5ada83
Binary files /dev/null and b/site/current-book/images/logos/maven-feather.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/maas_arch.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/maas_arch.png b/site/current-book/images/maas_arch.png
new file mode 100644
index 0000000..8409f70
Binary files /dev/null and b/site/current-book/images/maas_arch.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/metron-logo.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/metron-logo.png b/site/current-book/images/metron-logo.png
new file mode 100644
index 0000000..a0bc8cb
Binary files /dev/null and b/site/current-book/images/metron-logo.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/network-server.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/network-server.png b/site/current-book/images/network-server.png
new file mode 100644
index 0000000..1d12e19
Binary files /dev/null and b/site/current-book/images/network-server.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/package-x-generic.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/package-x-generic.png b/site/current-book/images/package-x-generic.png
new file mode 100644
index 0000000..8b7e9e6
Binary files /dev/null and b/site/current-book/images/package-x-generic.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/parser_arch.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/parser_arch.png b/site/current-book/images/parser_arch.png
new file mode 100644
index 0000000..2efea24
Binary files /dev/null and b/site/current-book/images/parser_arch.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/profiles/pre-release.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/profiles/pre-release.png b/site/current-book/images/profiles/pre-release.png
new file mode 100644
index 0000000..d448e85
Binary files /dev/null and b/site/current-book/images/profiles/pre-release.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/profiles/retired.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/profiles/retired.png b/site/current-book/images/profiles/retired.png
new file mode 100644
index 0000000..f89f6a2
Binary files /dev/null and b/site/current-book/images/profiles/retired.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/profiles/sandbox.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/profiles/sandbox.png b/site/current-book/images/profiles/sandbox.png
new file mode 100644
index 0000000..f88b362
Binary files /dev/null and b/site/current-book/images/profiles/sandbox.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/remove.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/remove.gif b/site/current-book/images/remove.gif
new file mode 100644
index 0000000..fc65631
Binary files /dev/null and b/site/current-book/images/remove.gif differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/rss.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/rss.png b/site/current-book/images/rss.png
new file mode 100644
index 0000000..a9850ee
Binary files /dev/null and b/site/current-book/images/rss.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/update.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/update.gif b/site/current-book/images/update.gif
new file mode 100644
index 0000000..b2a6d0b
Binary files /dev/null and b/site/current-book/images/update.gif differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/images/window-new.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/window-new.png b/site/current-book/images/window-new.png
new file mode 100644
index 0000000..0e12ef9
Binary files /dev/null and b/site/current-book/images/window-new.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/img/glyphicons-halflings-white.png
----------------------------------------------------------------------
diff --git a/site/current-book/img/glyphicons-halflings-white.png b/site/current-book/img/glyphicons-halflings-white.png
new file mode 100644
index 0000000..3bf6484
Binary files /dev/null and b/site/current-book/img/glyphicons-halflings-white.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/img/glyphicons-halflings.png
----------------------------------------------------------------------
diff --git a/site/current-book/img/glyphicons-halflings.png b/site/current-book/img/glyphicons-halflings.png
new file mode 100644
index 0000000..a996999
Binary files /dev/null and b/site/current-book/img/glyphicons-halflings.png differ
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/index.html b/site/current-book/index.html
new file mode 100644
index 0000000..ae246ce
--- /dev/null
+++ b/site/current-book/index.html
@@ -0,0 +1,257 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – </title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class=""></li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li class="active">
+
+ <a href="#"><i class="icon-chevron-down"></i>Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <p><a class="externalLink" href="https://travis-ci.org/apache/incubator-metron"><img src="https://travis-ci.org/apache/incubator-metron.svg?branch=master" alt="Build Status" /></a></p>
+<h1>Apache Metron (Incubating)</h1>
+<p><a name="Apache_Metron_Incubating"></a></p>
+<p>Metron integrates a variety of open source big data technologies in order to offer a centralized tool for security monitoring and analysis. Metron provides capabilities for log aggregation, full packet capture indexing, storage, advanced behavioral analytics and data enrichment, while applying the most current threat intelligence information to security telemetry within a single platform.</p>
+<p>Metron can be divided into 4 areas:</p>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p><b>A mechanism to capture, store, and normalize any type of security telemetry at extremely high rates.</b> Because security telemetry is constantly being generated, it requires a method for ingesting the data at high speeds and pushing it to various processing units for advanced computation and analytics.</p></li>
+
+<li>
+<p><b>Real time processing and application of enrichments</b> such as threat intelligence, geolocation, and DNS information to telemetry being collected. The immediate application of this information to incoming telemetry provides the context and situational awareness, as well as the who and where information critical for investigation</p></li>
+
+<li>
+<p><b>Efficient information storage</b> based on how the information will be used:</p>
+
+<ul>
+
+<li>Logs and telemetry are stored such that they can be efficiently mined and analyzed for concise security visibility</li>
+
+<li>The ability to extract and reconstruct full packets helps an analyst answer questions such as who the true attacker was, what data was leaked, and where that data was sent</li>
+
+<li>Long-term storage not only increases visibility over time, but also enables advanced analytics such as machine learning techniques to be used to create models on the information. Incoming data can then be scored against these stored models for advanced anomaly detection.</li>
+ </ul></li>
+
+<li>
+<p><b>An interface that gives a security investigator a centralized view of data and alerts passed through the system.</b> Metron’s interface presents alert summaries with threat intelligence and enrichment data specific to that alert on one single page. Furthermore, advanced search capabilities and full packet extraction tools are presented to the analyst for investigation without the need to pivot into additional tools. </p></li>
+</ol>
+<p>Big data is a natural fit for powerful security analytics. The Metron framework integrates a number of elements from the Hadoop ecosystem to provide a scalable platform for security analytics, incorporating such functionality as full-packet capture, stream processing, batch processing, real-time search, and telemetry aggregation. With Metron, our goal is to tie big data into security analytics and drive towards an extensible centralized platform to effectively enable rapid detection and rapid response for advanced security threats. </p>
+<p><a name="Obtaining_Metron"></a></p>
+<h1>Obtaining Metron</h1>
+<p>This repository is a collection of submodules for convenience which is regularly updated to point to the latest versions. Github provides multiple ways to obtain Metron’s code:</p>
+
+<ol style="list-style-type: decimal">
+
+<li>git clone –recursive <a class="externalLink" href="https://github.com/apache/incubator-metron">https://github.com/apache/incubator-metron</a></li>
+
+<li><a class="externalLink" href="https://github.com/apache/incubator-metron/archive/master.zip">Download ZIP</a></li>
+
+<li>Clone or download each repository individually</li>
+</ol>
+<p>Option 3 is more likely to have the latest code.</p>
+<p><a name="Building_Metron"></a></p>
+<h1>Building Metron</h1>
+<p>Build the full project and run tests:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ mvn clean install
+</pre></div></div>
+<p>Build without tests:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ mvn clean install -DskipTests
+</pre></div></div>
+<p>Build with the HDP profile:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ mvn clean install -PHDP-2.5.0.0
+</pre></div></div>
+<p>You can swap “install” for “package” in the commands above if you don’t want to deploy the artifacts to your local .m2 repo.</p>
+<p><a name="Navigating_the_Architecture"></a></p>
+<h1>Navigating the Architecture</h1>
+<p>Metron is at its core a Kappa architecture with Apache Storm as the processing component and Apache Kafka as the unified data bus.</p>
+<p>Some high level links to the relevant subparts of the architecture, for more information:</p>
+
+<ul>
+
+<li><a href="metron-platform/metron-parsers/index.html">Parsers</a> : Parsing data from kafka into the Metron data model and passing it downstream to Enrichment.</li>
+
+<li><a href="metron-platform/metron-enrichment/index.html">Enrichment</a> : Enriching data post-parsing and providing the ability to tag a message as an alert and assign a risk triage level via a custom rule language.</li>
+
+<li><a href="metron-platform/metron-indexing/index.html">Indexing</a> : Indexing the data post-enrichment into HDFS, Elasticsearch or Solr.</li>
+</ul>
+<p>Some useful utilities that cross all of these parts of the architecture:</p>
+
+<ul>
+
+<li><a href="metron-platform/metron-common/index.html">Stellar</a> : A custom data transformation language that is used throughout metron from simple field transformation to expressing triage rules.</li>
+
+<li><a href="metron-analytics/metron-maas-service/index.html">Model as a Service</a> : A Yarn application which can deploy machine learning and statistical models onto the cluster along with the associated Stellar functions to be able to call out to them in a scalable manner.</li>
+
+<li><a href="metron-platform/metron-data-management/index.html">Data management</a> : A set of data management utilities aimed at getting data into HBase in a format which will allow data flowing through metron to be enriched with the results. Contains integrations with threat intelligence feeds exposed via TAXII as well as simple flat file structures.</li>
+
+<li><a href="metron-analytics/metron-profiler/index.html">Profiler</a> : A feature extraction mechanism that can generate a profile describing the behavior of an entity. An entity might be a server, user, subnet or application. Once a profile has been generated defining what normal behavior looks-like, models can be built that identify anomalous behavior.</li>
+</ul>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
[11/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/js/apache-maven-fluido-1.3.0.min.js
----------------------------------------------------------------------
diff --git a/site/current-book/js/apache-maven-fluido-1.3.0.min.js b/site/current-book/js/apache-maven-fluido-1.3.0.min.js
new file mode 100644
index 0000000..677e611
--- /dev/null
+++ b/site/current-book/js/apache-maven-fluido-1.3.0.min.js
@@ -0,0 +1,21 @@
+/*!
+ * jQuery JavaScript Library v1.8.0
+ * http://jquery.com/
+ *
+ * Includes Sizzle.js
+ * http://sizzlejs.com/
+ *
+ * Copyright 2012 jQuery Foundation and other contributors
+ * Released under the MIT license
+ * http://jquery.org/license
+ *
+ * Date: Thu Aug 09 2012 16:24:48 GMT-0400 (Eastern Daylight Time)
+ */
+(function(a2,aB){var x,af,o=a2.document,aI=a2.location,d=a2.navigator,bg=a2.jQuery,I=a2.$,am=Array.prototype.push,a4=Array.prototype.slice,aK=Array.prototype.indexOf,A=Object.prototype.toString,V=Object.prototype.hasOwnProperty,aN=String.prototype.trim,bG=function(e,bZ){return new bG.fn.init(e,bZ,x)},bx=/[\-+]?(?:\d*\.|)\d+(?:[eE][\-+]?\d+|)/.source,aa=/\S/,aV=/\s+/,D=aa.test("\xA0")?(/^[\s\xA0]+|[\s\xA0]+$/g):/^\s+|\s+$/g,bo=/^(?:[^#<]*(<[\w\W]+>)[^>]*$|#([\w\-]*)$)/,a=/^<(\w+)\s*\/?>(?:<\/\1>|)$/,bf=/^[\],:{}\s]*$/,bi=/(?:^|:|,)(?:\s*\[)+/g,bD=/\\(?:["\\\/bfnrt]|u[\da-fA-F]{4})/g,a0=/"[^"\\\r\n]*"|true|false|null|-?(?:\d\d*\.|)\d+(?:[eE][\-+]?\d+|)/g,bP=/^-ms-/,aU=/-([\da-z])/gi,N=function(e,bZ){return(bZ+"").toUpperCase()},aF=function(){if(o.addEventListener){o.removeEventListener("DOMContentLoaded",aF,false);bG.ready()}else{if(o.readyState==="complete"){o.detachEvent("onreadystatechange",aF);bG.ready()}}},Z={};bG.fn=bG.prototype={constructor:bG,init:function(e,b2,b1){var b0,b3,b
Z,b4;if(!e){return this}if(e.nodeType){this.context=this[0]=e;this.length=1;return this}if(typeof e==="string"){if(e.charAt(0)==="<"&&e.charAt(e.length-1)===">"&&e.length>=3){b0=[null,e,null]}else{b0=bo.exec(e)}if(b0&&(b0[1]||!b2)){if(b0[1]){b2=b2 instanceof bG?b2[0]:b2;b4=(b2&&b2.nodeType?b2.ownerDocument||b2:o);e=bG.parseHTML(b0[1],b4,true);if(a.test(b0[1])&&bG.isPlainObject(b2)){this.attr.call(e,b2,true)}return bG.merge(this,e)}else{b3=o.getElementById(b0[2]);if(b3&&b3.parentNode){if(b3.id!==b0[2]){return b1.find(e)}this.length=1;this[0]=b3}this.context=o;this.selector=e;return this}}else{if(!b2||b2.jquery){return(b2||b1).find(e)}else{return this.constructor(b2).find(e)}}}else{if(bG.isFunction(e)){return b1.ready(e)}}if(e.selector!==aB){this.selector=e.selector;this.context=e.context}return bG.makeArray(e,this)},selector:"",jquery:"1.8.0",length:0,size:function(){return this.length},toArray:function(){return a4.call(this)},get:function(e){return e==null?this.toArray():(e<0?this[t
his.length+e]:this[e])},pushStack:function(bZ,b1,e){var b0=bG.merge(this.constructor(),bZ);b0.prevObject=this;b0.context=this.context;if(b1==="find"){b0.selector=this.selector+(this.selector?" ":"")+e}else{if(b1){b0.selector=this.selector+"."+b1+"("+e+")"}}return b0},each:function(bZ,e){return bG.each(this,bZ,e)},ready:function(e){bG.ready.promise().done(e);return this},eq:function(e){e=+e;return e===-1?this.slice(e):this.slice(e,e+1)},first:function(){return this.eq(0)},last:function(){return this.eq(-1)},slice:function(){return this.pushStack(a4.apply(this,arguments),"slice",a4.call(arguments).join(","))},map:function(e){return this.pushStack(bG.map(this,function(b0,bZ){return e.call(b0,bZ,b0)}))},end:function(){return this.prevObject||this.constructor(null)},push:am,sort:[].sort,splice:[].splice};bG.fn.init.prototype=bG.fn;bG.extend=bG.fn.extend=function(){var b7,b0,e,bZ,b4,b5,b3=arguments[0]||{},b2=1,b1=arguments.length,b6=false;if(typeof b3==="boolean"){b6=b3;b3=arguments[1]||{
};b2=2}if(typeof b3!=="object"&&!bG.isFunction(b3)){b3={}}if(b1===b2){b3=this;--b2}for(;b2<b1;b2++){if((b7=arguments[b2])!=null){for(b0 in b7){e=b3[b0];bZ=b7[b0];if(b3===bZ){continue}if(b6&&bZ&&(bG.isPlainObject(bZ)||(b4=bG.isArray(bZ)))){if(b4){b4=false;b5=e&&bG.isArray(e)?e:[]}else{b5=e&&bG.isPlainObject(e)?e:{}}b3[b0]=bG.extend(b6,b5,bZ)}else{if(bZ!==aB){b3[b0]=bZ}}}}}return b3};bG.extend({noConflict:function(e){if(a2.$===bG){a2.$=I}if(e&&a2.jQuery===bG){a2.jQuery=bg}return bG},isReady:false,readyWait:1,holdReady:function(e){if(e){bG.readyWait++}else{bG.ready(true)}},ready:function(e){if(e===true?--bG.readyWait:bG.isReady){return}if(!o.body){return setTimeout(bG.ready,1)}bG.isReady=true;if(e!==true&&--bG.readyWait>0){return}af.resolveWith(o,[bG]);if(bG.fn.trigger){bG(o).trigger("ready").off("ready")}},isFunction:function(e){return bG.type(e)==="function"},isArray:Array.isArray||function(e){return bG.type(e)==="array"},isWindow:function(e){return e!=null&&e==e.window},isNumeric:fu
nction(e){return !isNaN(parseFloat(e))&&isFinite(e)},type:function(e){return e==null?String(e):Z[A.call(e)]||"object"},isPlainObject:function(b1){if(!b1||bG.type(b1)!=="object"||b1.nodeType||bG.isWindow(b1)){return false}try{if(b1.constructor&&!V.call(b1,"constructor")&&!V.call(b1.constructor.prototype,"isPrototypeOf")){return false}}catch(b0){return false}var bZ;for(bZ in b1){}return bZ===aB||V.call(b1,bZ)},isEmptyObject:function(bZ){var e;for(e in bZ){return false}return true},error:function(e){throw new Error(e)},parseHTML:function(b1,b0,e){var bZ;if(!b1||typeof b1!=="string"){return null}if(typeof b0==="boolean"){e=b0;b0=0}b0=b0||o;if((bZ=a.exec(b1))){return[b0.createElement(bZ[1])]}bZ=bG.buildFragment([b1],b0,e?null:[]);return bG.merge([],(bZ.cacheable?bG.clone(bZ.fragment):bZ.fragment).childNodes)},parseJSON:function(e){if(!e||typeof e!=="string"){return null}e=bG.trim(e);if(a2.JSON&&a2.JSON.parse){return a2.JSON.parse(e)}if(bf.test(e.replace(bD,"@").replace(a0,"]").replace(bi
,""))){return(new Function("return "+e))()}bG.error("Invalid JSON: "+e)},parseXML:function(b1){var bZ,b0;if(!b1||typeof b1!=="string"){return null}try{if(a2.DOMParser){b0=new DOMParser();bZ=b0.parseFromString(b1,"text/xml")}else{bZ=new ActiveXObject("Microsoft.XMLDOM");bZ.async="false";bZ.loadXML(b1)}}catch(b2){bZ=aB}if(!bZ||!bZ.documentElement||bZ.getElementsByTagName("parsererror").length){bG.error("Invalid XML: "+b1)}return bZ},noop:function(){},globalEval:function(e){if(e&&aa.test(e)){(a2.execScript||function(bZ){a2["eval"].call(a2,bZ)})(e)}},camelCase:function(e){return e.replace(bP,"ms-").replace(aU,N)},nodeName:function(bZ,e){return bZ.nodeName&&bZ.nodeName.toUpperCase()===e.toUpperCase()},each:function(b3,b4,b0){var bZ,b1=0,b2=b3.length,e=b2===aB||bG.isFunction(b3);if(b0){if(e){for(bZ in b3){if(b4.apply(b3[bZ],b0)===false){break}}}else{for(;b1<b2;){if(b4.apply(b3[b1++],b0)===false){break}}}}else{if(e){for(bZ in b3){if(b4.call(b3[bZ],bZ,b3[bZ])===false){break}}}else{for(;b1<b
2;){if(b4.call(b3[b1],b1,b3[b1++])===false){break}}}}return b3},trim:aN?function(e){return e==null?"":aN.call(e)}:function(e){return e==null?"":e.toString().replace(D,"")},makeArray:function(e,b0){var b1,bZ=b0||[];if(e!=null){b1=bG.type(e);if(e.length==null||b1==="string"||b1==="function"||b1==="regexp"||bG.isWindow(e)){am.call(bZ,e)}else{bG.merge(bZ,e)}}return bZ},inArray:function(b1,bZ,b0){var e;if(bZ){if(aK){return aK.call(bZ,b1,b0)}e=bZ.length;b0=b0?b0<0?Math.max(0,e+b0):b0:0;for(;b0<e;b0++){if(b0 in bZ&&bZ[b0]===b1){return b0}}}return -1},merge:function(b2,b0){var e=b0.length,b1=b2.length,bZ=0;if(typeof e==="number"){for(;bZ<e;bZ++){b2[b1++]=b0[bZ]}}else{while(b0[bZ]!==aB){b2[b1++]=b0[bZ++]}}b2.length=b1;return b2},grep:function(bZ,b4,e){var b3,b0=[],b1=0,b2=bZ.length;e=!!e;for(;b1<b2;b1++){b3=!!b4(bZ[b1],b1);if(e!==b3){b0.push(bZ[b1])}}return b0},map:function(e,b5,b6){var b3,b4,b2=[],b0=0,bZ=e.length,b1=e instanceof bG||bZ!==aB&&typeof bZ==="number"&&((bZ>0&&e[0]&&e[bZ-1])||bZ
===0||bG.isArray(e));if(b1){for(;b0<bZ;b0++){b3=b5(e[b0],b0,b6);if(b3!=null){b2[b2.length]=b3}}}else{for(b4 in e){b3=b5(e[b4],b4,b6);if(b3!=null){b2[b2.length]=b3}}}return b2.concat.apply([],b2)},guid:1,proxy:function(b2,b1){var b0,e,bZ;if(typeof b1==="string"){b0=b2[b1];b1=b2;b2=b0}if(!bG.isFunction(b2)){return aB}e=a4.call(arguments,2);bZ=function(){return b2.apply(b1,e.concat(a4.call(arguments)))};bZ.guid=b2.guid=b2.guid||bZ.guid||bG.guid++;return bZ},access:function(e,b4,b7,b5,b2,b8,b6){var b0,b3=b7==null,b1=0,bZ=e.length;if(b7&&typeof b7==="object"){for(b1 in b7){bG.access(e,b4,b1,b7[b1],1,b8,b5)}b2=1}else{if(b5!==aB){b0=b6===aB&&bG.isFunction(b5);if(b3){if(b0){b0=b4;b4=function(ca,b9,cb){return b0.call(bG(ca),cb)}}else{b4.call(e,b5);b4=null}}if(b4){for(;b1<bZ;b1++){b4(e[b1],b7,b0?b5.call(e[b1],b1,b4(e[b1],b7)):b5,b6)}}b2=1}}return b2?e:b3?b4.call(e):bZ?b4(e[0],b7):b8},now:function(){return(new Date()).getTime()}});bG.ready.promise=function(b2){if(!af){af=bG.Deferred();if(o.rea
dyState==="complete"||(o.readyState!=="loading"&&o.addEventListener)){setTimeout(bG.ready,1)}else{if(o.addEventListener){o.addEventListener("DOMContentLoaded",aF,false);a2.addEventListener("load",bG.ready,false)}else{o.attachEvent("onreadystatechange",aF);a2.attachEvent("onload",bG.ready);var b1=false;try{b1=a2.frameElement==null&&o.documentElement}catch(b0){}if(b1&&b1.doScroll){(function bZ(){if(!bG.isReady){try{b1.doScroll("left")}catch(b3){return setTimeout(bZ,50)}bG.ready()}})()}}}}return af.promise(b2)};bG.each("Boolean Number String Function Array Date RegExp Object".split(" "),function(bZ,e){Z["[object "+e+"]"]=e.toLowerCase()});x=bG(o);var bU={};function ac(bZ){var e=bU[bZ]={};bG.each(bZ.split(aV),function(b1,b0){e[b0]=true});return e}bG.Callbacks=function(b8){b8=typeof b8==="string"?(bU[b8]||ac(b8)):bG.extend({},b8);var b1,e,b2,b0,b3,b4,b5=[],b6=!b8.once&&[],bZ=function(b9){b1=b8.memory&&b9;e=true;b4=b0||0;b0=0;b3=b5.length;b2=true;for(;b5&&b4<b3;b4++){if(b5[b4].apply(b9[0]
,b9[1])===false&&b8.stopOnFalse){b1=false;break}}b2=false;if(b5){if(b6){if(b6.length){bZ(b6.shift())}}else{if(b1){b5=[]}else{b7.disable()}}}},b7={add:function(){if(b5){var ca=b5.length;(function b9(cb){bG.each(cb,function(cd,cc){if(bG.isFunction(cc)&&(!b8.unique||!b7.has(cc))){b5.push(cc)}else{if(cc&&cc.length){b9(cc)}}})})(arguments);if(b2){b3=b5.length}else{if(b1){b0=ca;bZ(b1)}}}return this},remove:function(){if(b5){bG.each(arguments,function(cb,b9){var ca;while((ca=bG.inArray(b9,b5,ca))>-1){b5.splice(ca,1);if(b2){if(ca<=b3){b3--}if(ca<=b4){b4--}}}})}return this},has:function(b9){return bG.inArray(b9,b5)>-1},empty:function(){b5=[];return this},disable:function(){b5=b6=b1=aB;return this},disabled:function(){return !b5},lock:function(){b6=aB;if(!b1){b7.disable()}return this},locked:function(){return !b6},fireWith:function(ca,b9){b9=b9||[];b9=[ca,b9.slice?b9.slice():b9];if(b5&&(!e||b6)){if(b2){b6.push(b9)}else{bZ(b9)}}return this},fire:function(){b7.fireWith(this,arguments);return th
is},fired:function(){return !!e}};return b7};bG.extend({Deferred:function(b0){var bZ=[["resolve","done",bG.Callbacks("once memory"),"resolved"],["reject","fail",bG.Callbacks("once memory"),"rejected"],["notify","progress",bG.Callbacks("memory")]],b1="pending",b2={state:function(){return b1},always:function(){e.done(arguments).fail(arguments);return this},then:function(){var b3=arguments;return bG.Deferred(function(b4){bG.each(bZ,function(b6,b5){var b8=b5[0],b7=b3[b6];e[b5[1]](bG.isFunction(b7)?function(){var b9=b7.apply(this,arguments);if(b9&&bG.isFunction(b9.promise)){b9.promise().done(b4.resolve).fail(b4.reject).progress(b4.notify)}else{b4[b8+"With"](this===e?b4:this,[b9])}}:b4[b8])});b3=null}).promise()},promise:function(b3){return typeof b3==="object"?bG.extend(b3,b2):b2}},e={};b2.pipe=b2.then;bG.each(bZ,function(b4,b3){var b6=b3[2],b5=b3[3];b2[b3[1]]=b6.add;if(b5){b6.add(function(){b1=b5},bZ[b4^1][2].disable,bZ[2][2].lock)}e[b3[0]]=b6.fire;e[b3[0]+"With"]=b6.fireWith});b2.promi
se(e);if(b0){b0.call(e,e)}return e},when:function(b2){var b0=0,b4=a4.call(arguments),e=b4.length,bZ=e!==1||(b2&&bG.isFunction(b2.promise))?e:0,b7=bZ===1?b2:bG.Deferred(),b1=function(b9,ca,b8){return function(cb){ca[b9]=this;b8[b9]=arguments.length>1?a4.call(arguments):cb;if(b8===b6){b7.notifyWith(ca,b8)}else{if(!(--bZ)){b7.resolveWith(ca,b8)}}}},b6,b3,b5;if(e>1){b6=new Array(e);b3=new Array(e);b5=new Array(e);for(;b0<e;b0++){if(b4[b0]&&bG.isFunction(b4[b0].promise)){b4[b0].promise().done(b1(b0,b5,b4)).fail(b7.reject).progress(b1(b0,b3,b6))}else{--bZ}}}if(!bZ){b7.resolveWith(b5,b4)}return b7.promise()}});bG.support=(function(){var cb,ca,b8,b9,b2,b7,b6,b4,b3,b1,bZ,b0=o.createElement("div");b0.setAttribute("className","t");b0.innerHTML=" <link/><table></table><a href='/a'>a</a><input type='checkbox'/>";ca=b0.getElementsByTagName("*");b8=b0.getElementsByTagName("a")[0];b8.style.cssText="top:1px;float:left;opacity:.5";if(!ca||!ca.length||!b8){return{}}b9=o.createElement("select");b2=b9.
appendChild(o.createElement("option"));b7=b0.getElementsByTagName("input")[0];cb={leadingWhitespace:(b0.firstChild.nodeType===3),tbody:!b0.getElementsByTagName("tbody").length,htmlSerialize:!!b0.getElementsByTagName("link").length,style:/top/.test(b8.getAttribute("style")),hrefNormalized:(b8.getAttribute("href")==="/a"),opacity:/^0.5/.test(b8.style.opacity),cssFloat:!!b8.style.cssFloat,checkOn:(b7.value==="on"),optSelected:b2.selected,getSetAttribute:b0.className!=="t",enctype:!!o.createElement("form").enctype,html5Clone:o.createElement("nav").cloneNode(true).outerHTML!=="<:nav></:nav>",boxModel:(o.compatMode==="CSS1Compat"),submitBubbles:true,changeBubbles:true,focusinBubbles:false,deleteExpando:true,noCloneEvent:true,inlineBlockNeedsLayout:false,shrinkWrapBlocks:false,reliableMarginRight:true,boxSizingReliable:true,pixelPosition:false};b7.checked=true;cb.noCloneChecked=b7.cloneNode(true).checked;b9.disabled=true;cb.optDisabled=!b2.disabled;try{delete b0.test}catch(b5){cb.deleteExp
ando=false}if(!b0.addEventListener&&b0.attachEvent&&b0.fireEvent){b0.attachEvent("onclick",bZ=function(){cb.noCloneEvent=false});b0.cloneNode(true).fireEvent("onclick");b0.detachEvent("onclick",bZ)}b7=o.createElement("input");b7.value="t";b7.setAttribute("type","radio");cb.radioValue=b7.value==="t";b7.setAttribute("checked","checked");b7.setAttribute("name","t");b0.appendChild(b7);b6=o.createDocumentFragment();b6.appendChild(b0.lastChild);cb.checkClone=b6.cloneNode(true).cloneNode(true).lastChild.checked;cb.appendChecked=b7.checked;b6.removeChild(b7);b6.appendChild(b0);if(b0.attachEvent){for(b3 in {submit:true,change:true,focusin:true}){b4="on"+b3;b1=(b4 in b0);if(!b1){b0.setAttribute(b4,"return;");b1=(typeof b0[b4]==="function")}cb[b3+"Bubbles"]=b1}}bG(function(){var cc,cg,ce,cf,cd="padding:0;margin:0;border:0;display:block;overflow:hidden;",e=o.getElementsByTagName("body")[0];if(!e){return}cc=o.createElement("div");cc.style.cssText="visibility:hidden;border:0;width:0;height:0;posi
tion:static;top:0;margin-top:1px";e.insertBefore(cc,e.firstChild);cg=o.createElement("div");cc.appendChild(cg);cg.innerHTML="<table><tr><td></td><td>t</td></tr></table>";ce=cg.getElementsByTagName("td");ce[0].style.cssText="padding:0;margin:0;border:0;display:none";b1=(ce[0].offsetHeight===0);ce[0].style.display="";ce[1].style.display="none";cb.reliableHiddenOffsets=b1&&(ce[0].offsetHeight===0);cg.innerHTML="";cg.style.cssText="box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box;padding:1px;border:1px;display:block;width:4px;margin-top:1%;position:absolute;top:1%;";cb.boxSizing=(cg.offsetWidth===4);cb.doesNotIncludeMarginInBodyOffset=(e.offsetTop!==1);if(a2.getComputedStyle){cb.pixelPosition=(a2.getComputedStyle(cg,null)||{}).top!=="1%";cb.boxSizingReliable=(a2.getComputedStyle(cg,null)||{width:"4px"}).width==="4px";cf=o.createElement("div");cf.style.cssText=cg.style.cssText=cd;cf.style.marginRight=cf.style.width="0";cg.style.width="1px";cg.appendChild(cf)
;cb.reliableMarginRight=!parseFloat((a2.getComputedStyle(cf,null)||{}).marginRight)}if(typeof cg.style.zoom!=="undefined"){cg.innerHTML="";cg.style.cssText=cd+"width:1px;padding:1px;display:inline;zoom:1";cb.inlineBlockNeedsLayout=(cg.offsetWidth===3);cg.style.display="block";cg.style.overflow="visible";cg.innerHTML="<div></div>";cg.firstChild.style.width="5px";cb.shrinkWrapBlocks=(cg.offsetWidth!==3);cc.style.zoom=1}e.removeChild(cc);cc=cg=ce=cf=null});b6.removeChild(b0);ca=b8=b9=b2=b7=b6=b0=null;return cb})();var bt=/^(?:\{.*\}|\[.*\])$/,aL=/([A-Z])/g;bG.extend({cache:{},deletedIds:[],uuid:0,expando:"jQuery"+(bG.fn.jquery+Math.random()).replace(/\D/g,""),noData:{embed:true,object:"clsid:D27CDB6E-AE6D-11cf-96B8-444553540000",applet:true},hasData:function(e){e=e.nodeType?bG.cache[e[bG.expando]]:e[bG.expando];return !!e&&!O(e)},data:function(b1,bZ,b3,b2){if(!bG.acceptData(b1)){return}var b4,b6,b7=bG.expando,b5=typeof bZ==="string",b8=b1.nodeType,e=b8?bG.cache:b1,b0=b8?b1[b7]:b1[b7]&&
b7;if((!b0||!e[b0]||(!b2&&!e[b0].data))&&b5&&b3===aB){return}if(!b0){if(b8){b1[b7]=b0=bG.deletedIds.pop()||++bG.uuid}else{b0=b7}}if(!e[b0]){e[b0]={};if(!b8){e[b0].toJSON=bG.noop}}if(typeof bZ==="object"||typeof bZ==="function"){if(b2){e[b0]=bG.extend(e[b0],bZ)}else{e[b0].data=bG.extend(e[b0].data,bZ)}}b4=e[b0];if(!b2){if(!b4.data){b4.data={}}b4=b4.data}if(b3!==aB){b4[bG.camelCase(bZ)]=b3}if(b5){b6=b4[bZ];if(b6==null){b6=b4[bG.camelCase(bZ)]}}else{b6=b4}return b6},removeData:function(b1,bZ,b2){if(!bG.acceptData(b1)){return}var b5,b4,b3,b6=b1.nodeType,e=b6?bG.cache:b1,b0=b6?b1[bG.expando]:bG.expando;if(!e[b0]){return}if(bZ){b5=b2?e[b0]:e[b0].data;if(b5){if(!bG.isArray(bZ)){if(bZ in b5){bZ=[bZ]}else{bZ=bG.camelCase(bZ);if(bZ in b5){bZ=[bZ]}else{bZ=bZ.split(" ")}}}for(b4=0,b3=bZ.length;b4<b3;b4++){delete b5[bZ[b4]]}if(!(b2?O:bG.isEmptyObject)(b5)){return}}}if(!b2){delete e[b0].data;if(!O(e[b0])){return}}if(b6){bG.cleanData([b1],true)}else{if(bG.support.deleteExpando||e!=e.window){delete
e[b0]}else{e[b0]=null}}},_data:function(bZ,e,b0){return bG.data(bZ,e,b0,true)},acceptData:function(bZ){var e=bZ.nodeName&&bG.noData[bZ.nodeName.toLowerCase()];return !e||e!==true&&bZ.getAttribute("classid")===e}});bG.fn.extend({data:function(b7,b6){var b2,bZ,b5,e,b1,b0=this[0],b4=0,b3=null;if(b7===aB){if(this.length){b3=bG.data(b0);if(b0.nodeType===1&&!bG._data(b0,"parsedAttrs")){b5=b0.attributes;for(b1=b5.length;b4<b1;b4++){e=b5[b4].name;if(e.indexOf("data-")===0){e=bG.camelCase(e.substring(5));bv(b0,e,b3[e])}}bG._data(b0,"parsedAttrs",true)}}return b3}if(typeof b7==="object"){return this.each(function(){bG.data(this,b7)})}b2=b7.split(".",2);b2[1]=b2[1]?"."+b2[1]:"";bZ=b2[1]+"!";return bG.access(this,function(b8){if(b8===aB){b3=this.triggerHandler("getData"+bZ,[b2[0]]);if(b3===aB&&b0){b3=bG.data(b0,b7);b3=bv(b0,b7,b3)}return b3===aB&&b2[1]?this.data(b2[0]):b3}b2[1]=b8;this.each(function(){var b9=bG(this);b9.triggerHandler("setData"+bZ,b2);bG.data(this,b7,b8);b9.triggerHandler("cha
ngeData"+bZ,b2)})},null,b6,arguments.length>1,null,false)},removeData:function(e){return this.each(function(){bG.removeData(this,e)})}});function bv(b1,b0,b2){if(b2===aB&&b1.nodeType===1){var bZ="data-"+b0.replace(aL,"-$1").toLowerCase();b2=b1.getAttribute(bZ);if(typeof b2==="string"){try{b2=b2==="true"?true:b2==="false"?false:b2==="null"?null:+b2+""===b2?+b2:bt.test(b2)?bG.parseJSON(b2):b2}catch(b3){}bG.data(b1,b0,b2)}else{b2=aB}}return b2}function O(bZ){var e;for(e in bZ){if(e==="data"&&bG.isEmptyObject(bZ[e])){continue}if(e!=="toJSON"){return false}}return true}bG.extend({queue:function(b0,bZ,b1){var e;if(b0){bZ=(bZ||"fx")+"queue";e=bG._data(b0,bZ);if(b1){if(!e||bG.isArray(b1)){e=bG._data(b0,bZ,bG.makeArray(b1))}else{e.push(b1)}}return e||[]}},dequeue:function(b3,b2){b2=b2||"fx";var bZ=bG.queue(b3,b2),b1=bZ.shift(),e=bG._queueHooks(b3,b2),b0=function(){bG.dequeue(b3,b2)};if(b1==="inprogress"){b1=bZ.shift()}if(b1){if(b2==="fx"){bZ.unshift("inprogress")}delete e.stop;b1.call(b3,b0,
e)}if(!bZ.length&&e){e.empty.fire()}},_queueHooks:function(b0,bZ){var e=bZ+"queueHooks";return bG._data(b0,e)||bG._data(b0,e,{empty:bG.Callbacks("once memory").add(function(){bG.removeData(b0,bZ+"queue",true);bG.removeData(b0,e,true)})})}});bG.fn.extend({queue:function(e,bZ){var b0=2;if(typeof e!=="string"){bZ=e;e="fx";b0--}if(arguments.length<b0){return bG.queue(this[0],e)}return bZ===aB?this:this.each(function(){var b1=bG.queue(this,e,bZ);bG._queueHooks(this,e);if(e==="fx"&&b1[0]!=="inprogress"){bG.dequeue(this,e)}})},dequeue:function(e){return this.each(function(){bG.dequeue(this,e)})},delay:function(bZ,e){bZ=bG.fx?bG.fx.speeds[bZ]||bZ:bZ;e=e||"fx";return this.queue(e,function(b1,b0){var b2=setTimeout(b1,bZ);b0.stop=function(){clearTimeout(b2)}})},clearQueue:function(e){return this.queue(e||"fx",[])},promise:function(b0,b4){var bZ,b1=1,b5=bG.Deferred(),b3=this,e=this.length,b2=function(){if(!(--b1)){b5.resolveWith(b3,[b3])}};if(typeof b0!=="string"){b4=b0;b0=aB}b0=b0||"fx";while(
e--){if((bZ=bG._data(b3[e],b0+"queueHooks"))&&bZ.empty){b1++;bZ.empty.add(b2)}}b2();return b5.promise(b4)}});var a7,bV,n,bJ=/[\t\r\n]/g,ai=/\r/g,j=/^(?:button|input)$/i,aA=/^(?:button|input|object|select|textarea)$/i,E=/^a(?:rea|)$/i,M=/^(?:autofocus|autoplay|async|checked|controls|defer|disabled|hidden|loop|multiple|open|readonly|required|scoped|selected)$/i,bL=bG.support.getSetAttribute;bG.fn.extend({attr:function(e,bZ){return bG.access(this,bG.attr,e,bZ,arguments.length>1)},removeAttr:function(e){return this.each(function(){bG.removeAttr(this,e)})},prop:function(e,bZ){return bG.access(this,bG.prop,e,bZ,arguments.length>1)},removeProp:function(e){e=bG.propFix[e]||e;return this.each(function(){try{this[e]=aB;delete this[e]}catch(bZ){}})},addClass:function(b2){var b4,b0,bZ,b1,b3,b5,e;if(bG.isFunction(b2)){return this.each(function(b6){bG(this).addClass(b2.call(this,b6,this.className))})}if(b2&&typeof b2==="string"){b4=b2.split(aV);for(b0=0,bZ=this.length;b0<bZ;b0++){b1=this[b0];if(b
1.nodeType===1){if(!b1.className&&b4.length===1){b1.className=b2}else{b3=" "+b1.className+" ";for(b5=0,e=b4.length;b5<e;b5++){if(!~b3.indexOf(" "+b4[b5]+" ")){b3+=b4[b5]+" "}}b1.className=bG.trim(b3)}}}}return this},removeClass:function(b4){var b1,b2,b3,b5,bZ,b0,e;if(bG.isFunction(b4)){return this.each(function(b6){bG(this).removeClass(b4.call(this,b6,this.className))})}if((b4&&typeof b4==="string")||b4===aB){b1=(b4||"").split(aV);for(b0=0,e=this.length;b0<e;b0++){b3=this[b0];if(b3.nodeType===1&&b3.className){b2=(" "+b3.className+" ").replace(bJ," ");for(b5=0,bZ=b1.length;b5<bZ;b5++){while(b2.indexOf(" "+b1[b5]+" ")>-1){b2=b2.replace(" "+b1[b5]+" "," ")}}b3.className=b4?bG.trim(b2):""}}}return this},toggleClass:function(b1,bZ){var b0=typeof b1,e=typeof bZ==="boolean";if(bG.isFunction(b1)){return this.each(function(b2){bG(this).toggleClass(b1.call(this,b2,this.className,bZ),bZ)})}return this.each(function(){if(b0==="string"){var b4,b3=0,b2=bG(this),b5=bZ,b6=b1.split(aV);while((b4=b6[
b3++])){b5=e?b5:!b2.hasClass(b4);b2[b5?"addClass":"removeClass"](b4)}}else{if(b0==="undefined"||b0==="boolean"){if(this.className){bG._data(this,"__className__",this.className)}this.className=this.className||b1===false?"":bG._data(this,"__className__")||""}}})},hasClass:function(e){var b1=" "+e+" ",b0=0,bZ=this.length;for(;b0<bZ;b0++){if(this[b0].nodeType===1&&(" "+this[b0].className+" ").replace(bJ," ").indexOf(b1)>-1){return true}}return false},val:function(b1){var e,bZ,b2,b0=this[0];if(!arguments.length){if(b0){e=bG.valHooks[b0.type]||bG.valHooks[b0.nodeName.toLowerCase()];if(e&&"get" in e&&(bZ=e.get(b0,"value"))!==aB){return bZ}bZ=b0.value;return typeof bZ==="string"?bZ.replace(ai,""):bZ==null?"":bZ}return}b2=bG.isFunction(b1);return this.each(function(b4){var b5,b3=bG(this);if(this.nodeType!==1){return}if(b2){b5=b1.call(this,b4,b3.val())}else{b5=b1}if(b5==null){b5=""}else{if(typeof b5==="number"){b5+=""}else{if(bG.isArray(b5)){b5=bG.map(b5,function(b6){return b6==null?"":b6+""}
)}}}e=bG.valHooks[this.type]||bG.valHooks[this.nodeName.toLowerCase()];if(!e||!("set" in e)||e.set(this,b5,"value")===aB){this.value=b5}})}});bG.extend({valHooks:{option:{get:function(e){var bZ=e.attributes.value;return !bZ||bZ.specified?e.value:e.text}},select:{get:function(e){var b4,bZ,b3,b1,b2=e.selectedIndex,b5=[],b6=e.options,b0=e.type==="select-one";if(b2<0){return null}bZ=b0?b2:0;b3=b0?b2+1:b6.length;for(;bZ<b3;bZ++){b1=b6[bZ];if(b1.selected&&(bG.support.optDisabled?!b1.disabled:b1.getAttribute("disabled")===null)&&(!b1.parentNode.disabled||!bG.nodeName(b1.parentNode,"optgroup"))){b4=bG(b1).val();if(b0){return b4}b5.push(b4)}}if(b0&&!b5.length&&b6.length){return bG(b6[b2]).val()}return b5},set:function(bZ,b0){var e=bG.makeArray(b0);bG(bZ).find("option").each(function(){this.selected=bG.inArray(bG(this).val(),e)>=0});if(!e.length){bZ.selectedIndex=-1}return e}}},attrFn:{},attr:function(b4,b1,b5,b3){var b0,e,b2,bZ=b4.nodeType;if(!b4||bZ===3||bZ===8||bZ===2){return}if(b3&&bG.isF
unction(bG.fn[b1])){return bG(b4)[b1](b5)}if(typeof b4.getAttribute==="undefined"){return bG.prop(b4,b1,b5)}b2=bZ!==1||!bG.isXMLDoc(b4);if(b2){b1=b1.toLowerCase();e=bG.attrHooks[b1]||(M.test(b1)?bV:a7)}if(b5!==aB){if(b5===null){bG.removeAttr(b4,b1);return}else{if(e&&"set" in e&&b2&&(b0=e.set(b4,b5,b1))!==aB){return b0}else{b4.setAttribute(b1,""+b5);return b5}}}else{if(e&&"get" in e&&b2&&(b0=e.get(b4,b1))!==null){return b0}else{b0=b4.getAttribute(b1);return b0===null?aB:b0}}},removeAttr:function(b1,b3){var b2,b4,bZ,e,b0=0;if(b3&&b1.nodeType===1){b4=b3.split(aV);for(;b0<b4.length;b0++){bZ=b4[b0];if(bZ){b2=bG.propFix[bZ]||bZ;e=M.test(bZ);if(!e){bG.attr(b1,bZ,"")}b1.removeAttribute(bL?bZ:b2);if(e&&b2 in b1){b1[b2]=false}}}}},attrHooks:{type:{set:function(e,bZ){if(j.test(e.nodeName)&&e.parentNode){bG.error("type property can't be changed")}else{if(!bG.support.radioValue&&bZ==="radio"&&bG.nodeName(e,"input")){var b0=e.value;e.setAttribute("type",bZ);if(b0){e.value=b0}return bZ}}}},value:{
get:function(bZ,e){if(a7&&bG.nodeName(bZ,"button")){return a7.get(bZ,e)}return e in bZ?bZ.value:null},set:function(bZ,b0,e){if(a7&&bG.nodeName(bZ,"button")){return a7.set(bZ,b0,e)}bZ.value=b0}}},propFix:{tabindex:"tabIndex",readonly:"readOnly","for":"htmlFor","class":"className",maxlength:"maxLength",cellspacing:"cellSpacing",cellpadding:"cellPadding",rowspan:"rowSpan",colspan:"colSpan",usemap:"useMap",frameborder:"frameBorder",contenteditable:"contentEditable"},prop:function(b3,b1,b4){var b0,e,b2,bZ=b3.nodeType;if(!b3||bZ===3||bZ===8||bZ===2){return}b2=bZ!==1||!bG.isXMLDoc(b3);if(b2){b1=bG.propFix[b1]||b1;e=bG.propHooks[b1]}if(b4!==aB){if(e&&"set" in e&&(b0=e.set(b3,b4,b1))!==aB){return b0}else{return(b3[b1]=b4)}}else{if(e&&"get" in e&&(b0=e.get(b3,b1))!==null){return b0}else{return b3[b1]}}},propHooks:{tabIndex:{get:function(bZ){var e=bZ.getAttributeNode("tabindex");return e&&e.specified?parseInt(e.value,10):aA.test(bZ.nodeName)||E.test(bZ.nodeName)&&bZ.href?0:aB}}}});bV={get:func
tion(bZ,e){var b1,b0=bG.prop(bZ,e);return b0===true||typeof b0!=="boolean"&&(b1=bZ.getAttributeNode(e))&&b1.nodeValue!==false?e.toLowerCase():aB},set:function(bZ,b1,e){var b0;if(b1===false){bG.removeAttr(bZ,e)}else{b0=bG.propFix[e]||e;if(b0 in bZ){bZ[b0]=true}bZ.setAttribute(e,e.toLowerCase())}return e}};if(!bL){n={name:true,id:true,coords:true};a7=bG.valHooks.button={get:function(b0,bZ){var e;e=b0.getAttributeNode(bZ);return e&&(n[bZ]?e.value!=="":e.specified)?e.value:aB},set:function(b0,b1,bZ){var e=b0.getAttributeNode(bZ);if(!e){e=o.createAttribute(bZ);b0.setAttributeNode(e)}return(e.value=b1+"")}};bG.each(["width","height"],function(bZ,e){bG.attrHooks[e]=bG.extend(bG.attrHooks[e],{set:function(b0,b1){if(b1===""){b0.setAttribute(e,"auto");return b1}}})});bG.attrHooks.contenteditable={get:a7.get,set:function(bZ,b0,e){if(b0===""){b0="false"}a7.set(bZ,b0,e)}}}if(!bG.support.hrefNormalized){bG.each(["href","src","width","height"],function(bZ,e){bG.attrHooks[e]=bG.extend(bG.attrHooks[
e],{get:function(b1){var b0=b1.getAttribute(e,2);return b0===null?aB:b0}})})}if(!bG.support.style){bG.attrHooks.style={get:function(e){return e.style.cssText.toLowerCase()||aB},set:function(e,bZ){return(e.style.cssText=""+bZ)}}}if(!bG.support.optSelected){bG.propHooks.selected=bG.extend(bG.propHooks.selected,{get:function(bZ){var e=bZ.parentNode;if(e){e.selectedIndex;if(e.parentNode){e.parentNode.selectedIndex}}return null}})}if(!bG.support.enctype){bG.propFix.enctype="encoding"}if(!bG.support.checkOn){bG.each(["radio","checkbox"],function(){bG.valHooks[this]={get:function(e){return e.getAttribute("value")===null?"on":e.value}}})}bG.each(["radio","checkbox"],function(){bG.valHooks[this]=bG.extend(bG.valHooks[this],{set:function(e,bZ){if(bG.isArray(bZ)){return(e.checked=bG.inArray(bG(e).val(),bZ)>=0)}}})});var bE=/^(?:textarea|input|select)$/i,br=/^([^\.]*|)(?:\.(.+)|)$/,ba=/(?:^|\s)hover(\.\S+|)\b/,a3=/^key/,bK=/^(?:mouse|contextmenu)|click/,by=/^(?:focusinfocus|focusoutblur)$/,aq=f
unction(e){return bG.event.special.hover?e:e.replace(ba,"mouseenter$1 mouseleave$1")};bG.event={add:function(b1,b5,cc,b3,b2){var b6,b4,cd,cb,ca,b8,e,b9,bZ,b0,b7;if(b1.nodeType===3||b1.nodeType===8||!b5||!cc||!(b6=bG._data(b1))){return}if(cc.handler){bZ=cc;cc=bZ.handler;b2=bZ.selector}if(!cc.guid){cc.guid=bG.guid++}cd=b6.events;if(!cd){b6.events=cd={}}b4=b6.handle;if(!b4){b6.handle=b4=function(ce){return typeof bG!=="undefined"&&(!ce||bG.event.triggered!==ce.type)?bG.event.dispatch.apply(b4.elem,arguments):aB};b4.elem=b1}b5=bG.trim(aq(b5)).split(" ");for(cb=0;cb<b5.length;cb++){ca=br.exec(b5[cb])||[];b8=ca[1];e=(ca[2]||"").split(".").sort();b7=bG.event.special[b8]||{};b8=(b2?b7.delegateType:b7.bindType)||b8;b7=bG.event.special[b8]||{};b9=bG.extend({type:b8,origType:ca[1],data:b3,handler:cc,guid:cc.guid,selector:b2,namespace:e.join(".")},bZ);b0=cd[b8];if(!b0){b0=cd[b8]=[];b0.delegateCount=0;if(!b7.setup||b7.setup.call(b1,b3,e,b4)===false){if(b1.addEventListener){b1.addEventListener(b8
,b4,false)}else{if(b1.attachEvent){b1.attachEvent("on"+b8,b4)}}}}if(b7.add){b7.add.call(b1,b9);if(!b9.handler.guid){b9.handler.guid=cc.guid}}if(b2){b0.splice(b0.delegateCount++,0,b9)}else{b0.push(b9)}bG.event.global[b8]=true}b1=null},global:{},remove:function(b1,b6,cc,b2,b5){var cd,ce,b9,b0,bZ,b3,b4,cb,b8,e,ca,b7=bG.hasData(b1)&&bG._data(b1);if(!b7||!(cb=b7.events)){return}b6=bG.trim(aq(b6||"")).split(" ");for(cd=0;cd<b6.length;cd++){ce=br.exec(b6[cd])||[];b9=b0=ce[1];bZ=ce[2];if(!b9){for(b9 in cb){bG.event.remove(b1,b9+b6[cd],cc,b2,true)}continue}b8=bG.event.special[b9]||{};b9=(b2?b8.delegateType:b8.bindType)||b9;e=cb[b9]||[];b3=e.length;bZ=bZ?new RegExp("(^|\\.)"+bZ.split(".").sort().join("\\.(?:.*\\.|)")+"(\\.|$)"):null;for(b4=0;b4<e.length;b4++){ca=e[b4];if((b5||b0===ca.origType)&&(!cc||cc.guid===ca.guid)&&(!bZ||bZ.test(ca.namespace))&&(!b2||b2===ca.selector||b2==="**"&&ca.selector)){e.splice(b4--,1);if(ca.selector){e.delegateCount--}if(b8.remove){b8.remove.call(b1,ca)}}}if(e.le
ngth===0&&b3!==e.length){if(!b8.teardown||b8.teardown.call(b1,bZ,b7.handle)===false){bG.removeEvent(b1,b9,b7.handle)}delete cb[b9]}}if(bG.isEmptyObject(cb)){delete b7.handle;bG.removeData(b1,"events",true)}},customEvent:{getData:true,setData:true,changeData:true},trigger:function(bZ,b6,b4,cd){if(b4&&(b4.nodeType===3||b4.nodeType===8)){return}var e,b1,b7,cb,b3,b2,b9,b8,b5,cc,ca=bZ.type||bZ,b0=[];if(by.test(ca+bG.event.triggered)){return}if(ca.indexOf("!")>=0){ca=ca.slice(0,-1);b1=true}if(ca.indexOf(".")>=0){b0=ca.split(".");ca=b0.shift();b0.sort()}if((!b4||bG.event.customEvent[ca])&&!bG.event.global[ca]){return}bZ=typeof bZ==="object"?bZ[bG.expando]?bZ:new bG.Event(ca,bZ):new bG.Event(ca);bZ.type=ca;bZ.isTrigger=true;bZ.exclusive=b1;bZ.namespace=b0.join(".");bZ.namespace_re=bZ.namespace?new RegExp("(^|\\.)"+b0.join("\\.(?:.*\\.|)")+"(\\.|$)"):null;b2=ca.indexOf(":")<0?"on"+ca:"";if(!b4){e=bG.cache;for(b7 in e){if(e[b7].events&&e[b7].events[ca]){bG.event.trigger(bZ,b6,e[b7].handle.ele
m,true)}}return}bZ.result=aB;if(!bZ.target){bZ.target=b4}b6=b6!=null?bG.makeArray(b6):[];b6.unshift(bZ);b9=bG.event.special[ca]||{};if(b9.trigger&&b9.trigger.apply(b4,b6)===false){return}b5=[[b4,b9.bindType||ca]];if(!cd&&!b9.noBubble&&!bG.isWindow(b4)){cc=b9.delegateType||ca;cb=by.test(cc+ca)?b4:b4.parentNode;for(b3=b4;cb;cb=cb.parentNode){b5.push([cb,cc]);b3=cb}if(b3===(b4.ownerDocument||o)){b5.push([b3.defaultView||b3.parentWindow||a2,cc])}}for(b7=0;b7<b5.length&&!bZ.isPropagationStopped();b7++){cb=b5[b7][0];bZ.type=b5[b7][1];b8=(bG._data(cb,"events")||{})[bZ.type]&&bG._data(cb,"handle");if(b8){b8.apply(cb,b6)}b8=b2&&cb[b2];if(b8&&bG.acceptData(cb)&&b8.apply(cb,b6)===false){bZ.preventDefault()}}bZ.type=ca;if(!cd&&!bZ.isDefaultPrevented()){if((!b9._default||b9._default.apply(b4.ownerDocument,b6)===false)&&!(ca==="click"&&bG.nodeName(b4,"a"))&&bG.acceptData(b4)){if(b2&&b4[ca]&&((ca!=="focus"&&ca!=="blur")||bZ.target.offsetWidth!==0)&&!bG.isWindow(b4)){b3=b4[b2];if(b3){b4[b2]=null}bG
.event.triggered=ca;b4[ca]();bG.event.triggered=aB;if(b3){b4[b2]=b3}}}}return bZ.result},dispatch:function(cb){cb=bG.event.fix(cb||a2.event);var cd,ca,b2,b4,ce,cc,b5,b0,e,b9,cf,b7=((bG._data(this,"events")||{})[cb.type]||[]),b6=b7.delegateCount,b1=[].slice.call(arguments),b8=!cb.exclusive&&!cb.namespace,b3=bG.event.special[cb.type]||{},bZ=[];b1[0]=cb;cb.delegateTarget=this;if(b3.preDispatch&&b3.preDispatch.call(this,cb)===false){return}if(b6&&!(cb.button&&cb.type==="click")){b4=bG(this);b4.context=this;for(b2=cb.target;b2!=this;b2=b2.parentNode||this){if(b2.disabled!==true||cb.type!=="click"){cc={};b0=[];b4[0]=b2;for(cd=0;cd<b6;cd++){e=b7[cd];b9=e.selector;if(cc[b9]===aB){cc[b9]=b4.is(b9)}if(cc[b9]){b0.push(e)}}if(b0.length){bZ.push({elem:b2,matches:b0})}}}}if(b7.length>b6){bZ.push({elem:this,matches:b7.slice(b6)})}for(cd=0;cd<bZ.length&&!cb.isPropagationStopped();cd++){b5=bZ[cd];cb.currentTarget=b5.elem;for(ca=0;ca<b5.matches.length&&!cb.isImmediatePropagationStopped();ca++){e=b5.m
atches[ca];if(b8||(!cb.namespace&&!e.namespace)||cb.namespace_re&&cb.namespace_re.test(e.namespace)){cb.data=e.data;cb.handleObj=e;ce=((bG.event.special[e.origType]||{}).handle||e.handler).apply(b5.elem,b1);if(ce!==aB){cb.result=ce;if(ce===false){cb.preventDefault();cb.stopPropagation()}}}}}if(b3.postDispatch){b3.postDispatch.call(this,cb)}return cb.result},props:"attrChange attrName relatedNode srcElement altKey bubbles cancelable ctrlKey currentTarget eventPhase metaKey relatedTarget shiftKey target timeStamp view which".split(" "),fixHooks:{},keyHooks:{props:"char charCode key keyCode".split(" "),filter:function(bZ,e){if(bZ.which==null){bZ.which=e.charCode!=null?e.charCode:e.keyCode}return bZ}},mouseHooks:{props:"button buttons clientX clientY fromElement offsetX offsetY pageX pageY screenX screenY toElement".split(" "),filter:function(b1,b0){var b2,b3,e,bZ=b0.button,b4=b0.fromElement;if(b1.pageX==null&&b0.clientX!=null){b2=b1.target.ownerDocument||o;b3=b2.documentElement;e=b2.bo
dy;b1.pageX=b0.clientX+(b3&&b3.scrollLeft||e&&e.scrollLeft||0)-(b3&&b3.clientLeft||e&&e.clientLeft||0);b1.pageY=b0.clientY+(b3&&b3.scrollTop||e&&e.scrollTop||0)-(b3&&b3.clientTop||e&&e.clientTop||0)}if(!b1.relatedTarget&&b4){b1.relatedTarget=b4===b1.target?b0.toElement:b4}if(!b1.which&&bZ!==aB){b1.which=(bZ&1?1:(bZ&2?3:(bZ&4?2:0)))}return b1}},fix:function(b0){if(b0[bG.expando]){return b0}var bZ,b3,e=b0,b1=bG.event.fixHooks[b0.type]||{},b2=b1.props?this.props.concat(b1.props):this.props;b0=bG.Event(e);for(bZ=b2.length;bZ;){b3=b2[--bZ];b0[b3]=e[b3]}if(!b0.target){b0.target=e.srcElement||o}if(b0.target.nodeType===3){b0.target=b0.target.parentNode}b0.metaKey=!!b0.metaKey;return b1.filter?b1.filter(b0,e):b0},special:{ready:{setup:bG.bindReady},load:{noBubble:true},focus:{delegateType:"focusin"},blur:{delegateType:"focusout"},beforeunload:{setup:function(b0,bZ,e){if(bG.isWindow(this)){this.onbeforeunload=e}},teardown:function(bZ,e){if(this.onbeforeunload===e){this.onbeforeunload=null}}}}
,simulate:function(b0,b2,b1,bZ){var b3=bG.extend(new bG.Event(),b1,{type:b0,isSimulated:true,originalEvent:{}});if(bZ){bG.event.trigger(b3,null,b2)}else{bG.event.dispatch.call(b2,b3)}if(b3.isDefaultPrevented()){b1.preventDefault()}}};bG.event.handle=bG.event.dispatch;bG.removeEvent=o.removeEventListener?function(bZ,e,b0){if(bZ.removeEventListener){bZ.removeEventListener(e,b0,false)}}:function(b0,bZ,b1){var e="on"+bZ;if(b0.detachEvent){if(typeof b0[e]==="undefined"){b0[e]=null}b0.detachEvent(e,b1)}};bG.Event=function(bZ,e){if(!(this instanceof bG.Event)){return new bG.Event(bZ,e)}if(bZ&&bZ.type){this.originalEvent=bZ;this.type=bZ.type;this.isDefaultPrevented=(bZ.defaultPrevented||bZ.returnValue===false||bZ.getPreventDefault&&bZ.getPreventDefault())?R:X}else{this.type=bZ}if(e){bG.extend(this,e)}this.timeStamp=bZ&&bZ.timeStamp||bG.now();this[bG.expando]=true};function X(){return false}function R(){return true}bG.Event.prototype={preventDefault:function(){this.isDefaultPrevented=R;var b
Z=this.originalEvent;if(!bZ){return}if(bZ.preventDefault){bZ.preventDefault()}else{bZ.returnValue=false}},stopPropagation:function(){this.isPropagationStopped=R;var bZ=this.originalEvent;if(!bZ){return}if(bZ.stopPropagation){bZ.stopPropagation()}bZ.cancelBubble=true},stopImmediatePropagation:function(){this.isImmediatePropagationStopped=R;this.stopPropagation()},isDefaultPrevented:X,isPropagationStopped:X,isImmediatePropagationStopped:X};bG.each({mouseenter:"mouseover",mouseleave:"mouseout"},function(bZ,e){bG.event.special[bZ]={delegateType:e,bindType:e,handle:function(b3){var b1,b5=this,b4=b3.relatedTarget,b2=b3.handleObj,b0=b2.selector;if(!b4||(b4!==b5&&!bG.contains(b5,b4))){b3.type=b2.origType;b1=b2.handler.apply(this,arguments);b3.type=e}return b1}}});if(!bG.support.submitBubbles){bG.event.special.submit={setup:function(){if(bG.nodeName(this,"form")){return false}bG.event.add(this,"click._submit keypress._submit",function(b1){var b0=b1.target,bZ=bG.nodeName(b0,"input")||bG.nodeN
ame(b0,"button")?b0.form:aB;if(bZ&&!bG._data(bZ,"_submit_attached")){bG.event.add(bZ,"submit._submit",function(e){e._submit_bubble=true});bG._data(bZ,"_submit_attached",true)}})},postDispatch:function(e){if(e._submit_bubble){delete e._submit_bubble;if(this.parentNode&&!e.isTrigger){bG.event.simulate("submit",this.parentNode,e,true)}}},teardown:function(){if(bG.nodeName(this,"form")){return false}bG.event.remove(this,"._submit")}}}if(!bG.support.changeBubbles){bG.event.special.change={setup:function(){if(bE.test(this.nodeName)){if(this.type==="checkbox"||this.type==="radio"){bG.event.add(this,"propertychange._change",function(e){if(e.originalEvent.propertyName==="checked"){this._just_changed=true}});bG.event.add(this,"click._change",function(e){if(this._just_changed&&!e.isTrigger){this._just_changed=false}bG.event.simulate("change",this,e,true)})}return false}bG.event.add(this,"beforeactivate._change",function(b0){var bZ=b0.target;if(bE.test(bZ.nodeName)&&!bG._data(bZ,"_change_attach
ed")){bG.event.add(bZ,"change._change",function(e){if(this.parentNode&&!e.isSimulated&&!e.isTrigger){bG.event.simulate("change",this.parentNode,e,true)}});bG._data(bZ,"_change_attached",true)}})},handle:function(bZ){var e=bZ.target;if(this!==e||bZ.isSimulated||bZ.isTrigger||(e.type!=="radio"&&e.type!=="checkbox")){return bZ.handleObj.handler.apply(this,arguments)}},teardown:function(){bG.event.remove(this,"._change");return bE.test(this.nodeName)}}}if(!bG.support.focusinBubbles){bG.each({focus:"focusin",blur:"focusout"},function(b1,e){var bZ=0,b0=function(b2){bG.event.simulate(e,b2.target,bG.event.fix(b2),true)};bG.event.special[e]={setup:function(){if(bZ++===0){o.addEventListener(b1,b0,true)}},teardown:function(){if(--bZ===0){o.removeEventListener(b1,b0,true)}}}})}bG.fn.extend({on:function(b0,e,b3,b2,bZ){var b4,b1;if(typeof b0==="object"){if(typeof e!=="string"){b3=b3||e;e=aB}for(b1 in b0){this.on(b1,e,b3,b0[b1],bZ)}return this}if(b3==null&&b2==null){b2=e;b3=e=aB}else{if(b2==null){
if(typeof e==="string"){b2=b3;b3=aB}else{b2=b3;b3=e;e=aB}}}if(b2===false){b2=X}else{if(!b2){return this}}if(bZ===1){b4=b2;b2=function(b5){bG().off(b5);return b4.apply(this,arguments)};b2.guid=b4.guid||(b4.guid=bG.guid++)}return this.each(function(){bG.event.add(this,b0,b2,b3,e)})},one:function(bZ,e,b1,b0){return this.on(bZ,e,b1,b0,1)},off:function(b0,e,b2){var bZ,b1;if(b0&&b0.preventDefault&&b0.handleObj){bZ=b0.handleObj;bG(b0.delegateTarget).off(bZ.namespace?bZ.origType+"."+bZ.namespace:bZ.origType,bZ.selector,bZ.handler);return this}if(typeof b0==="object"){for(b1 in b0){this.off(b1,e,b0[b1])}return this}if(e===false||typeof e==="function"){b2=e;e=aB}if(b2===false){b2=X}return this.each(function(){bG.event.remove(this,b0,b2,e)})},bind:function(e,b0,bZ){return this.on(e,null,b0,bZ)},unbind:function(e,bZ){return this.off(e,null,bZ)},live:function(e,b0,bZ){bG(this.context).on(e,this.selector,b0,bZ);return this},die:function(e,bZ){bG(this.context).off(e,this.selector||"**",bZ);return
this},delegate:function(e,bZ,b1,b0){return this.on(bZ,e,b1,b0)},undelegate:function(e,bZ,b0){return arguments.length==1?this.off(e,"**"):this.off(bZ,e||"**",b0)},trigger:function(e,bZ){return this.each(function(){bG.event.trigger(e,bZ,this)})},triggerHandler:function(e,bZ){if(this[0]){return bG.event.trigger(e,bZ,this[0],true)}},toggle:function(b1){var bZ=arguments,e=b1.guid||bG.guid++,b0=0,b2=function(b3){var b4=(bG._data(this,"lastToggle"+b1.guid)||0)%b0;bG._data(this,"lastToggle"+b1.guid,b4+1);b3.preventDefault();return bZ[b4].apply(this,arguments)||false};b2.guid=e;while(b0<bZ.length){bZ[b0++].guid=e}return this.click(b2)},hover:function(e,bZ){return this.mouseenter(e).mouseleave(bZ||e)}});bG.each(("blur focus focusin focusout load resize scroll unload click dblclick mousedown mouseup mousemove mouseover mouseout mouseenter mouseleave change select submit keydown keypress keyup error contextmenu").split(" "),function(bZ,e){bG.fn[e]=function(b1,b0){if(b0==null){b0=b1;b1=null}retu
rn arguments.length>0?this.on(e,null,b1,b0):this.trigger(e)};if(a3.test(e)){bG.event.fixHooks[e]=bG.event.keyHooks}if(bK.test(e)){bG.event.fixHooks[e]=bG.event.mouseHooks}});
+/*!
+ * Sizzle CSS Selector Engine
+ * Copyright 2012 jQuery Foundation and other contributors
+ * Released under the MIT license
+ * http://sizzlejs.com/
+ */
+(function(cO,cg){var cT,co,cf,b2,b8,b6=cO.document,b9=b6.documentElement,cw="undefined",ca=false,b7=true,ce=0,cj=[].slice,cS=[].push,cW=("sizcache"+Math.random()).replace(".",""),cz="[\\x20\\t\\r\\n\\f]",ci="(?:\\\\.|[-\\w]|[^\\x00-\\xa0])+",ch=ci.replace("w","w#"),c1="([*^$|!~]?=)",cL="\\["+cz+"*("+ci+")"+cz+"*(?:"+c1+cz+"*(?:(['\"])((?:\\\\.|[^\\\\])*?)\\3|("+ch+")|)|)"+cz+"*\\]",c2=":("+ci+")(?:\\((?:(['\"])((?:\\\\.|[^\\\\])*?)\\2|((?:[^,]|\\\\,|(?:,(?=[^\\[]*\\]))|(?:,(?=[^\\(]*\\))))*))\\)|)",cB=":(nth|eq|gt|lt|first|last|even|odd)(?:\\((\\d*)\\)|)(?=[^-]|$)",cd=cz+"*([\\x20\\t\\r\\n\\f>+~])"+cz+"*",cc="(?=[^\\x20\\t\\r\\n\\f])(?:\\\\.|"+cL+"|"+c2.replace(2,7)+"|[^\\\\(),])+",cU=new RegExp("^"+cz+"+|((?:^|[^\\\\])(?:\\\\.)*)"+cz+"+$","g"),cF=new RegExp("^"+cd),ct=new RegExp(cc+"?(?="+cz+"*,|$)","g"),cJ=new RegExp("^(?:(?!,)(?:(?:^|,)"+cz+"*"+cc+")*?|"+cz+"*(.*?))(\\)|$)"),cZ=new RegExp(cc.slice(19,-6)+"\\x20\\t\\r\\n\\f>+~])+|"+cd,"g"),cK=/^(?:#([\w\-]+)|(\w+)|\.([\w\-]+))$/,c
P=/[\x20\t\r\n\f]*[+~]/,cX=/:not\($/,cp=/h\d/i,cM=/input|select|textarea|button/i,cs=/\\(?!\\)/g,cE={ID:new RegExp("^#("+ci+")"),CLASS:new RegExp("^\\.("+ci+")"),NAME:new RegExp("^\\[name=['\"]?("+ci+")['\"]?\\]"),TAG:new RegExp("^("+ci.replace("[-","[-\\*")+")"),ATTR:new RegExp("^"+cL),PSEUDO:new RegExp("^"+c2),CHILD:new RegExp("^:(only|nth|last|first)-child(?:\\("+cz+"*(even|odd|(([+-]|)(\\d*)n|)"+cz+"*(?:([+-]|)"+cz+"*(\\d+)|))"+cz+"*\\)|)","i"),POS:new RegExp(cB,"ig"),needsContext:new RegExp("^"+cz+"*[>+~]|"+cB,"i")},cR={},cq=[],cl={},cu=[],cY=function(e){e.sizzleFilter=true;return e},b3=function(e){return function(c3){return c3.nodeName.toLowerCase()==="input"&&c3.type===e}},cr=function(e){return function(c4){var c3=c4.nodeName.toLowerCase();return(c3==="input"||c3==="button")&&c4.type===e}},cH=function(c3){var c4=false,c6=b6.createElement("div");try{c4=c3(c6)}catch(c5){}c6=null;return c4},cn=cH(function(c3){c3.innerHTML="<select></select>";var e=typeof c3.lastChild.getAttribut
e("multiple");return e!=="boolean"&&e!=="string"}),b0=cH(function(c3){c3.id=cW+0;c3.innerHTML="<a name='"+cW+"'></a><div name='"+cW+"'></div>";b9.insertBefore(c3,b9.firstChild);var e=b6.getElementsByName&&b6.getElementsByName(cW).length===2+b6.getElementsByName(cW+0).length;b8=!b6.getElementById(cW);b9.removeChild(c3);return e}),b5=cH(function(e){e.appendChild(b6.createComment(""));return e.getElementsByTagName("*").length===0}),cD=cH(function(e){e.innerHTML="<a href='#'></a>";return e.firstChild&&typeof e.firstChild.getAttribute!==cw&&e.firstChild.getAttribute("href")==="#"}),cC=cH(function(e){e.innerHTML="<div class='hidden e'></div><div class='hidden'></div>";if(!e.getElementsByClassName||e.getElementsByClassName("e").length===0){return false}e.lastChild.className="e";return e.getElementsByClassName("e").length!==1});var cN=function(c5,e,c7,da){c7=c7||[];e=e||b6;var c8,c3,c9,c4,c6=e.nodeType;if(c6!==1&&c6!==9){return[]}if(!c5||typeof c5!=="string"){return c7}c9=ck(e);if(!c9&&!da)
{if((c8=cK.exec(c5))){if((c4=c8[1])){if(c6===9){c3=e.getElementById(c4);if(c3&&c3.parentNode){if(c3.id===c4){c7.push(c3);return c7}}else{return c7}}else{if(e.ownerDocument&&(c3=e.ownerDocument.getElementById(c4))&&cA(e,c3)&&c3.id===c4){c7.push(c3);return c7}}}else{if(c8[2]){cS.apply(c7,cj.call(e.getElementsByTagName(c5),0));return c7}else{if((c4=c8[3])&&cC&&e.getElementsByClassName){cS.apply(c7,cj.call(e.getElementsByClassName(c4),0));return c7}}}}}return cV(c5,e,c7,da,c9)};var cG=cN.selectors={cacheLength:50,match:cE,order:["ID","TAG"],attrHandle:{},createPseudo:cY,find:{ID:b8?function(c5,c4,c3){if(typeof c4.getElementById!==cw&&!c3){var e=c4.getElementById(c5);return e&&e.parentNode?[e]:[]}}:function(c5,c4,c3){if(typeof c4.getElementById!==cw&&!c3){var e=c4.getElementById(c5);return e?e.id===c5||typeof e.getAttributeNode!==cw&&e.getAttributeNode("id").value===c5?[e]:cg:[]}},TAG:b5?function(e,c3){if(typeof c3.getElementsByTagName!==cw){return c3.getElementsByTagName(e)}}:function(e
,c6){var c5=c6.getElementsByTagName(e);if(e==="*"){var c7,c4=[],c3=0;for(;(c7=c5[c3]);c3++){if(c7.nodeType===1){c4.push(c7)}}return c4}return c5}},relative:{">":{dir:"parentNode",first:true}," ":{dir:"parentNode"},"+":{dir:"previousSibling",first:true},"~":{dir:"previousSibling"}},preFilter:{ATTR:function(e){e[1]=e[1].replace(cs,"");e[3]=(e[4]||e[5]||"").replace(cs,"");if(e[2]==="~="){e[3]=" "+e[3]+" "}return e.slice(0,4)},CHILD:function(e){e[1]=e[1].toLowerCase();if(e[1]==="nth"){if(!e[2]){cN.error(e[0])}e[3]=+(e[3]?e[4]+(e[5]||1):2*(e[2]==="even"||e[2]==="odd"));e[4]=+((e[6]+e[7])||e[2]==="odd")}else{if(e[2]){cN.error(e[0])}}return e},PSEUDO:function(e){var c3,c4=e[4];if(cE.CHILD.test(e[0])){return null}if(c4&&(c3=cJ.exec(c4))&&c3.pop()){e[0]=e[0].slice(0,c3[0].length-c4.length-1);c4=c3[0].slice(0,-1)}e.splice(2,3,c4||e[3]);return e}},filter:{ID:b8?function(e){e=e.replace(cs,"");return function(c3){return c3.getAttribute("id")===e}}:function(e){e=e.replace(cs,"");return function(c
4){var c3=typeof c4.getAttributeNode!==cw&&c4.getAttributeNode("id");return c3&&c3.value===e}},TAG:function(e){if(e==="*"){return function(){return true}}e=e.replace(cs,"").toLowerCase();return function(c3){return c3.nodeName&&c3.nodeName.toLowerCase()===e}},CLASS:function(e){var c3=cR[e];if(!c3){c3=cR[e]=new RegExp("(^|"+cz+")"+e+"("+cz+"|$)");cq.push(e);if(cq.length>cG.cacheLength){delete cR[cq.shift()]}}return function(c4){return c3.test(c4.className||(typeof c4.getAttribute!==cw&&c4.getAttribute("class"))||"")}},ATTR:function(c4,c3,e){if(!c3){return function(c5){return cN.attr(c5,c4)!=null}}return function(c6){var c5=cN.attr(c6,c4),c7=c5+"";if(c5==null){return c3==="!="}switch(c3){case"=":return c7===e;case"!=":return c7!==e;case"^=":return e&&c7.indexOf(e)===0;case"*=":return e&&c7.indexOf(e)>-1;case"$=":return e&&c7.substr(c7.length-e.length)===e;case"~=":return(" "+c7+" ").indexOf(e)>-1;case"|=":return c7===e||c7.substr(0,e.length+1)===e+"-"}}},CHILD:function(c3,c5,c6,c4){if(
c3==="nth"){var e=ce++;return function(da){var c7,db,c9=0,c8=da;if(c6===1&&c4===0){return true}c7=da.parentNode;if(c7&&(c7[cW]!==e||!da.sizset)){for(c8=c7.firstChild;c8;c8=c8.nextSibling){if(c8.nodeType===1){c8.sizset=++c9;if(c8===da){break}}}c7[cW]=e}db=da.sizset-c4;if(c6===0){return db===0}else{return(db%c6===0&&db/c6>=0)}}}return function(c8){var c7=c8;switch(c3){case"only":case"first":while((c7=c7.previousSibling)){if(c7.nodeType===1){return false}}if(c3==="first"){return true}c7=c8;case"last":while((c7=c7.nextSibling)){if(c7.nodeType===1){return false}}return true}}},PSEUDO:function(c6,c5,c3,e){var c4=cG.pseudos[c6]||cG.pseudos[c6.toLowerCase()];if(!c4){cN.error("unsupported pseudo: "+c6)}if(!c4.sizzleFilter){return c4}return c4(c5,c3,e)}},pseudos:{not:cY(function(e,c4,c3){var c5=cb(e.replace(cU,"$1"),c4,c3);return function(c6){return !c5(c6)}}),enabled:function(e){return e.disabled===false},disabled:function(e){return e.disabled===true},checked:function(e){var c3=e.nodeName.to
LowerCase();return(c3==="input"&&!!e.checked)||(c3==="option"&&!!e.selected)},selected:function(e){if(e.parentNode){e.parentNode.selectedIndex}return e.selected===true},parent:function(e){return !cG.pseudos.empty(e)},empty:function(c3){var e;c3=c3.firstChild;while(c3){if(c3.nodeName>"@"||(e=c3.nodeType)===3||e===4){return false}c3=c3.nextSibling}return true},contains:cY(function(e){return function(c3){return(c3.textContent||c3.innerText||bZ(c3)).indexOf(e)>-1}}),has:cY(function(e){return function(c3){return cN(e,c3).length>0}}),header:function(e){return cp.test(e.nodeName)},text:function(c4){var c3,e;return c4.nodeName.toLowerCase()==="input"&&(c3=c4.type)==="text"&&((e=c4.getAttribute("type"))==null||e.toLowerCase()===c3)},radio:b3("radio"),checkbox:b3("checkbox"),file:b3("file"),password:b3("password"),image:b3("image"),submit:cr("submit"),reset:cr("reset"),button:function(c3){var e=c3.nodeName.toLowerCase();return e==="input"&&c3.type==="button"||e==="button"},input:function(e){r
eturn cM.test(e.nodeName)},focus:function(e){var c3=e.ownerDocument;return e===c3.activeElement&&(!c3.hasFocus||c3.hasFocus())&&!!(e.type||e.href)},active:function(e){return e===e.ownerDocument.activeElement}},setFilters:{first:function(c4,c3,e){return e?c4.slice(1):[c4[0]]},last:function(c5,c4,c3){var e=c5.pop();return c3?c5:[e]},even:function(c7,c6,c5){var c4=[],c3=c5?1:0,e=c7.length;for(;c3<e;c3=c3+2){c4.push(c7[c3])}return c4},odd:function(c7,c6,c5){var c4=[],c3=c5?0:1,e=c7.length;for(;c3<e;c3=c3+2){c4.push(c7[c3])}return c4},lt:function(c4,c3,e){return e?c4.slice(+c3):c4.slice(0,+c3)},gt:function(c4,c3,e){return e?c4.slice(0,+c3+1):c4.slice(+c3+1)},eq:function(c5,c4,c3){var e=c5.splice(+c4,1);return c3?c5:e}}};cG.setFilters.nth=cG.setFilters.eq;cG.filters=cG.pseudos;if(!cD){cG.attrHandle={href:function(e){return e.getAttribute("href",2)},type:function(e){return e.getAttribute("type")}}}if(b0){cG.order.push("NAME");cG.find.NAME=function(e,c3){if(typeof c3.getElementsByName!==cw)
{return c3.getElementsByName(e)}}}if(cC){cG.order.splice(1,0,"CLASS");cG.find.CLASS=function(c4,c3,e){if(typeof c3.getElementsByClassName!==cw&&!e){return c3.getElementsByClassName(c4)}}}try{cj.call(b9.childNodes,0)[0].nodeType}catch(c0){cj=function(c3){var c4,e=[];for(;(c4=this[c3]);c3++){e.push(c4)}return e}}var ck=cN.isXML=function(e){var c3=e&&(e.ownerDocument||e).documentElement;return c3?c3.nodeName!=="HTML":false};var cA=cN.contains=b9.compareDocumentPosition?function(c3,e){return !!(c3.compareDocumentPosition(e)&16)}:b9.contains?function(c3,e){var c5=c3.nodeType===9?c3.documentElement:c3,c4=e.parentNode;return c3===c4||!!(c4&&c4.nodeType===1&&c5.contains&&c5.contains(c4))}:function(c3,e){while((e=e.parentNode)){if(e===c3){return true}}return false};var bZ=cN.getText=function(c6){var c5,c3="",c4=0,e=c6.nodeType;if(e){if(e===1||e===9||e===11){if(typeof c6.textContent==="string"){return c6.textContent}else{for(c6=c6.firstChild;c6;c6=c6.nextSibling){c3+=bZ(c6)}}}else{if(e===3||e
===4){return c6.nodeValue}}}else{for(;(c5=c6[c4]);c4++){c3+=bZ(c5)}}return c3};cN.attr=function(c5,c4){var e,c3=ck(c5);if(!c3){c4=c4.toLowerCase()}if(cG.attrHandle[c4]){return cG.attrHandle[c4](c5)}if(cn||c3){return c5.getAttribute(c4)}e=c5.getAttributeNode(c4);return e?typeof c5[c4]==="boolean"?c5[c4]?c4:null:e.specified?e.value:null:null};cN.error=function(e){throw new Error("Syntax error, unrecognized expression: "+e)};[0,0].sort(function(){return(b7=0)});if(b9.compareDocumentPosition){cf=function(c3,e){if(c3===e){ca=true;return 0}return(!c3.compareDocumentPosition||!e.compareDocumentPosition?c3.compareDocumentPosition:c3.compareDocumentPosition(e)&4)?-1:1}}else{cf=function(da,c9){if(da===c9){ca=true;return 0}else{if(da.sourceIndex&&c9.sourceIndex){return da.sourceIndex-c9.sourceIndex}}var c7,c3,c4=[],e=[],c6=da.parentNode,c8=c9.parentNode,db=c6;if(c6===c8){return b2(da,c9)}else{if(!c6){return -1}else{if(!c8){return 1}}}while(db){c4.unshift(db);db=db.parentNode}db=c8;while(db){e.
unshift(db);db=db.parentNode}c7=c4.length;c3=e.length;for(var c5=0;c5<c7&&c5<c3;c5++){if(c4[c5]!==e[c5]){return b2(c4[c5],e[c5])}}return c5===c7?b2(da,e[c5],-1):b2(c4[c5],c9,1)};b2=function(c3,e,c4){if(c3===e){return c4}var c5=c3.nextSibling;while(c5){if(c5===e){return -1}c5=c5.nextSibling}return 1}}cN.uniqueSort=function(c3){var c4,e=1;if(cf){ca=b7;c3.sort(cf);if(ca){for(;(c4=c3[e]);e++){if(c4===c3[e-1]){c3.splice(e--,1)}}}}return c3};function cm(c3,c7,c6,c4){var c5=0,e=c7.length;for(;c5<e;c5++){cN(c3,c7[c5],c6,c4)}}function cI(e,c4,c8,c9,c3,c7){var c5,c6=cG.setFilters[c4.toLowerCase()];if(!c6){cN.error(c4)}if(e||!(c5=c3)){cm(e||"*",c9,(c5=[]),c3)}return c5.length>0?c6(c5,c8,c7):[]}function cQ(dc,e,da,c4,dg){var c7,c3,c6,di,c9,dh,db,df,dd=0,de=dg.length,c5=cE.POS,c8=new RegExp("^"+c5.source+"(?!"+cz+")","i"),dj=function(){var dl=1,dk=arguments.length-2;for(;dl<dk;dl++){if(arguments[dl]===cg){c7[dl]=cg}}};for(;dd<de;dd++){c5.exec("");dc=dg[dd];di=[];c6=0;c9=c4;while((c7=c5.exec(dc))
){df=c5.lastIndex=c7.index+c7[0].length;if(df>c6){db=dc.slice(c6,c7.index);c6=df;dh=[e];if(cF.test(db)){if(c9){dh=c9}c9=c4}if((c3=cX.test(db))){db=db.slice(0,-5).replace(cF,"$&*")}if(c7.length>1){c7[0].replace(c8,dj)}c9=cI(db,c7[1],c7[2],dh,c9,c3)}}if(c9){di=di.concat(c9);if((db=dc.slice(c6))&&db!==")"){if(cF.test(db)){cm(db,di,da,c4)}else{cN(db,e,da,c4?c4.concat(c9):c9)}}else{cS.apply(da,di)}}else{cN(dc,e,da,c4)}}return de===1?da:cN.uniqueSort(da)}function b1(c8,c4,db){var dd,dc,de,c6=[],c9=0,da=cJ.exec(c8),c3=!da.pop()&&!da.pop(),df=c3&&c8.match(ct)||[""],e=cG.preFilter,c5=cG.filter,c7=!db&&c4!==b6;for(;(dc=df[c9])!=null&&c3;c9++){c6.push(dd=[]);if(c7){dc=" "+dc}while(dc){c3=false;if((da=cF.exec(dc))){dc=dc.slice(da[0].length);c3=dd.push({part:da.pop().replace(cU," "),captures:da})}for(de in c5){if((da=cE[de].exec(dc))&&(!e[de]||(da=e[de](da,c4,db)))){dc=dc.slice(da.shift().length);c3=dd.push({part:de,captures:da})}}if(!c3){break}}}if(!c3){cN.error(c8)}return c6}function cx(c6,c5,
c4){var e=c5.dir,c3=ce++;if(!c6){c6=function(c7){return c7===c4}}return c5.first?function(c8,c7){while((c8=c8[e])){if(c8.nodeType===1){return c6(c8,c7)&&c8}}}:function(c9,c8){var c7,da=c3+"."+co,db=da+"."+cT;while((c9=c9[e])){if(c9.nodeType===1){if((c7=c9[cW])===db){return c9.sizset}else{if(typeof c7==="string"&&c7.indexOf(da)===0){if(c9.sizset){return c9}}else{c9[cW]=db;if(c6(c9,c8)){c9.sizset=true;return c9}c9.sizset=false}}}}}}function cv(e,c3){return e?function(c6,c5){var c4=c3(c6,c5);return c4&&e(c4===true?c6:c4,c5)}:c3}function cy(c7,c5,e){var c4,c6,c3=0;for(;(c4=c7[c3]);c3++){if(cG.relative[c4.part]){c6=cx(c6,cG.relative[c4.part],c5)}else{c4.captures.push(c5,e);c6=cv(c6,cG.filter[c4.part].apply(null,c4.captures))}}return c6}function b4(e){return function(c5,c4){var c6,c3=0;for(;(c6=e[c3]);c3++){if(c6(c5,c4)){return true}}return false}}var cb=cN.compile=function(e,c5,c3){var c8,c7,c4,c6=cl[e];if(c6&&c6.context===c5){return c6}c7=b1(e,c5,c3);for(c4=0;(c8=c7[c4]);c4++){c7[c4]=cy
(c8,c5,c3)}c6=cl[e]=b4(c7);c6.context=c5;c6.runs=c6.dirruns=0;cu.push(e);if(cu.length>cG.cacheLength){delete cl[cu.shift()]}return c6};cN.matches=function(c3,e){return cN(c3,null,null,e)};cN.matchesSelector=function(e,c3){return cN(c3,null,null,[e]).length>0};var cV=function(c6,c3,c8,dc,db){c6=c6.replace(cU,"$1");var e,dd,c9,de,c4,c5,dg,dh,c7,da=c6.match(ct),df=c6.match(cZ),di=c3.nodeType;if(cE.POS.test(c6)){return cQ(c6,c3,c8,dc,da)}if(dc){e=cj.call(dc,0)}else{if(da&&da.length===1){if(df.length>1&&di===9&&!db&&(da=cE.ID.exec(df[0]))){c3=cG.find.ID(da[1],c3,db)[0];if(!c3){return c8}c6=c6.slice(df.shift().length)}dh=((da=cP.exec(df[0]))&&!da.index&&c3.parentNode)||c3;c7=df.pop();c5=c7.split(":not")[0];for(c9=0,de=cG.order.length;c9<de;c9++){dg=cG.order[c9];if((da=cE[dg].exec(c5))){e=cG.find[dg]((da[1]||"").replace(cs,""),dh,db);if(e==null){continue}if(c5===c7){c6=c6.slice(0,c6.length-c7.length)+c5.replace(cE[dg],"");if(!c6){cS.apply(c8,cj.call(e,0))}}break}}}}if(c6){dd=cb(c6,c3,db);c
o=dd.dirruns++;if(e==null){e=cG.find.TAG("*",(cP.test(c6)&&c3.parentNode)||c3)}for(c9=0;(c4=e[c9]);c9++){cT=dd.runs++;if(dd(c4,c3)){c8.push(c4)}}}return c8};if(b6.querySelectorAll){(function(){var c7,c8=cV,c6=/'|\\/g,c4=/\=[\x20\t\r\n\f]*([^'"\]]*)[\x20\t\r\n\f]*\]/g,c3=[],e=[":active"],c5=b9.matchesSelector||b9.mozMatchesSelector||b9.webkitMatchesSelector||b9.oMatchesSelector||b9.msMatchesSelector;cH(function(c9){c9.innerHTML="<select><option selected></option></select>";if(!c9.querySelectorAll("[selected]").length){c3.push("\\["+cz+"*(?:checked|disabled|ismap|multiple|readonly|selected|value)")}if(!c9.querySelectorAll(":checked").length){c3.push(":checked")}});cH(function(c9){c9.innerHTML="<p test=''></p>";if(c9.querySelectorAll("[test^='']").length){c3.push("[*^$]="+cz+"*(?:\"\"|'')")}c9.innerHTML="<input type='hidden'>";if(!c9.querySelectorAll(":enabled").length){c3.push(":enabled",":disabled")}});c3=c3.length&&new RegExp(c3.join("|"));cV=function(de,da,df,dh,dg){if(!dh&&!dg&&(!
c3||!c3.test(de))){if(da.nodeType===9){try{cS.apply(df,cj.call(da.querySelectorAll(de),0));return df}catch(dd){}}else{if(da.nodeType===1&&da.nodeName.toLowerCase()!=="object"){var dc=da.getAttribute("id"),c9=dc||cW,db=cP.test(de)&&da.parentNode||da;if(dc){c9=c9.replace(c6,"\\$&")}else{da.setAttribute("id",c9)}try{cS.apply(df,cj.call(db.querySelectorAll(de.replace(ct,"[id='"+c9+"'] $&")),0));return df}catch(dd){}finally{if(!dc){da.removeAttribute("id")}}}}}return c8(de,da,df,dh,dg)};if(c5){cH(function(da){c7=c5.call(da,"div");try{c5.call(da,"[test!='']:sizzle");e.push(cG.match.PSEUDO)}catch(c9){}});e=new RegExp(e.join("|"));cN.matchesSelector=function(da,dc){dc=dc.replace(c4,"='$1']");if(!ck(da)&&!e.test(dc)&&(!c3||!c3.test(dc))){try{var c9=c5.call(da,dc);if(c9||c7||da.document&&da.document.nodeType!==11){return c9}}catch(db){}}return cN(dc,null,null,[da]).length>0}}})()}cN.attr=bG.attr;bG.find=cN;bG.expr=cN.selectors;bG.expr[":"]=bG.expr.pseudos;bG.unique=cN.uniqueSort;bG.text=cN.ge
tText;bG.isXMLDoc=cN.isXML;bG.contains=cN.contains})(a2);var ag=/Until$/,bq=/^(?:parents|prev(?:Until|All))/,al=/^.[^:#\[\.,]*$/,z=bG.expr.match.needsContext,bu={children:true,contents:true,next:true,prev:true};bG.fn.extend({find:function(e){var b2,bZ,b4,b5,b3,b1,b0=this;if(typeof e!=="string"){return bG(e).filter(function(){for(b2=0,bZ=b0.length;b2<bZ;b2++){if(bG.contains(b0[b2],this)){return true}}})}b1=this.pushStack("","find",e);for(b2=0,bZ=this.length;b2<bZ;b2++){b4=b1.length;bG.find(e,this[b2],b1);if(b2>0){for(b5=b4;b5<b1.length;b5++){for(b3=0;b3<b4;b3++){if(b1[b3]===b1[b5]){b1.splice(b5--,1);break}}}}}return b1},has:function(b1){var b0,bZ=bG(b1,this),e=bZ.length;return this.filter(function(){for(b0=0;b0<e;b0++){if(bG.contains(this,bZ[b0])){return true}}})},not:function(e){return this.pushStack(aM(this,e,false),"not",e)},filter:function(e){return this.pushStack(aM(this,e,true),"filter",e)},is:function(e){return !!e&&(typeof e==="string"?z.test(e)?bG(e,this.context).index(this[
0])>=0:bG.filter(e,this).length>0:this.filter(e).length>0)},closest:function(b2,b1){var b3,b0=0,e=this.length,bZ=[],b4=z.test(b2)||typeof b2!=="string"?bG(b2,b1||this.context):0;for(;b0<e;b0++){b3=this[b0];while(b3&&b3.ownerDocument&&b3!==b1&&b3.nodeType!==11){if(b4?b4.index(b3)>-1:bG.find.matchesSelector(b3,b2)){bZ.push(b3);break}b3=b3.parentNode}}bZ=bZ.length>1?bG.unique(bZ):bZ;return this.pushStack(bZ,"closest",b2)},index:function(e){if(!e){return(this[0]&&this[0].parentNode)?this.prevAll().length:-1}if(typeof e==="string"){return bG.inArray(this[0],bG(e))}return bG.inArray(e.jquery?e[0]:e,this)},add:function(e,bZ){var b1=typeof e==="string"?bG(e,bZ):bG.makeArray(e&&e.nodeType?[e]:e),b0=bG.merge(this.get(),b1);return this.pushStack(aR(b1[0])||aR(b0[0])?b0:bG.unique(b0))},addBack:function(e){return this.add(e==null?this.prevObject:this.prevObject.filter(e))}});bG.fn.andSelf=bG.fn.addBack;function aR(e){return !e||!e.parentNode||e.parentNode.nodeType===11}function aY(bZ,e){do{bZ=bZ
[e]}while(bZ&&bZ.nodeType!==1);return bZ}bG.each({parent:function(bZ){var e=bZ.parentNode;return e&&e.nodeType!==11?e:null},parents:function(e){return bG.dir(e,"parentNode")},parentsUntil:function(bZ,e,b0){return bG.dir(bZ,"parentNode",b0)},next:function(e){return aY(e,"nextSibling")},prev:function(e){return aY(e,"previousSibling")},nextAll:function(e){return bG.dir(e,"nextSibling")},prevAll:function(e){return bG.dir(e,"previousSibling")},nextUntil:function(bZ,e,b0){return bG.dir(bZ,"nextSibling",b0)},prevUntil:function(bZ,e,b0){return bG.dir(bZ,"previousSibling",b0)},siblings:function(e){return bG.sibling((e.parentNode||{}).firstChild,e)},children:function(e){return bG.sibling(e.firstChild)},contents:function(e){return bG.nodeName(e,"iframe")?e.contentDocument||e.contentWindow.document:bG.merge([],e.childNodes)}},function(e,bZ){bG.fn[e]=function(b2,b0){var b1=bG.map(this,bZ,b2);if(!ag.test(e)){b0=b2}if(b0&&typeof b0==="string"){b1=bG.filter(b0,b1)}b1=this.length>1&&!bu[e]?bG.unique
(b1):b1;if(this.length>1&&bq.test(e)){b1=b1.reverse()}return this.pushStack(b1,e,a4.call(arguments).join(","))}});bG.extend({filter:function(b0,e,bZ){if(bZ){b0=":not("+b0+")"}return e.length===1?bG.find.matchesSelector(e[0],b0)?[e[0]]:[]:bG.find.matches(b0,e)},dir:function(b0,bZ,b2){var e=[],b1=b0[bZ];while(b1&&b1.nodeType!==9&&(b2===aB||b1.nodeType!==1||!bG(b1).is(b2))){if(b1.nodeType===1){e.push(b1)}b1=b1[bZ]}return e},sibling:function(b0,bZ){var e=[];for(;b0;b0=b0.nextSibling){if(b0.nodeType===1&&b0!==bZ){e.push(b0)}}return e}});function aM(b1,b0,e){b0=b0||0;if(bG.isFunction(b0)){return bG.grep(b1,function(b3,b2){var b4=!!b0.call(b3,b2,b3);return b4===e})}else{if(b0.nodeType){return bG.grep(b1,function(b3,b2){return(b3===b0)===e})}else{if(typeof b0==="string"){var bZ=bG.grep(b1,function(b2){return b2.nodeType===1});if(al.test(b0)){return bG.filter(b0,bZ,!e)}else{b0=bG.filter(b0,bZ)}}}}return bG.grep(b1,function(b3,b2){return(bG.inArray(b3,b0)>=0)===e})}function B(e){var b0=c.spli
t("|"),bZ=e.createDocumentFragment();if(bZ.createElement){while(b0.length){bZ.createElement(b0.pop())}}return bZ}var c="abbr|article|aside|audio|bdi|canvas|data|datalist|details|figcaption|figure|footer|header|hgroup|mark|meter|nav|output|progress|section|summary|time|video",av=/ jQuery\d+="(?:null|\d+)"/g,bY=/^\s+/,ay=/<(?!area|br|col|embed|hr|img|input|link|meta|param)(([\w:]+)[^>]*)\/>/gi,p=/<([\w:]+)/,bT=/<tbody/i,J=/<|&#?\w+;/,aj=/<(?:script|style|link)/i,ap=/<(?:script|object|embed|option|style)/i,K=new RegExp("<(?:"+c+")[\\s/>]","i"),aE=/^(?:checkbox|radio)$/,bR=/checked\s*(?:[^=]|=\s*.checked.)/i,bw=/\/(java|ecma)script/i,aH=/^\s*<!(?:\[CDATA\[|\-\-)|[\]\-]{2}>\s*$/g,T={option:[1,"<select multiple='multiple'>","</select>"],legend:[1,"<fieldset>","</fieldset>"],thead:[1,"<table>","</table>"],tr:[2,"<table><tbody>","</tbody></table>"],td:[3,"<table><tbody><tr>","</tr></tbody></table>"],col:[2,"<table><tbody></tbody><colgroup>","</colgroup></table>"],area:[1,"<map>","</map>"],_
default:[0,"",""]},aQ=B(o),l=aQ.appendChild(o.createElement("div"));T.optgroup=T.option;T.tbody=T.tfoot=T.colgroup=T.caption=T.thead;T.th=T.td;if(!bG.support.htmlSerialize){T._default=[1,"X<div>","</div>"]}bG.fn.extend({text:function(e){return bG.access(this,function(bZ){return bZ===aB?bG.text(this):this.empty().append((this[0]&&this[0].ownerDocument||o).createTextNode(bZ))},null,e,arguments.length)},wrapAll:function(e){if(bG.isFunction(e)){return this.each(function(b0){bG(this).wrapAll(e.call(this,b0))})}if(this[0]){var bZ=bG(e,this[0].ownerDocument).eq(0).clone(true);if(this[0].parentNode){bZ.insertBefore(this[0])}bZ.map(function(){var b0=this;while(b0.firstChild&&b0.firstChild.nodeType===1){b0=b0.firstChild}return b0}).append(this)}return this},wrapInner:function(e){if(bG.isFunction(e)){return this.each(function(bZ){bG(this).wrapInner(e.call(this,bZ))})}return this.each(function(){var bZ=bG(this),b0=bZ.contents();if(b0.length){b0.wrapAll(e)}else{bZ.append(e)}})},wrap:function(e){
var bZ=bG.isFunction(e);return this.each(function(b0){bG(this).wrapAll(bZ?e.call(this,b0):e)})},unwrap:function(){return this.parent().each(function(){if(!bG.nodeName(this,"body")){bG(this).replaceWith(this.childNodes)}}).end()},append:function(){return this.domManip(arguments,true,function(e){if(this.nodeType===1||this.nodeType===11){this.appendChild(e)}})},prepend:function(){return this.domManip(arguments,true,function(e){if(this.nodeType===1||this.nodeType===11){this.insertBefore(e,this.firstChild)}})},before:function(){if(!aR(this[0])){return this.domManip(arguments,false,function(bZ){this.parentNode.insertBefore(bZ,this)})}if(arguments.length){var e=bG.clean(arguments);return this.pushStack(bG.merge(e,this),"before",this.selector)}},after:function(){if(!aR(this[0])){return this.domManip(arguments,false,function(bZ){this.parentNode.insertBefore(bZ,this.nextSibling)})}if(arguments.length){var e=bG.clean(arguments);return this.pushStack(bG.merge(this,e),"after",this.selector)}},re
move:function(e,b1){var b0,bZ=0;for(;(b0=this[bZ])!=null;bZ++){if(!e||bG.filter(e,[b0]).length){if(!b1&&b0.nodeType===1){bG.cleanData(b0.getElementsByTagName("*"));bG.cleanData([b0])}if(b0.parentNode){b0.parentNode.removeChild(b0)}}}return this},empty:function(){var bZ,e=0;for(;(bZ=this[e])!=null;e++){if(bZ.nodeType===1){bG.cleanData(bZ.getElementsByTagName("*"))}while(bZ.firstChild){bZ.removeChild(bZ.firstChild)}}return this},clone:function(bZ,e){bZ=bZ==null?false:bZ;e=e==null?bZ:e;return this.map(function(){return bG.clone(this,bZ,e)})},html:function(e){return bG.access(this,function(b2){var b1=this[0]||{},b0=0,bZ=this.length;if(b2===aB){return b1.nodeType===1?b1.innerHTML.replace(av,""):aB}if(typeof b2==="string"&&!aj.test(b2)&&(bG.support.htmlSerialize||!K.test(b2))&&(bG.support.leadingWhitespace||!bY.test(b2))&&!T[(p.exec(b2)||["",""])[1].toLowerCase()]){b2=b2.replace(ay,"<$1></$2>");try{for(;b0<bZ;b0++){b1=this[b0]||{};if(b1.nodeType===1){bG.cleanData(b1.getElementsByTagName("
*"));b1.innerHTML=b2}}b1=0}catch(b3){}}if(b1){this.empty().append(b2)}},null,e,arguments.length)},replaceWith:function(e){if(!aR(this[0])){if(bG.isFunction(e)){return this.each(function(b1){var b0=bG(this),bZ=b0.html();b0.replaceWith(e.call(this,b1,bZ))})}if(typeof e!=="string"){e=bG(e).detach()}return this.each(function(){var b0=this.nextSibling,bZ=this.parentNode;bG(this).remove();if(b0){bG(b0).before(e)}else{bG(bZ).append(e)}})}return this.length?this.pushStack(bG(bG.isFunction(e)?e():e),"replaceWith",e):this},detach:function(e){return this.remove(e,true)},domManip:function(b4,b8,b7){b4=[].concat.apply([],b4);var b0,b2,b3,b6,b1=0,b5=b4[0],bZ=[],e=this.length;if(!bG.support.checkClone&&e>1&&typeof b5==="string"&&bR.test(b5)){return this.each(function(){bG(this).domManip(b4,b8,b7)})}if(bG.isFunction(b5)){return this.each(function(ca){var b9=bG(this);b4[0]=b5.call(this,ca,b8?b9.html():aB);b9.domManip(b4,b8,b7)})}if(this[0]){b0=bG.buildFragment(b4,this,bZ);b3=b0.fragment;b2=b3.firstC
hild;if(b3.childNodes.length===1){b3=b2}if(b2){b8=b8&&bG.nodeName(b2,"tr");for(b6=b0.cacheable||e-1;b1<e;b1++){b7.call(b8&&bG.nodeName(this[b1],"table")?y(this[b1],"tbody"):this[b1],b1===b6?b3:bG.clone(b3,true,true))}}b3=b2=null;if(bZ.length){bG.each(bZ,function(b9,ca){if(ca.src){if(bG.ajax){bG.ajax({url:ca.src,type:"GET",dataType:"script",async:false,global:false,"throws":true})}else{bG.error("no ajax")}}else{bG.globalEval((ca.text||ca.textContent||ca.innerHTML||"").replace(aH,""))}if(ca.parentNode){ca.parentNode.removeChild(ca)}})}}return this}});function y(bZ,e){return bZ.getElementsByTagName(e)[0]||bZ.appendChild(bZ.ownerDocument.createElement(e))}function ao(b5,bZ){if(bZ.nodeType!==1||!bG.hasData(b5)){return}var b2,b1,e,b4=bG._data(b5),b3=bG._data(bZ,b4),b0=b4.events;if(b0){delete b3.handle;b3.events={};for(b2 in b0){for(b1=0,e=b0[b2].length;b1<e;b1++){bG.event.add(bZ,b2,b0[b2][b1])}}}if(b3.data){b3.data=bG.extend({},b3.data)}}function G(bZ,e){var b0;if(e.nodeType!==1){return}i
f(e.clearAttributes){e.clearAttributes()}if(e.mergeAttributes){e.mergeAttributes(bZ)}b0=e.nodeName.toLowerCase();if(b0==="object"){if(e.parentNode){e.outerHTML=bZ.outerHTML}if(bG.support.html5Clone&&(bZ.innerHTML&&!bG.trim(e.innerHTML))){e.innerHTML=bZ.innerHTML}}else{if(b0==="input"&&aE.test(bZ.type)){e.defaultChecked=e.checked=bZ.checked;if(e.value!==bZ.value){e.value=bZ.value}}else{if(b0==="option"){e.selected=bZ.defaultSelected}else{if(b0==="input"||b0==="textarea"){e.defaultValue=bZ.defaultValue}else{if(b0==="script"&&e.text!==bZ.text){e.text=bZ.text}}}}}e.removeAttribute(bG.expando)}bG.buildFragment=function(b1,b2,bZ){var b0,e,b3,b4=b1[0];b2=b2||o;b2=(b2[0]||b2).ownerDocument||b2[0]||b2;if(typeof b2.createDocumentFragment==="undefined"){b2=o}if(b1.length===1&&typeof b4==="string"&&b4.length<512&&b2===o&&b4.charAt(0)==="<"&&!ap.test(b4)&&(bG.support.checkClone||!bR.test(b4))&&(bG.support.html5Clone||!K.test(b4))){e=true;b0=bG.fragments[b4];b3=b0!==aB}if(!b0){b0=b2.createDocumen
tFragment();bG.clean(b1,b2,b0,bZ);if(e){bG.fragments[b4]=b3&&b0}}return{fragment:b0,cacheable:e}};bG.fragments={};bG.each({appendTo:"append",prependTo:"prepend",insertBefore:"before",insertAfter:"after",replaceAll:"replaceWith"},function(e,bZ){bG.fn[e]=function(b0){var b2,b4=0,b3=[],b6=bG(b0),b1=b6.length,b5=this.length===1&&this[0].parentNode;if((b5==null||b5&&b5.nodeType===11&&b5.childNodes.length===1)&&b1===1){b6[bZ](this[0]);return this}else{for(;b4<b1;b4++){b2=(b4>0?this.clone(true):this).get();bG(b6[b4])[bZ](b2);b3=b3.concat(b2)}return this.pushStack(b3,e,b6.selector)}}});function m(e){if(typeof e.getElementsByTagName!=="undefined"){return e.getElementsByTagName("*")}else{if(typeof e.querySelectorAll!=="undefined"){return e.querySelectorAll("*")}else{return[]}}}function bS(e){if(aE.test(e.type)){e.defaultChecked=e.checked}}bG.extend({clone:function(b2,b4,b0){var e,bZ,b1,b3;if(bG.support.html5Clone||bG.isXMLDoc(b2)||!K.test("<"+b2.nodeName+">")){b3=b2.cloneNode(true)}else{l.inn
erHTML=b2.outerHTML;l.removeChild(b3=l.firstChild)}if((!bG.support.noCloneEvent||!bG.support.noCloneChecked)&&(b2.nodeType===1||b2.nodeType===11)&&!bG.isXMLDoc(b2)){G(b2,b3);e=m(b2);bZ=m(b3);for(b1=0;e[b1];++b1){if(bZ[b1]){G(e[b1],bZ[b1])}}}if(b4){ao(b2,b3);if(b0){e=m(b2);bZ=m(b3);for(b1=0;e[b1];++b1){ao(e[b1],bZ[b1])}}}e=bZ=null;return b3},clean:function(cb,b0,e,b1){var b7,b3,ca,cf,b4,ce,b5,b2,bZ,b9,cd,b6,b8=0,cc=[];if(!b0||typeof b0.createDocumentFragment==="undefined"){b0=o}for(b3=b0===o&&aQ;(ca=cb[b8])!=null;b8++){if(typeof ca==="number"){ca+=""}if(!ca){continue}if(typeof ca==="string"){if(!J.test(ca)){ca=b0.createTextNode(ca)}else{b3=b3||B(b0);b5=b5||b3.appendChild(b0.createElement("div"));ca=ca.replace(ay,"<$1></$2>");cf=(p.exec(ca)||["",""])[1].toLowerCase();b4=T[cf]||T._default;ce=b4[0];b5.innerHTML=b4[1]+ca+b4[2];while(ce--){b5=b5.lastChild}if(!bG.support.tbody){b2=bT.test(ca);bZ=cf==="table"&&!b2?b5.firstChild&&b5.firstChild.childNodes:b4[1]==="<table>"&&!b2?b5.childNodes:
[];for(b7=bZ.length-1;b7>=0;--b7){if(bG.nodeName(bZ[b7],"tbody")&&!bZ[b7].childNodes.length){bZ[b7].parentNode.removeChild(bZ[b7])}}}if(!bG.support.leadingWhitespace&&bY.test(ca)){b5.insertBefore(b0.createTextNode(bY.exec(ca)[0]),b5.firstChild)}ca=b5.childNodes;b5=b3.lastChild}}if(ca.nodeType){cc.push(ca)}else{cc=bG.merge(cc,ca)}}if(b5){b3.removeChild(b5);ca=b5=b3=null}if(!bG.support.appendChecked){for(b8=0;(ca=cc[b8])!=null;b8++){if(bG.nodeName(ca,"input")){bS(ca)}else{if(typeof ca.getElementsByTagName!=="undefined"){bG.grep(ca.getElementsByTagName("input"),bS)}}}}if(e){cd=function(cg){if(!cg.type||bw.test(cg.type)){return b1?b1.push(cg.parentNode?cg.parentNode.removeChild(cg):cg):e.appendChild(cg)}};for(b8=0;(ca=cc[b8])!=null;b8++){if(!(bG.nodeName(ca,"script")&&cd(ca))){e.appendChild(ca);if(typeof ca.getElementsByTagName!=="undefined"){b6=bG.grep(bG.merge([],ca.getElementsByTagName("script")),cd);cc.splice.apply(cc,[b8+1,0].concat(b6));b8+=b6.length}}}}return cc},cleanData:functi
on(bZ,b7){var b2,b0,b1,b6,b3=0,b8=bG.expando,e=bG.cache,b4=bG.support.deleteExpando,b5=bG.event.special;for(;(b1=bZ[b3])!=null;b3++){if(b7||bG.acceptData(b1)){b0=b1[b8];b2=b0&&e[b0];if(b2){if(b2.events){for(b6 in b2.events){if(b5[b6]){bG.event.remove(b1,b6)}else{bG.removeEvent(b1,b6,b2.handle)}}}if(e[b0]){delete e[b0];if(b4){delete b1[b8]}else{if(b1.removeAttribute){b1.removeAttribute(b8)}else{b1[b8]=null}}bG.deletedIds.push(b0)}}}}}});(function(){var e,bZ;bG.uaMatch=function(b1){b1=b1.toLowerCase();var b0=/(chrome)[ \/]([\w.]+)/.exec(b1)||/(webkit)[ \/]([\w.]+)/.exec(b1)||/(opera)(?:.*version|)[ \/]([\w.]+)/.exec(b1)||/(msie) ([\w.]+)/.exec(b1)||b1.indexOf("compatible")<0&&/(mozilla)(?:.*? rv:([\w.]+)|)/.exec(b1)||[];return{browser:b0[1]||"",version:b0[2]||"0"}};e=bG.uaMatch(d.userAgent);bZ={};if(e.browser){bZ[e.browser]=true;bZ.version=e.version}if(bZ.webkit){bZ.safari=true}bG.browser=bZ;bG.sub=function(){function b0(b3,b4){return new b0.fn.init(b3,b4)}bG.extend(true,b0,this);b0.s
uperclass=this;b0.fn=b0.prototype=this();b0.fn.constructor=b0;b0.sub=this.sub;b0.fn.init=function b2(b3,b4){if(b4&&b4 instanceof bG&&!(b4 instanceof b0)){b4=b0(b4)}return bG.fn.init.call(this,b3,b4,b1)};b0.fn.init.prototype=b0.fn;var b1=b0(o);return b0}})();var F,az,aW,be=/alpha\([^)]*\)/i,aS=/opacity=([^)]*)/,bk=/^(top|right|bottom|left)$/,aZ=/^margin/,a8=new RegExp("^("+bx+")(.*)$","i"),W=new RegExp("^("+bx+")(?!px)[a-z%]+$","i"),S=new RegExp("^([-+])=("+bx+")","i"),bh={},a9={position:"absolute",visibility:"hidden",display:"block"},bA={letterSpacing:0,fontWeight:400,lineHeight:1},bQ=["Top","Right","Bottom","Left"],ar=["Webkit","O","Moz","ms"],aJ=bG.fn.toggle;function b(b1,bZ){if(bZ in b1){return bZ}var b2=bZ.charAt(0).toUpperCase()+bZ.slice(1),e=bZ,b0=ar.length;while(b0--){bZ=ar[b0]+b2;if(bZ in b1){return bZ}}return e}function Q(bZ,e){bZ=e||bZ;return bG.css(bZ,"display")==="none"||!bG.contains(bZ.ownerDocument,bZ)}function t(b3,e){var b2,b4,bZ=[],b0=0,b1=b3.length;for(;b0<b1;b0++)
{b2=b3[b0];if(!b2.style){continue}bZ[b0]=bG._data(b2,"olddisplay");if(e){if(!bZ[b0]&&b2.style.display==="none"){b2.style.display=""}if(b2.style.display===""&&Q(b2)){bZ[b0]=bG._data(b2,"olddisplay",bC(b2.nodeName))}}else{b4=F(b2,"display");if(!bZ[b0]&&b4!=="none"){bG._data(b2,"olddisplay",b4)}}}for(b0=0;b0<b1;b0++){b2=b3[b0];if(!b2.style){continue}if(!e||b2.style.display==="none"||b2.style.display===""){b2.style.display=e?bZ[b0]||"":"none"}}return b3}bG.fn.extend({css:function(e,bZ){return bG.access(this,function(b1,b0,b2){return b2!==aB?bG.style(b1,b0,b2):bG.css(b1,b0)},e,bZ,arguments.length>1)},show:function(){return t(this,true)},hide:function(){return t(this)},toggle:function(b0,bZ){var e=typeof b0==="boolean";if(bG.isFunction(b0)&&bG.isFunction(bZ)){return aJ.apply(this,arguments)}return this.each(function(){if(e?b0:Q(this)){bG(this).show()}else{bG(this).hide()}})}});bG.extend({cssHooks:{opacity:{get:function(b0,bZ){if(bZ){var e=F(b0,"opacity");return e===""?"1":e}}}},cssNumber:
{fillOpacity:true,fontWeight:true,lineHeight:true,opacity:true,orphans:true,widows:true,zIndex:true,zoom:true},cssProps:{"float":bG.support.cssFloat?"cssFloat":"styleFloat"},style:function(b1,b0,b7,b2){if(!b1||b1.nodeType===3||b1.nodeType===8||!b1.style){return}var b5,b6,b8,b3=bG.camelCase(b0),bZ=b1.style;b0=bG.cssProps[b3]||(bG.cssProps[b3]=b(bZ,b3));b8=bG.cssHooks[b0]||bG.cssHooks[b3];if(b7!==aB){b6=typeof b7;if(b6==="string"&&(b5=S.exec(b7))){b7=(b5[1]+1)*b5[2]+parseFloat(bG.css(b1,b0));b6="number"}if(b7==null||b6==="number"&&isNaN(b7)){return}if(b6==="number"&&!bG.cssNumber[b3]){b7+="px"}if(!b8||!("set" in b8)||(b7=b8.set(b1,b7,b2))!==aB){try{bZ[b0]=b7}catch(b4){}}}else{if(b8&&"get" in b8&&(b5=b8.get(b1,false,b2))!==aB){return b5}return bZ[b0]}},css:function(b4,b2,b3,bZ){var b5,b1,e,b0=bG.camelCase(b2);b2=bG.cssProps[b0]||(bG.cssProps[b0]=b(b4.style,b0));e=bG.cssHooks[b2]||bG.cssHooks[b0];if(e&&"get" in e){b5=e.get(b4,true,bZ)}if(b5===aB){b5=F(b4,b2)}if(b5==="normal"&&b2 in bA){
b5=bA[b2]}if(b3||bZ!==aB){b1=parseFloat(b5);return b3||bG.isNumeric(b1)?b1||0:b5}return b5},swap:function(b2,b1,b3){var b0,bZ,e={};for(bZ in b1){e[bZ]=b2.style[bZ];b2.style[bZ]=b1[bZ]}b0=b3.call(b2);for(bZ in b1){b2.style[bZ]=e[bZ]}return b0}});if(a2.getComputedStyle){F=function(b5,bZ){var e,b2,b1,b4,b3=getComputedStyle(b5,null),b0=b5.style;if(b3){e=b3[bZ];if(e===""&&!bG.contains(b5.ownerDocument.documentElement,b5)){e=bG.style(b5,bZ)}if(W.test(e)&&aZ.test(bZ)){b2=b0.width;b1=b0.minWidth;b4=b0.maxWidth;b0.minWidth=b0.maxWidth=b0.width=e;e=b3.width;b0.width=b2;b0.minWidth=b1;b0.maxWidth=b4}}return e}}else{if(o.documentElement.currentStyle){F=function(b2,b0){var b3,e,bZ=b2.currentStyle&&b2.currentStyle[b0],b1=b2.style;if(bZ==null&&b1&&b1[b0]){bZ=b1[b0]}if(W.test(bZ)&&!bk.test(b0)){b3=b1.left;e=b2.runtimeStyle&&b2.runtimeStyle.left;if(e){b2.runtimeStyle.left=b2.currentStyle.left}b1.left=b0==="fontSize"?"1em":bZ;bZ=b1.pixelLeft+"px";b1.left=b3;if(e){b2.runtimeStyle.left=e}}return bZ==="
"?"auto":bZ}}}function aG(e,b0,b1){var bZ=a8.exec(b0);return bZ?Math.max(0,bZ[1]-(b1||0))+(bZ[2]||"px"):b0}function at(b1,bZ,e,b3){var b0=e===(b3?"border":"content")?4:bZ==="width"?1:0,b2=0;for(;b0<4;b0+=2){if(e==="margin"){b2+=bG.css(b1,e+bQ[b0],true)}if(b3){if(e==="content"){b2-=parseFloat(F(b1,"padding"+bQ[b0]))||0}if(e!=="margin"){b2-=parseFloat(F(b1,"border"+bQ[b0]+"Width"))||0}}else{b2+=parseFloat(F(b1,"padding"+bQ[b0]))||0;if(e!=="padding"){b2+=parseFloat(F(b1,"border"+bQ[b0]+"Width"))||0}}}return b2}function v(b1,bZ,e){var b2=bZ==="width"?b1.offsetWidth:b1.offsetHeight,b0=true,b3=bG.support.boxSizing&&bG.css(b1,"boxSizing")==="border-box";if(b2<=0){b2=F(b1,bZ);if(b2<0||b2==null){b2=b1.style[bZ]}if(W.test(b2)){return b2}b0=b3&&(bG.support.boxSizingReliable||b2===b1.style[bZ]);b2=parseFloat(b2)||0}return(b2+at(b1,bZ,e||(b3?"border":"content"),b0))+"px"}function bC(b0){if(bh[b0]){return bh[b0]}var e=bG("<"+b0+">").appendTo(o.body),bZ=e.css("display");e.remove();if(bZ==="none"||
bZ===""){az=o.body.appendChild(az||bG.extend(o.createElement("iframe"),{frameBorder:0,width:0,height:0}));if(!aW||!az.createElement){aW=(az.contentWindow||az.contentDocument).document;aW.write("<!doctype html><html><body>");aW.close()}e=aW.body.appendChild(aW.createElement(b0));bZ=F(e,"display");o.body.removeChild(az)}bh[b0]=bZ;return bZ}bG.each(["height","width"],function(bZ,e){bG.cssHooks[e]={get:function(b2,b1,b0){if(b1){if(b2.offsetWidth!==0||F(b2,"display")!=="none"){return v(b2,e,b0)}else{return bG.swap(b2,a9,function(){return v(b2,e,b0)})}}},set:function(b1,b2,b0){return aG(b1,b2,b0?at(b1,e,b0,bG.support.boxSizing&&bG.css(b1,"boxSizing")==="border-box"):0)}}});if(!bG.support.opacity){bG.cssHooks.opacity={get:function(bZ,e){return aS.test((e&&bZ.currentStyle?bZ.currentStyle.filter:bZ.style.filter)||"")?(0.01*parseFloat(RegExp.$1))+"":e?"1":""},set:function(b2,b3){var b1=b2.style,bZ=b2.currentStyle,e=bG.isNumeric(b3)?"alpha(opacity="+b3*100+")":"",b0=bZ&&bZ.filter||b1.filter||"
";b1.zoom=1;if(b3>=1&&bG.trim(b0.replace(be,""))===""&&b1.removeAttribute){b1.removeAttribute("filter");if(bZ&&!bZ.filter){return}}b1.filter=be.test(b0)?b0.replace(be,e):b0+" "+e}}}bG(function(){if(!bG.support.reliableMarginRight){bG.cssHooks.marginRight={get:function(bZ,e){return bG.swap(bZ,{display:"inline-block"},function(){if(e){return F(bZ,"marginRight")}})}}}if(!bG.support.pixelPosition&&bG.fn.position){bG.each(["top","left"],function(e,bZ){bG.cssHooks[bZ]={get:function(b2,b1){if(b1){var b0=F(b2,bZ);return W.test(b0)?bG(b2).position()[bZ]+"px":b0}}}})}});if(bG.expr&&bG.expr.filters){bG.expr.filters.hidden=function(e){return(e.offsetWidth===0&&e.offsetHeight===0)||(!bG.support.reliableHiddenOffsets&&((e.style&&e.style.display)||F(e,"display"))==="none")};bG.expr.filters.visible=function(e){return !bG.expr.filters.hidden(e)}}bG.each({margin:"",padding:"",border:"Width"},function(e,bZ){bG.cssHooks[e+bZ]={expand:function(b2){var b1,b3=typeof b2==="string"?b2.split(" "):[b2],b0={};
for(b1=0;b1<4;b1++){b0[e+bQ[b1]+bZ]=b3[b1]||b3[b1-2]||b3[0]}return b0}};if(!aZ.test(e)){bG.cssHooks[e+bZ].set=aG}});var bs=/%20/g,aP=/\[\]$/,U=/\r?\n/g,bz=/^(?:color|date|datetime|datetime-local|email|hidden|month|number|password|range|search|tel|text|time|url|week)$/i,aD=/^(?:select|textarea)/i;bG.fn.extend({serialize:function(){return bG.param(this.serializeArray())},serializeArray:function(){return this.map(function(){return this.elements?bG.makeArray(this.elements):this}).filter(function(){return this.name&&!this.disabled&&(this.checked||aD.test(this.nodeName)||bz.test(this.type))}).map(function(e,bZ){var b0=bG(this).val();return b0==null?null:bG.isArray(b0)?bG.map(b0,function(b2,b1){return{name:bZ.name,value:b2.replace(U,"\r\n")}}):{name:bZ.name,value:b0.replace(U,"\r\n")}}).get()}});bG.param=function(e,b0){var b1,bZ=[],b2=function(b3,b4){b4=bG.isFunction(b4)?b4():(b4==null?"":b4);bZ[bZ.length]=encodeURIComponent(b3)+"="+encodeURIComponent(b4)};if(b0===aB){b0=bG.ajaxSettings&&b
G.ajaxSettings.traditional}if(bG.isArray(e)||(e.jquery&&!bG.isPlainObject(e))){bG.each(e,function(){b2(this.name,this.value)})}else{for(b1 in e){k(b1,e[b1],b0,b2)}}return bZ.join("&").replace(bs,"+")};function k(b0,b2,bZ,b1){var e;if(bG.isArray(b2)){bG.each(b2,function(b4,b3){if(bZ||aP.test(b0)){b1(b0,b3)}else{k(b0+"["+(typeof b3==="object"?b4:"")+"]",b3,bZ,b1)}})}else{if(!bZ&&bG.type(b2)==="object"){for(e in b2){k(b0+"["+e+"]",b2[e],bZ,b1)}}else{b1(b0,b2)}}}var Y,bX,an=/#.*$/,ad=/^(.*?):[ \t]*([^\r\n]*)\r?$/mg,C=/^(?:about|app|app\-storage|.+\-extension|file|res|widget):$/,s=/^(?:GET|HEAD)$/,aC=/^\/\//,bN=/\?/,g=/<script\b[^<]*(?:(?!<\/script>)<[^<]*)*<\/script>/gi,P=/([?&])_=[^&]*/,aT=/^([\w\+\.\-]+:)(?:\/\/([^\/?#:]*)(?::(\d+)|)|)/,bW=bG.fn.load,w={},a6={},aX=["*/"]+["*"];try{Y=aI.href}catch(bd){Y=o.createElement("a");Y.href="";Y=Y.href}bX=aT.exec(Y.toLowerCase())||[];function bI(e){return function(b2,b4){if(typeof b2!=="string"){b4=b2;b2="*"}var bZ,b5,b6,b1=b2.toLowerCase().spli
t(aV),b0=0,b3=b1.length;if(bG.isFunction(b4)){for(;b0<b3;b0++){bZ=b1[b0];b6=/^\+/.test(bZ);if(b6){bZ=bZ.substr(1)||"*"}b5=e[bZ]=e[bZ]||[];b5[b6?"unshift":"push"](b4)}}}}function r(bZ,b8,b3,b6,b5,b1){b5=b5||b8.dataTypes[0];b1=b1||{};b1[b5]=true;var b7,b4=bZ[b5],b0=0,e=b4?b4.length:0,b2=(bZ===w);for(;b0<e&&(b2||!b7);b0++){b7=b4[b0](b8,b3,b6);if(typeof b7==="string"){if(!b2||b1[b7]){b7=aB}else{b8.dataTypes.unshift(b7);b7=r(bZ,b8,b3,b6,b7,b1)}}}if((b2||!b7)&&!b1["*"]){b7=r(bZ,b8,b3,b6,"*",b1)}return b7}function u(b0,b1){var bZ,e,b2=bG.ajaxSettings.flatOptions||{};for(bZ in b1){if(b1[bZ]!==aB){(b2[bZ]?b0:(e||(e={})))[bZ]=b1[bZ]}}if(e){bG.extend(true,b0,e)}}bG.fn.load=function(b1,b4,b5){if(typeof b1!=="string"&&bW){return bW.apply(this,arguments)}if(!this.length){return this}var e,b2,b0,bZ=this,b3=b1.indexOf(" ");if(b3>=0){e=b1.slice(b3,b1.length);b1=b1.slice(0,b3)}if(bG.isFunction(b4)){b5=b4;b4=aB}else{if(typeof b4==="object"){b2="POST"}}bG.ajax({url:b1,type:b2,dataType:"html",data:b4,co
mplete:function(b7,b6){if(b5){bZ.each(b5,b0||[b7.responseText,b6,b7])}}}).done(function(b6){b0=arguments;bZ.html(e?bG("<div>").append(b6.replace(g,"")).find(e):b6)});return this};bG.each("ajaxStart ajaxStop ajaxComplete ajaxError ajaxSuccess ajaxSend".split(" "),function(e,bZ){bG.fn[bZ]=function(b0){return this.on(bZ,b0)}});bG.each(["get","post"],function(e,bZ){bG[bZ]=function(b0,b2,b3,b1){if(bG.isFunction(b2)){b1=b1||b3;b3=b2;b2=aB}return bG.ajax({type:bZ,url:b0,data:b2,success:b3,dataType:b1})}});bG.extend({getScript:function(e,bZ){return bG.get(e,aB,bZ,"script")},getJSON:function(e,bZ,b0){return bG.get(e,bZ,b0,"json")},ajaxSetup:function(bZ,e){if(e){u(bZ,bG.ajaxSettings)}else{e=bZ;bZ=bG.ajaxSettings}u(bZ,e);return bZ},ajaxSettings:{url:Y,isLocal:C.test(bX[1]),global:true,type:"GET",contentType:"application/x-www-form-urlencoded; charset=UTF-8",processData:true,async:true,accepts:{xml:"application/xml, text/xml",html:"text/html",text:"text/plain",json:"application/json, text/javas
cript","*":aX},contents:{xml:/xml/,html:/html/,json:/json/},responseFields:{xml:"responseXML",text:"responseText"},converters:{"* text":a2.String,"text html":true,"text json":bG.parseJSON,"text xml":bG.parseXML},flatOptions:{context:true,url:true}},ajaxPrefilter:bI(w),ajaxTransport:bI(a6),ajax:function(b4,b1){if(typeof b4==="object"){b1=b4;b4=aB}b1=b1||{};var b7,cl,b2,cg,b9,cd,b0,cf,b8=bG.ajaxSetup({},b1),cn=b8.context||b8,cb=cn!==b8&&(cn.nodeType||cn instanceof bG)?bG(cn):bG.event,cm=bG.Deferred(),ci=bG.Callbacks("once memory"),b5=b8.statusCode||{},cc={},cj={},b3=0,b6="canceled",ce={readyState:0,setRequestHeader:function(co,cp){if(!b3){var e=co.toLowerCase();co=cj[e]=cj[e]||co;cc[co]=cp}return this},getAllResponseHeaders:function(){return b3===2?cl:null},getResponseHeader:function(co){var e;if(b3===2){if(!b2){b2={};while((e=ad.exec(cl))){b2[e[1].toLowerCase()]=e[2]}}e=b2[co.toLowerCase()]}return e===aB?null:e},overrideMimeType:function(e){if(!b3){b8.mimeType=e}return this},abort:fu
nction(e){e=e||b6;if(cg){cg.abort(e)}ca(0,e);return this}};function ca(cs,co,ct,cq){var e,cw,cu,cr,cv,cp=co;if(b3===2){return}b3=2;if(b9){clearTimeout(b9)}cg=aB;cl=cq||"";ce.readyState=cs>0?4:0;if(ct){cr=h(b8,ce,ct)}if(cs>=200&&cs<300||cs===304){if(b8.ifModified){cv=ce.getResponseHeader("Last-Modified");if(cv){bG.lastModified[b7]=cv}cv=ce.getResponseHeader("Etag");if(cv){bG.etag[b7]=cv}}if(cs===304){cp="notmodified";e=true}else{e=ae(b8,cr);cp=e.state;cw=e.data;cu=e.error;e=!cu}}else{cu=cp;if(!cp||cs){cp="error";if(cs<0){cs=0}}}ce.status=cs;ce.statusText=""+(co||cp);if(e){cm.resolveWith(cn,[cw,cp,ce])}else{cm.rejectWith(cn,[ce,cp,cu])}ce.statusCode(b5);b5=aB;if(b0){cb.trigger("ajax"+(e?"Success":"Error"),[ce,b8,e?cw:cu])}ci.fireWith(cn,[ce,cp]);if(b0){cb.trigger("ajaxComplete",[ce,b8]);if(!(--bG.active)){bG.event.trigger("ajaxStop")}}}cm.promise(ce);ce.success=ce.done;ce.error=ce.fail;ce.complete=ci.add;ce.statusCode=function(co){if(co){var e;if(b3<2){for(e in co){b5[e]=[b5[e],co[e]]
}}else{e=co[ce.status];ce.always(e)}}return this};b8.url=((b4||b8.url)+"").replace(an,"").replace(aC,bX[1]+"//");b8.dataTypes=bG.trim(b8.dataType||"*").toLowerCase().split(aV);if(b8.crossDomain==null){cd=aT.exec(b8.url.toLowerCase());b8.crossDomain=!!(cd&&(cd[1]!=bX[1]||cd[2]!=bX[2]||(cd[3]||(cd[1]==="http:"?80:443))!=(bX[3]||(bX[1]==="http:"?80:443))))}if(b8.data&&b8.processData&&typeof b8.data!=="string"){b8.data=bG.param(b8.data,b8.traditional)}r(w,b8,b1,ce);if(b3===2){return ce}b0=b8.global;b8.type=b8.type.toUpperCase();b8.hasContent=!s.test(b8.type);if(b0&&bG.active++===0){bG.event.trigger("ajaxStart")}if(!b8.hasContent){if(b8.data){b8.url+=(bN.test(b8.url)?"&":"?")+b8.data;delete b8.data}b7=b8.url;if(b8.cache===false){var bZ=bG.now(),ck=b8.url.replace(P,"$1_="+bZ);b8.url=ck+((ck===b8.url)?(bN.test(b8.url)?"&":"?")+"_="+bZ:"")}}if(b8.data&&b8.hasContent&&b8.contentType!==false||b1.contentType){ce.setRequestHeader("Content-Type",b8.contentType)}if(b8.ifModified){b7=b7||b8.url;if
(bG.lastModified[b7]){ce.setRequestHeader("If-Modified-Since",bG.lastModified[b7])}if(bG.etag[b7]){ce.setRequestHeader("If-None-Match",bG.etag[b7])}}ce.setRequestHeader("Accept",b8.dataTypes[0]&&b8.accepts[b8.dataTypes[0]]?b8.accepts[b8.dataTypes[0]]+(b8.dataTypes[0]!=="*"?", "+aX+"; q=0.01":""):b8.accepts["*"]);for(cf in b8.headers){ce.setRequestHeader(cf,b8.headers[cf])}if(b8.beforeSend&&(b8.beforeSend.call(cn,ce,b8)===false||b3===2)){return ce.abort()}b6="abort";for(cf in {success:1,error:1,complete:1}){ce[cf](b8[cf])}cg=r(a6,b8,b1,ce);if(!cg){ca(-1,"No Transport")}else{ce.readyState=1;if(b0){cb.trigger("ajaxSend",[ce,b8])}if(b8.async&&b8.timeout>0){b9=setTimeout(function(){ce.abort("timeout")},b8.timeout)}try{b3=1;cg.send(cc,ca)}catch(ch){if(b3<2){ca(-1,ch)}else{throw ch}}}return ce},active:0,lastModified:{},etag:{}});function h(b7,b6,b3){var b2,b4,b1,e,bZ=b7.contents,b5=b7.dataTypes,b0=b7.responseFields;for(b4 in b0){if(b4 in b3){b6[b0[b4]]=b3[b4]}}while(b5[0]==="*"){b5.shift()
;if(b2===aB){b2=b7.mimeType||b6.getResponseHeader("content-type")}}if(b2){for(b4 in bZ){if(bZ[b4]&&bZ[b4].test(b2)){b5.unshift(b4);break}}}if(b5[0] in b3){b1=b5[0]}else{for(b4 in b3){if(!b5[0]||b7.converters[b4+" "+b5[0]]){b1=b4;break}if(!e){e=b4}}b1=b1||e}if(b1){if(b1!==b5[0]){b5.unshift(b1)}return b3[b1]}}function ae(b9,b1){var b7,bZ,b5,b3,b6=b9.dataTypes.slice(),b0=b6[0],b8={},b2=0;if(b9.dataFilter){b1=b9.dataFilter(b1,b9.dataType)}if(b6[1]){for(b7 in b9.converters){b8[b7.toLowerCase()]=b9.converters[b7]}}for(;(b5=b6[++b2]);){if(b5!=="*"){if(b0!=="*"&&b0!==b5){b7=b8[b0+" "+b5]||b8["* "+b5];if(!b7){for(bZ in b8){b3=bZ.split("
<TRUNCATED>
[07/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html b/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html
new file mode 100644
index 0000000..e01ec59
--- /dev/null
+++ b/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html
@@ -0,0 +1,239 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Overview</title>
+ <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../../css/site.css" />
+ <link rel="stylesheet" href="../../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Overview</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-right"></i>
+ Roles</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Overview</h1>
+<p><a name="Overview"></a></p>
+<p>The Metron ansible-docker container is provided in an effort reduce the installation burden of deploying Metron in a live envirionment. It is provisioned with software required to sucessfully run the deployment scripts.</p>
+<div class="section">
+<h2><a name="Building_the_Container"></a>Building the Container</h2>
+
+<ol style="list-style-type: decimal">
+
+<li>Install Docker [https://www.docker.com/products/overview]</li>
+
+<li>Navigate to <project-directory>/metron-deployment/packaging/docker/ansible-docker</li>
+
+<li>Build the container <tt>docker build -t ansible-docker:2.0.0.2 .</tt></li>
+</ol></div>
+<div class="section">
+<h2><a name="Using_the_Container"></a>Using the Container</h2>
+<p>Full instructions are found on the wiki [https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144361].</p>
+<p>tl;dr</p>
+
+<ol style="list-style-type: decimal">
+
+<li>docker run -it -v <project-directory>:/root/incubator-metron ansible-docker:2.0.0.2 bash</li>
+
+<li>cd /root/incubator-metron</li>
+
+<li>mvn clean package -DskipTests</li>
+</ol></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html b/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html
new file mode 100644
index 0000000..ae6948e
--- /dev/null
+++ b/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html
@@ -0,0 +1,226 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Overview</title>
+ <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../../css/site.css" />
+ <link rel="stylesheet" href="../../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Overview</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-right"></i>
+ Roles</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Overview</h1>
+<p><a name="Overview"></a></p>
+<p>The Metron ansible-docker container is provided in an effort reduce the installation burden of deploying Metron in a live envirionment. It is provisioned with software required to sucessfully run the deployment scripts.</p>
+<div class="section">
+<h2><a name="Building_the_Container"></a>Building the Container</h2>
+
+<ol style="list-style-type: decimal">
+
+<li>Install Docker [https://www.docker.com/products/overview]</li>
+
+<li>Navigate to <project-directory>/metron-deployment/packaging/rpm-docker</li>
+
+<li>Build the container <tt>docker build -t rpm-docker .</tt></li>
+</ol></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/packer-build/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/packer-build/index.html b/site/current-book/metron-deployment/packer-build/index.html
new file mode 100644
index 0000000..a18e5ab
--- /dev/null
+++ b/site/current-book/metron-deployment/packer-build/index.html
@@ -0,0 +1,274 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Build Metron Images</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Build Metron Images</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-right"></i>
+ Roles</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Build Metron Images</h1>
+<p>Based on the fantastic <a class="externalLink" href="https://github.com/chef/bento">Bento</a> project developed by Chef.</p>
+<div class="section">
+<h2><a name="Images_Provided"></a>Images Provided</h2>
+
+<ul>
+
+<li>hdp-centos-6.7: Centos 6.7 + HDP. Used in the quick-dev-platform Vagrant image</li>
+
+<li>metron-centos-6.7: Centos 6.7 + HDP + Metron. Used for the codelab-platform Vagrant image.</li>
+</ul></div>
+<div class="section">
+<h2><a name="Prerequisites"></a>Prerequisites</h2>
+
+<ul>
+
+<li><a class="externalLink" href="https://www.packer.io/">Packer</a> 0.10.1</li>
+
+<li><a class="externalLink" href="https://www.virtualbox.org/">Virtualbox</a> 5.0.16</li>
+
+<li>Be sure to build Metron prior to building the images (cd <i>your-project-directory</i>/metron-platform && mvn clean package -DskipTests)</li>
+</ul>
+<p>Build Both Images ———————- Navigate to <your-project-directory>/metron-deployment/packer-build Execute bin/bento build</p>
+<p>Packer will build both images and export .box files to the ./builds directory.</p>
+<p>Build Single Images ———————- Navigate to <i>your-project-directory</i>/metron-deployment/packer-build</p>
+
+<ul>
+
+<li>HDP Centos</li>
+</ul>
+
+<div class="source">
+<div class="source">
+<pre>bin/bento build hdp-centos-6.7.json
+</pre></div></div>
+
+<ul>
+
+<li>Full Metron</li>
+</ul>
+
+<div class="source">
+<div class="source">
+<pre>bin/bento build metron-centos-6.7.json
+</pre></div></div>
+<p>Using Your New Box File ———————- Modify the relevant Vagrantfile (codelab-platform or quick-dev-platform) replacing the lines:</p>
+
+<div class="source">
+<div class="source">
+<pre><pre><code>config.vm.box = "<i>box_name</i>"
+config.ssh.insert_key = true</code></pre>
+</pre></div></div>
+<p>with</p>
+
+<div class="source">
+<div class="source">
+<pre><pre></code>config.vm.box = "<i>test_box_name</i>"
+config.vm.box = "<i>PathToBoxfile/Boxfilename</i>"
+config.ssh.insert_key = true</code></pre>
+</pre></div></div>
+<p>Launch the image as usual.</p>
+<p>Node: Vagrant will cache boxes, you can force Vagrant to reload your box by running <tt>vagrant box remove <i>test_box_name</i></tt> before launching your new image.</p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/roles/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/index.html b/site/current-book/metron-deployment/roles/index.html
new file mode 100644
index 0000000..cdcf082
--- /dev/null
+++ b/site/current-book/metron-deployment/roles/index.html
@@ -0,0 +1,271 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Ansible Roles</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Ansible Roles</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="icon-chevron-down"></i>Roles</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-deployment/roles/kibana/index.html" title="Kibana">
+ <i class="none"></i>
+ Kibana</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/roles/monit/index.html" title="Monit">
+ <i class="none"></i>
+ Monit</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
+ <i class="none"></i>
+ Opentaxii</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
+ <i class="none"></i>
+ Pcap_replay</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
+ <i class="none"></i>
+ Sensor-stubs</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
+ <i class="none"></i>
+ Sensor-test-mode</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Ansible Roles</h1>
+<p><a name="Ansible_Roles"></a></p>
+
+<ul>
+
+<li>Kibana</li>
+
+<li>Monit</li>
+
+<li>OpenTaxii</li>
+
+<li>Pcap Relay</li>
+
+<li>Sensor Test Mode</li>
+</ul>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/roles/kibana/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/kibana/index.html b/site/current-book/metron-deployment/roles/kibana/index.html
new file mode 100644
index 0000000..19388f9
--- /dev/null
+++ b/site/current-book/metron-deployment/roles/kibana/index.html
@@ -0,0 +1,285 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Kibana 4</title>
+ <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../css/site.css" />
+ <link rel="stylesheet" href="../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Kibana 4</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-down"></i>
+ Roles</a>
+ <ul class="nav nav-list">
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Kibana</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/monit/index.html" title="Monit">
+ <i class="none"></i>
+ Monit</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
+ <i class="none"></i>
+ Opentaxii</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
+ <i class="none"></i>
+ Pcap_replay</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
+ <i class="none"></i>
+ Sensor-stubs</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
+ <i class="none"></i>
+ Sensor-test-mode</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <div class="section">
+<h2><a name="Kibana_4"></a>Kibana 4</h2>
+<p>This role installs Kibana along with the default Metron Dashboard.</p>
+<div class="section">
+<h3><a name="FAQ"></a>FAQ</h3>
+<div class="section">
+<h4><a name="How_do_I_change_Metrons_default_dashboard"></a>How do I change Metron’s default dashboard?</h4>
+<p>Kibana stores all configuration elements within an Elasticsearch index called <tt>.kibana</tt>. To deploy Kibana in a desired state, including the Metron Dashboard, we simply take an extract from a functioning Kibana instance and store that in <tt>templates/kibana-index.json</tt>. The deployment process then restores the index from this extract.</p>
+<p>(1) Stand-up an instance of Apache Metron and create the Kibana index patterns, visualizations, and dashboard as you see fit.</p>
+<p>(2) Run the following command to extract the definitions for all the components that you have created. Be sure to delete anything that you don’t want to be part of this extract. It will include all artifacts present in your <tt>.kibana</tt> index.</p>
+
+<div class="source">
+<div class="source">
+<pre> elasticdump --input=http://ec2-52-41-121-175.us-west-2.compute.amazonaws.com:9200/.kibana \
+ --output=kibana-index.json \
+ --type=data \
+ --searchBody='{"filter": { "or": [ {"type": {"value": "search"}}, {"type": {"value":"dashboard"}}, {"type": {"value":"visualization"}},{"type": {"value": "config"}},{"type": {"value": "url"}},{"type": {"value": "index-pattern"}} ] }}'
+</pre></div></div>
+<p>(3) This will result in a file containing the JSON-based definitions. Overwrite <tt>templates/kibana-index.json</tt>.</p>
+<p>(4) After redeploying the code, your changes should now be a part of the default Metron dashboard.</p></div>
+<div class="section">
+<h4><a name="Why_do_my_dashboard_components_change_their_order_when_reloading_the_dashboard"></a>Why do my dashboard components change their order when reloading the dashboard?</h4>
+<p>This has been a problem in Kibana 4.5.1 and perhaps other versions too. To address this problem find the definition for your dashboard in the Kibana index extract. It will look like the following.</p>
+
+<div class="source">
+<div class="source">
+<pre>{"_index":".kibana","_type":"dashboard","_id":"Metron-Dashboard",...
+</pre></div></div>
+<p>Extract the <tt>panelsJSON</tt> field from the dashboard definition. Reorder the definition of these panels so that they are ordered by row and column. The component in row 1 should come before the component in row 2, etc. After you have ordered the components in this way, Kibana will maintain the order of components in the dashboard.</p></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/roles/monit/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/monit/index.html b/site/current-book/metron-deployment/roles/monit/index.html
new file mode 100644
index 0000000..07f3bcd
--- /dev/null
+++ b/site/current-book/metron-deployment/roles/monit/index.html
@@ -0,0 +1,327 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Monit Integration</title>
+ <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../css/site.css" />
+ <link rel="stylesheet" href="../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Monit Integration</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-down"></i>
+ Roles</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/kibana/index.html" title="Kibana">
+ <i class="none"></i>
+ Kibana</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Monit</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
+ <i class="none"></i>
+ Opentaxii</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
+ <i class="none"></i>
+ Pcap_replay</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
+ <i class="none"></i>
+ Sensor-stubs</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
+ <i class="none"></i>
+ Sensor-test-mode</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Monit Integration</h1>
+<p><a name="Monit_Integration"></a></p>
+<p>This role will leverage Monit as a process watchdog to manage sensors, topologies, and core services. </p>
+
+<ul>
+
+<li>Monit can be used to start, stop, or check status of any of the sensors or topologies.</li>
+
+<li>When monitoring is enabled (on by default) if a process dies, it will be restarted.</li>
+
+<li>The Monit web interface is exposed at <a class="externalLink" href="http://hostname:2812">http://hostname:2812</a>.</li>
+
+<li>The web interface username and password is defined by the <tt>monit_user</tt> and <tt>monit_password</tt> variables. These default to <tt>admin</tt>/<tt>monit</tt>.</li>
+
+<li>Monit CLI tools can also be used to simplify the process of managing Metron components.</li>
+
+<li>
+<p>The post-deployment report for Amazon-EC2 provides links to Monit’s web interface labeled as ‘Sensor Status’ and ‘Topology Status.’</p>
+
+<div class="source">
+<div class="source">
+<pre> ok: [localhost] => {
+"Success": [
+ "Apache Metron deployed successfully",
+ " Metron @ http://ec2-52-39-143-62.us-west-2.compute.amazonaws.com:5000",
+ " Ambari @ http://ec2-52-39-4-93.us-west-2.compute.amazonaws.com:8080",
+ " Sensor Status @ http://ec2-52-39-4-93.us-west-2.compute.amazonaws.com:2812",
+ " Topology Status @ http://ec2-52-39-130-62.us-west-2.compute.amazonaws.com:2812",
+ "For additional information, see https://metron.incubator.apache.org/'"
+]
+ }
+</pre></div></div></li>
+</ul>
+<div class="section">
+<h2><a name="Usage"></a>Usage</h2>
+<p>Start all Metron components</p>
+
+<div class="source">
+<div class="source">
+<pre>monit start all
+</pre></div></div>
+<p>Stop all Metron components</p>
+
+<div class="source">
+<div class="source">
+<pre>monit stop all
+</pre></div></div>
+<p>Start an individual Metron component</p>
+
+<div class="source">
+<div class="source">
+<pre>monit start bro-parser
+</pre></div></div>
+<p>Start all components required to ingest Bro data</p>
+
+<div class="source">
+<div class="source">
+<pre>monit -g bro start
+</pre></div></div>
+<p>Start all parsers</p>
+
+<div class="source">
+<div class="source">
+<pre>monit -g parsers start
+</pre></div></div>
+<p>What is running?</p>
+
+<div class="source">
+<div class="source">
+<pre>monit summary
+</pre></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
[04/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-platform/metron-common/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-common/index.html b/site/current-book/metron-platform/metron-common/index.html
new file mode 100644
index 0000000..c116276
--- /dev/null
+++ b/site/current-book/metron-platform/metron-common/index.html
@@ -0,0 +1,2233 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Contents</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Contents</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-down"></i>
+ Platform</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-platform/metron-api/index.html" title="Api">
+ <i class="none"></i>
+ Api</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Common</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
+ <i class="none"></i>
+ Data-management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
+ <i class="none"></i>
+ Enrichment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
+ <i class="none"></i>
+ Indexing</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-management/index.html" title="Management">
+ <i class="none"></i>
+ Management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
+ <i class="none"></i>
+ Parsers</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
+ <i class="none"></i>
+ Pcap-backend</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Contents</h1>
+<p><a name="Contents"></a></p>
+
+<ul>
+
+<li><a href="#Stellar_Language">Stellar Language</a>
+
+<ul>
+
+<li><a href="#Stellar_Language_Keywords">Stellar Language Keywords</a></li>
+
+<li><a href="#Stellar_Core_Functions">Stellar Core Functions</a></li>
+
+<li><a href="#Stellar_Shell">Stellar Shell</a></li>
+ </ul></li>
+
+<li><a href="#Global_Configuration">Global Configuration</a></li>
+
+<li><a href="#Management_Utility">Management Utility</a></li>
+</ul>
+<p><a name="Stellar_Language"></a></p>
+<h1>Stellar Language</h1>
+<p>For a variety of components (threat intelligence triage and field transformations) we have the need to do simple computation and transformation using the data from messages as variables.<br />For those purposes, there exists a simple, scaled down DSL created to do simple computation and transformation.</p>
+<p>The query language supports the following:</p>
+
+<ul>
+
+<li>Referencing fields in the enriched JSON</li>
+
+<li>Simple boolean operations: <tt>and</tt>, <tt>not</tt>, <tt>or</tt></li>
+
+<li>Simple arithmetic operations: <tt>*</tt>, <tt>/</tt>, <tt>+</tt>, <tt>-</tt> on real numbers or integers</li>
+
+<li>Simple comparison operations <tt><</tt>, <tt>></tt>, <tt><=</tt>, <tt>>=</tt></li>
+
+<li>Simple equality comparison operations <tt>==</tt>, <tt>!=</tt></li>
+
+<li>if/then/else comparisons (i.e. <tt>if var1 < 10 then 'less than 10' else '10 or more'</tt>)</li>
+
+<li>Determining whether a field exists (via <tt>exists</tt>)</li>
+
+<li>An <tt>in</tt> operator that works like the <tt>in</tt> in Python</li>
+
+<li>The ability to have parenthesis to make order of operations explicit</li>
+
+<li>User defined functions</li>
+</ul>
+<div class="section">
+<h2><a name="Stellar_Language_Keywords"></a>Stellar Language Keywords</h2>
+<p>The following keywords need to be single quote escaped in order to be used in Stellar expressions:</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th align="center"> </th>
+
+<th align="center"> </th>
+
+<th align="center"> </th>
+
+<th align="center"> </th>
+
+<th align="center"> </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td align="center">not </td>
+
+<td align="center">else </td>
+
+<td align="center">exists </td>
+
+<td align="center">if </td>
+
+<td align="center">then </td>
+ </tr>
+
+<tr class="a">
+
+<td align="center">and </td>
+
+<td align="center">or </td>
+
+<td align="center">in </td>
+
+<td align="center">== </td>
+
+<td align="center">!= </td>
+ </tr>
+
+<tr class="b">
+
+<td align="center"><= </td>
+
+<td align="center">> </td>
+
+<td align="center">>= </td>
+
+<td align="center">+ </td>
+
+<td align="center">- </td>
+ </tr>
+
+<tr class="a">
+
+<td align="center">< </td>
+
+<td align="center">? </td>
+
+<td align="center">* </td>
+
+<td align="center">/ </td>
+
+<td align="center">, </td>
+ </tr>
+ </tbody>
+</table>
+<p>Using parens such as: “foo” : “<ok>” requires escaping; “foo”: “'<ok>'”</p></div>
+<div class="section">
+<h2><a name="Stellar_Language_Inclusion_Checks_in_and_not_in"></a>Stellar Language Inclusion Checks (<tt>in</tt> and <tt>not in</tt>)</h2>
+
+<ol style="list-style-type: decimal">
+
+<li><tt>in</tt> supports string contains. e.g. <tt>'foo' in 'foobar' == true</tt></li>
+
+<li><tt>in</tt> supports collection contains. e.g. <tt>'foo' in [ 'foo', 'bar' ] == true</tt></li>
+
+<li><tt>in</tt> supports map key contains. e.g. <tt>'foo' in { 'foo' : 5} == true</tt></li>
+
+<li><tt>not in</tt> is the negation of the in expression. e.g. <tt>'grok' not in 'foobar' == true</tt></li>
+</ol></div>
+<div class="section">
+<h2><a name="Stellar_Language_Comparisons____"></a>Stellar Language Comparisons (<tt><</tt>, <tt><=</tt>, <tt>></tt>, <tt>>=</tt>)</h2>
+
+<ol style="list-style-type: decimal">
+
+<li>If either side of the comparison is null then return false.</li>
+
+<li>If both values being compared implement number then the following:
+
+<ul>
+
+<li>If either side is a double then get double value from both sides and compare using given operator.</li>
+
+<li>Else if either side is a float then get float value from both sides and compare using given operator.</li>
+
+<li>Else if either side is a long then get long value from both sides and compare using given operator.</li>
+
+<li>Otherwise get the int value from both sides and compare using given operator.</li>
+ </ul></li>
+
+<li>If both sides are of the same type and are comparable then use the compareTo method to compare values.</li>
+
+<li>If none of the above are met then an exception is thrown.</li>
+</ol></div>
+<div class="section">
+<h2><a name="Stellar_Language_Equality_Check__"></a>Stellar Language Equality Check (<tt>==</tt>, <tt>!=</tt>)</h2>
+<p>Below is how the <tt>==</tt> operator is expected to work:</p>
+
+<ol style="list-style-type: decimal">
+
+<li>If either side of the expression is null then check equality using Java’s <tt>==</tt> expression.</li>
+
+<li>Else if both sides of the expression are of Java’s type Number then:
+
+<ul>
+
+<li>If either side of the expression is a double then use the double value of both sides to test equality.</li>
+
+<li>Else if either side of the expression is a float then use the float value of both sides to test equality.</li>
+
+<li>Else if either side of the expression is a long then use long value of both sides to test equality.</li>
+
+<li>Otherwise use int value of both sides to test equality</li>
+ </ul></li>
+
+<li>Otherwise use equals method compare the left side with the right side.</li>
+</ol>
+<p>The <tt>!=</tt> operator is the negation of the above.</p></div>
+<div class="section">
+<h2><a name="Stellar_Core_Functions"></a>Stellar Core Functions</h2>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th> </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#ABS"> <tt>ABS</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#BIN"> <tt>BIN</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#BLOOM_ADD"> <tt>BLOOM_ADD</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#BLOOM_EXISTS"> <tt>BLOOM_EXISTS</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#BLOOM_INIT"> <tt>BLOOM_INIT</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#BLOOM_MERGE"> <tt>BLOOM_MERGE</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#DAY_OF_MONTH"> <tt>DAY_OF_MONTH</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#DAY_OF_WEEK"> <tt>DAY_OF_WEEK</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#DAY_OF_YEAR"> <tt>DAY_OF_YEAR</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#DOMAIN_REMOVE_SUBDOMAINS"> <tt>DOMAIN_REMOVE_SUBDOMAINS</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#DOMAIN_REMOVE_TLD"> <tt>DOMAIN_REMOVE_TLD</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#DOMAIN_TO_TLD"> <tt>DOMAIN_TO_TLD</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#ENDS_WITH"> <tt>ENDS_WITH</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#ENRICHMENT_EXISTS"> <tt>ENRICHMENT_EXISTS</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#ENRICHMENT_GET"> <tt>ENRICHMENT_GET</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#FILL_LEFT"> <tt>FILL_LEFT</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#FILL_RIGHT"> <tt>FILL_RIGHT</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#FORMAT"> <tt>FORMAT</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#HLLP_CARDINALITY"> <tt>HLLP_CARDINALITY</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#HLLP_INIT"> <tt>HLLP_INIT</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#HLLP_MERGE"> <tt>HLLP_MERGE</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#HLLP_OFFER"> <tt>HLLP_OFFER</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#GEO_GET"> <tt>GEO_GET</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#GET"> <tt>GET</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#GET_FIRST"> <tt>GET_FIRST</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#GET_LAST"> <tt>GET_LAST</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#IN_SUBNET"> <tt>IN_SUBNET</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#IS_DATE"> <tt>IS_DATE</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#IS_DOMAIN"> <tt>IS_DOMAIN</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#IS_EMAIL"> <tt>IS_EMAIL</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#IS_EMPTY"> <tt>IS_EMPTY</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#IS_INTEGER"> <tt>IS_INTEGER</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#IS_IP"> <tt>IS_IP</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#IS_URL"> <tt>IS_URL</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#JOIN"> <tt>JOIN</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#KAFKA_GET"> <tt>KAFKA_GET</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#KAFKA_PROPS"> <tt>KAFKA_PROPS</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#KAFKA_PUT"> <tt>KAFKA_PUT</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#KAFKA_TAIL"> <tt>KAFKA_TAIL</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#LENGTH"> <tt>LENGTH</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#MAAS_GET_ENDPOINT"> <tt>MAAS_GET_ENDPOINT</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#MAAS_MODEL_APPLY"> <tt>MAAS_MODEL_APPLY</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#MAP_EXISTS"> <tt>MAP_EXISTS</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#MONTH"> <tt>MONTH</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#PROFILE_GET"> <tt>PROFILE_GET</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#PROFILE_FIXED"> <tt>PROFILE_FIXED</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#PROTOCOL_TO_NAME"> <tt>PROTOCOL_TO_NAME</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#REGEXP_MATCH"> <tt>REGEXP_MATCH</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#SPLIT"> <tt>SPLIT</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#STARTS_WITH"> <tt>STARTS_WITH</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_ADD"> <tt>STATS_ADD</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_BIN"> <tt>STATS_BIN</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_COUNT"> <tt>STATS_COUNT</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_GEOMETRIC_MEAN"> <tt>STATS_GEOMETRIC_MEAN</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_INIT"> <tt>STATS_INIT</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_KURTOSIS"> <tt>STATS_KURTOSIS</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_MAX"> <tt>STATS_MAX</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_MEAN"> <tt>STATS_MEAN</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_MERGE"> <tt>STATS_MERGE</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_MIN"> <tt>STATS_MIN</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_PERCENTILE"> <tt>STATS_PERCENTILE</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_POPULATION_VARIANCE"> <tt>STATS_POPULATION_VARIANCE</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_QUADRATIC_MEAN"> <tt>STATS_QUADRATIC_MEAN</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_SD"> <tt>STATS_SD</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_SKEWNESS"> <tt>STATS_SKEWNESS</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_SUM"> <tt>STATS_SUM</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_SUM_LOGS"> <tt>STATS_SUM_LOGS</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_SUM_SQUARES"> <tt>STATS_SUM_SQUARES</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_VARIANCE"> <tt>STATS_VARIANCE</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#STRING_ENTROPY"> <tt>STRING_ENTROPY</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#SYSTEM_ENV_GET"> <tt>SYSTEM_ENV_GET</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#SYSTEM_PROPERTY_GET"> <tt>SYSTEM_PROPERTY_GET</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#TO_DOUBLE"> <tt>TO_DOUBLE</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#TO_EPOCH_TIMESTAMP"> <tt>TO_EPOCH_TIMESTAMP</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#TO_FLOAT"> <tt>TO_FLOAT</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#TO_INTEGER"> <tt>TO_INTEGER</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#TO_LONG"> <tt>TO_LONG</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#TO_LOWER"> <tt>TO_LOWER</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#TO_STRING"> <tt>TO_STRING</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#TO_UPPER"> <tt>TO_UPPER</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#TRIM"> <tt>TRIM</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#URL_TO_HOST"> <tt>URL_TO_HOST</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#URL_TO_PATH"> <tt>URL_TO_PATH</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#URL_TO_PORT"> <tt>URL_TO_PORT</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#URL_TO_PROTOCOL"> <tt>URL_TO_PROTOCOL</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#WEEK_OF_MONTH"> <tt>WEEK_OF_MONTH</tt></a> </td>
+ </tr>
+
+<tr class="b">
+
+<td><a href="#WEEK_OF_YEAR"> <tt>WEEK_OF_YEAR</tt></a> </td>
+ </tr>
+
+<tr class="a">
+
+<td><a href="#YEAR"> <tt>YEAR</tt></a> </td>
+ </tr>
+ </tbody>
+</table>
+<div class="section">
+<h3><a name="BLOOM_ADD"></a><tt>BLOOM_ADD</tt></h3>
+
+<ul>
+
+<li>Description: Adds an element to the bloom filter passed in</li>
+
+<li>Input:
+
+<ul>
+
+<li>bloom - The bloom filter</li>
+
+<li>value* - The values to add</li>
+ </ul></li>
+
+<li>Returns: Bloom Filter</li>
+</ul></div>
+<div class="section">
+<h3><a name="BLOOM_EXISTS"></a><tt>BLOOM_EXISTS</tt></h3>
+
+<ul>
+
+<li>Description: If the bloom filter contains the value</li>
+
+<li>Input:
+
+<ul>
+
+<li>bloom - The bloom filter</li>
+
+<li>value - The value to check</li>
+ </ul></li>
+
+<li>Returns: True if the filter might contain the value and false otherwise</li>
+</ul></div>
+<div class="section">
+<h3><a name="BLOOM_INIT"></a><tt>BLOOM_INIT</tt></h3>
+
+<ul>
+
+<li>Description: Returns an empty bloom filter</li>
+
+<li>Input:
+
+<ul>
+
+<li>expectedInsertions - The expected insertions</li>
+
+<li>falsePositiveRate - The false positive rate you are willing to tolerate</li>
+ </ul></li>
+
+<li>Returns: Bloom Filter</li>
+</ul></div>
+<div class="section">
+<h3><a name="BLOOM_MERGE"></a><tt>BLOOM_MERGE</tt></h3>
+
+<ul>
+
+<li>Description: Returns a merged bloom filter</li>
+
+<li>Input:
+
+<ul>
+
+<li>bloomfilters - A list of bloom filters to merge</li>
+ </ul></li>
+
+<li>Returns: Bloom Filter or null if the list is empty</li>
+</ul></div>
+<div class="section">
+<h3><a name="DAY_OF_MONTH"></a><tt>DAY_OF_MONTH</tt></h3>
+
+<ul>
+
+<li>Description: The numbered day within the month. The first day within the month has a value of 1.</li>
+
+<li>Input:
+
+<ul>
+
+<li>dateTime - The datetime as a long representing the milliseconds since unix epoch</li>
+ </ul></li>
+
+<li>Returns: The numbered day within the month.</li>
+</ul></div>
+<div class="section">
+<h3><a name="DAY_OF_WEEK"></a><tt>DAY_OF_WEEK</tt></h3>
+
+<ul>
+
+<li>Description: The numbered day within the week. The first day of the week, Sunday, has a value of 1.</li>
+
+<li>Input:
+
+<ul>
+
+<li>dateTime - The datetime as a long representing the milliseconds since unix epoch</li>
+ </ul></li>
+
+<li>Returns: The numbered day within the week.</li>
+</ul></div>
+<div class="section">
+<h3><a name="DAY_OF_YEAR"></a><tt>DAY_OF_YEAR</tt></h3>
+
+<ul>
+
+<li>Description: The day number within the year. The first day of the year has value of 1.</li>
+
+<li>Input:
+
+<ul>
+
+<li>dateTime - The datetime as a long representing the milliseconds since unix epoch</li>
+ </ul></li>
+
+<li>Returns: The day number within the year.</li>
+</ul></div>
+<div class="section">
+<h3><a name="DOMAIN_REMOVE_SUBDOMAINS"></a><tt>DOMAIN_REMOVE_SUBDOMAINS</tt></h3>
+
+<ul>
+
+<li>Description: Removes the subdomains from a domain.</li>
+
+<li>Input:
+
+<ul>
+
+<li>domain - Fully qualified domain name</li>
+ </ul></li>
+
+<li>Returns: The domain without the subdomains. (for example, DOMAIN_REMOVE_SUBDOMAINS(‘mail.yahoo.com’) yields ‘yahoo.com’)</li>
+</ul></div>
+<div class="section">
+<h3><a name="DOMAIN_REMOVE_TLD"></a><tt>DOMAIN_REMOVE_TLD</tt></h3>
+
+<ul>
+
+<li>Description: Removes the top level domain (TLD) suffix from a domain.</li>
+
+<li>Input:
+
+<ul>
+
+<li>domain - Fully qualified domain name</li>
+ </ul></li>
+
+<li>Returns: The domain without the TLD. (for example, DOMAIN_REMOVE_TLD(‘mail.yahoo.co.uk’) yields ‘mail.yahoo’)</li>
+</ul></div>
+<div class="section">
+<h3><a name="DOMAIN_TO_TLD"></a><tt>DOMAIN_TO_TLD</tt></h3>
+
+<ul>
+
+<li>Description: Extracts the top level domain from a domain</li>
+
+<li>Input:
+
+<ul>
+
+<li>domain - Fully qualified domain name</li>
+ </ul></li>
+
+<li>Returns: The TLD of the domain. (for example, DOMAIN_TO_TLD(‘mail.yahoo.co.uk’) yields ‘co.uk’)</li>
+</ul></div>
+<div class="section">
+<h3><a name="ENDS_WITH"></a><tt>ENDS_WITH</tt></h3>
+
+<ul>
+
+<li>Description: Determines whether a string ends with a specified suffix</li>
+
+<li>Input:
+
+<ul>
+
+<li>string - The string to test</li>
+
+<li>suffix - The proposed suffix</li>
+ </ul></li>
+
+<li>Returns: True if the string ends with the specified suffix and false if otherwise</li>
+</ul></div>
+<div class="section">
+<h3><a name="ENRICHMENT_EXISTS"></a><tt>ENRICHMENT_EXISTS</tt></h3>
+
+<ul>
+
+<li>Description: Interrogates the HBase table holding the simple hbase enrichment data and returns whether the enrichment type and indicator are in the table.</li>
+
+<li>Input:
+
+<ul>
+
+<li>enrichment_type - The enrichment type</li>
+
+<li>indicator - The string indicator to look up</li>
+
+<li>nosql_table - The NoSQL Table to use</li>
+
+<li>column_family - The Column Family to use</li>
+ </ul></li>
+
+<li>Returns: True if the enrichment indicator exists and false otherwise</li>
+</ul></div>
+<div class="section">
+<h3><a name="ENRICHMENT_GET"></a><tt>ENRICHMENT_GET</tt></h3>
+
+<ul>
+
+<li>Description: Interrogates the HBase table holding the simple hbase enrichment data and retrieves the tabular value associated with the enrichment type and indicator.</li>
+
+<li>Input:
+
+<ul>
+
+<li>enrichment_type - The enrichment type</li>
+
+<li>indicator - The string indicator to look up</li>
+
+<li>nosql_table - The NoSQL Table to use</li>
+
+<li>column_family - The Column Family to use</li>
+ </ul></li>
+
+<li>Returns: A Map associated with the indicator and enrichment type. Empty otherwise.</li>
+</ul></div>
+<div class="section">
+<h3><a name="FILL_LEFT"></a><tt>FILL_LEFT</tt></h3>
+
+<ul>
+
+<li>Description: Fills or pads a given string with a given character, to a given length on the left</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - string</li>
+
+<li>fill - the fill character</li>
+
+<li>len - the required length</li>
+ </ul></li>
+
+<li>Returns: the filled string</li>
+</ul></div>
+<div class="section">
+<h3><a name="FILL_RIGHT"></a><tt>FILL_RIGHT</tt></h3>
+
+<ul>
+
+<li>Description: Fills or pads a given string with a given character, to a given length on the right</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - string</li>
+
+<li>fill - the fill character string</li>
+
+<li>len - the required length</li>
+ </ul></li>
+
+<li>Returns: Last element of the list</li>
+</ul></div>
+<div class="section">
+<h3><a name="FORMAT"></a><tt>FORMAT</tt></h3>
+
+<ul>
+
+<li>Description: Returns a formatted string using the specified format string and arguments. Uses Java’s string formatting conventions.</li>
+
+<li>Input:
+
+<ul>
+
+<li>format - string</li>
+
+<li>arguments… - object(s)</li>
+ </ul></li>
+
+<li>Returns: A formatted string.</li>
+</ul></div>
+<div class="section">
+<h3><a name="GEO_GET"></a><tt>GEO_GET</tt></h3>
+
+<ul>
+
+<li>Description: Look up an IPV4 address and returns geographic information about it</li>
+
+<li>Input:
+
+<ul>
+
+<li>ip - The IPV4 address to lookup</li>
+
+<li>fields - Optional list of GeoIP fields to grab. Options are locID, country, city postalCode, dmaCode, latitude, longitude, location_point</li>
+ </ul></li>
+
+<li>Returns: If a Single field is requested a string of the field, If multiple fields a map of string of the fields, and null otherwise</li>
+</ul></div>
+<div class="section">
+<h3><a name="GET"></a><tt>GET</tt></h3>
+
+<ul>
+
+<li>Description: Returns the i’th element of the list</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - List</li>
+
+<li>i - The index (0-based)</li>
+ </ul></li>
+
+<li>Returns: First element of the list</li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_FIRST"></a><tt>GET_FIRST</tt></h3>
+
+<ul>
+
+<li>Description: Returns the first element of the list</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - List</li>
+ </ul></li>
+
+<li>Returns: First element of the list</li>
+</ul></div>
+<div class="section">
+<h3><a name="GET_LAST"></a><tt>GET_LAST</tt></h3>
+
+<ul>
+
+<li>Description: Returns the last element of the list</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - List</li>
+ </ul></li>
+
+<li>Returns: Last element of the list</li>
+</ul></div>
+<div class="section">
+<h3><a name="IN_SUBNET"></a><tt>IN_SUBNET</tt></h3>
+
+<ul>
+
+<li>Description: Returns true if an IP is within a subnet range.</li>
+
+<li>Input:
+
+<ul>
+
+<li>ip - The IP address in string form</li>
+
+<li>cidr+ - One or more IP ranges specified in CIDR notation (for example 192.168.0.0/24)</li>
+ </ul></li>
+
+<li>Returns: True if the IP address is within at least one of the network ranges and false if otherwise</li>
+</ul></div>
+<div class="section">
+<h3><a name="IS_DATE"></a><tt>IS_DATE</tt></h3>
+
+<ul>
+
+<li>Description: Determines if the date contained in the string conforms to the specified format.</li>
+
+<li>Input:
+
+<ul>
+
+<li>date - The date in string form</li>
+
+<li>format - The format of the date</li>
+ </ul></li>
+
+<li>Returns: True if the date is in the specified format and false if otherwise.</li>
+</ul></div>
+<div class="section">
+<h3><a name="IS_DOMAIN"></a><tt>IS_DOMAIN</tt></h3>
+
+<ul>
+
+<li>Description: Tests if a string refers to a valid domain name. Domain names are evaluated according to the standards RFC1034 section 3, and RFC1123 section 2.1.</li>
+
+<li>Input:
+
+<ul>
+
+<li>address - The string to test</li>
+ </ul></li>
+
+<li>Returns: True if the string refers to a valid domain name and false if otherwise</li>
+</ul></div>
+<div class="section">
+<h3><a name="IS_EMAIL"></a><tt>IS_EMAIL</tt></h3>
+
+<ul>
+
+<li>Description: Tests if a string is a valid email address</li>
+
+<li>Input:
+
+<ul>
+
+<li>address - The string to test</li>
+ </ul></li>
+
+<li>Returns: True if the string is a valid email address and false if otherwise.</li>
+</ul></div>
+<div class="section">
+<h3><a name="IS_EMPTY"></a><tt>IS_EMPTY</tt></h3>
+
+<ul>
+
+<li>Description: Returns true if string or collection is empty or null and false if otherwise.</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - Object of string or collection type (for example, list)</li>
+ </ul></li>
+
+<li>Returns: True if the string or collection is empty or null and false if otherwise.</li>
+</ul></div>
+<div class="section">
+<h3><a name="IS_INTEGER"></a><tt>IS_INTEGER</tt></h3>
+
+<ul>
+
+<li>Description: Determines whether or not an object is an integer.</li>
+
+<li>Input:
+
+<ul>
+
+<li>x - The object to test</li>
+ </ul></li>
+
+<li>Returns: True if the object can be converted to an integer and false if otherwise.</li>
+</ul></div>
+<div class="section">
+<h3><a name="IS_IP"></a><tt>IS_IP</tt></h3>
+
+<ul>
+
+<li>Description: Determine if an string is an IP or not.</li>
+
+<li>Input:
+
+<ul>
+
+<li>ip - An object which we wish to test is an ip</li>
+
+<li>type (optional) - Object of string or collection type (e.g. list) one of IPV4 or IPV6 or both. The default is IPV4.</li>
+ </ul></li>
+
+<li>Returns: True if the string is an IP and false otherwise.</li>
+</ul></div>
+<div class="section">
+<h3><a name="IS_URL"></a><tt>IS_URL</tt></h3>
+
+<ul>
+
+<li>Description: Tests if a string is a valid URL</li>
+
+<li>Input:
+
+<ul>
+
+<li>url - The string to test</li>
+ </ul></li>
+
+<li>Returns: True if the string is a valid URL and false if otherwise.</li>
+</ul></div>
+<div class="section">
+<h3><a name="JOIN"></a><tt>JOIN</tt></h3>
+
+<ul>
+
+<li>Description: Joins the components in the list of strings with the specified delimiter.</li>
+
+<li>Input:
+
+<ul>
+
+<li>list - List of strings</li>
+
+<li>delim - String delimiter</li>
+ </ul></li>
+
+<li>Returns: String</li>
+</ul></div>
+<div class="section">
+<h3><a name="KAFKA_GET"></a><tt>KAFKA_GET</tt></h3>
+
+<ul>
+
+<li>Description: Retrieves messages from a Kafka topic. Subsequent calls will continue retrieving messages sequentially from the original offset.</li>
+
+<li>Input:
+
+<ul>
+
+<li>topic - The name of the Kafka topic.</li>
+
+<li>count - The number of Kafka messages to retrieve.</li>
+
+<li>config - Optional map of key/values that override any global properties.</li>
+ </ul></li>
+
+<li>Returns: List of String</li>
+</ul></div>
+<div class="section">
+<h3><a name="KAFKA_PROPS"></a><tt>KAFKA_PROPS</tt></h3>
+
+<ul>
+
+<li>Description: Retrieves the Kafka properties that are used by other KAFKA_* functions like KAFKA_GET and KAFKA_PUT. The Kafka properties are compiled from a set of default properties, the global properties, and any overrides.</li>
+
+<li>Input:
+
+<ul>
+
+<li>config - An optional map of key/values that override any global properties.</li>
+ </ul></li>
+
+<li>Returns: Map of key/value pairs</li>
+</ul></div>
+<div class="section">
+<h3><a name="KAFKA_PUT"></a><tt>KAFKA_PUT</tt></h3>
+
+<ul>
+
+<li>Description: Sends messages to a Kafka topic.</li>
+
+<li>Input:
+
+<ul>
+
+<li>topic - The name of the Kafka topic.</li>
+
+<li>messages - A list of messages to write.</li>
+
+<li>config - Optional map of key/values that override any global properties.</li>
+ </ul></li>
+
+<li>Returns: n/a</li>
+</ul></div>
+<div class="section">
+<h3><a name="KAFKA_TAIL"></a><tt>KAFKA_TAIL</tt></h3>
+
+<ul>
+
+<li>Description: etrieves messages from a Kafka topic always starting with the most recent message first.</li>
+
+<li>Input:
+
+<ul>
+
+<li>topic - The name of the Kafka topic.</li>
+
+<li>count - The number of Kafka messages to retrieve.</li>
+
+<li>config - Optional map of key/values that override any global properties.</li>
+ </ul></li>
+
+<li>Returns: List of String</li>
+</ul></div>
+<div class="section">
+<h3><a name="LENGTH"></a><tt>LENGTH</tt></h3>
+
+<ul>
+
+<li>Description: Returns the length of a string or size of a collection. Returns 0 for empty or null Strings</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - Object of string or collection type (e.g. list)</li>
+ </ul></li>
+
+<li>Returns: Integer</li>
+</ul></div>
+<div class="section">
+<h3><a name="MAAS_GET_ENDPOINT"></a><tt>MAAS_GET_ENDPOINT</tt></h3>
+
+<ul>
+
+<li>Description: Inspects ZooKeeper and returns a map containing the name, version and url for the model referred to by the input parameters.</li>
+
+<li>Input:
+
+<ul>
+
+<li>model_name - The name of the model</li>
+
+<li>model_version - The optional version of the model. If the model version is not specified, the most current version is used.</li>
+ </ul></li>
+
+<li>Returns: A map containing the name, version, and url for the REST endpoint (fields named name, version and url). Note that the output of this function is suitable for input into the first argument of MAAS_MODEL_APPLY.</li>
+</ul></div>
+<div class="section">
+<h3><a name="MAAS_MODEL_APPLY"></a><tt>MAAS_MODEL_APPLY</tt></h3>
+
+<ul>
+
+<li>Description: Returns the output of a model deployed via Model as a Service. NOTE: Results are cached locally for 10 minutes.</li>
+
+<li>Input:
+
+<ul>
+
+<li>endpoint - A map containing the name, version, and url for the REST endpoint</li>
+
+<li>function - The optional endpoint path; default is ‘apply’</li>
+
+<li>model_args - A Dictionary of arguments for the model (these become request params)</li>
+ </ul></li>
+
+<li>Returns: The output of the model deployed as a REST endpoint in Map form. Assumes REST endpoint returns a JSON Map.</li>
+</ul></div>
+<div class="section">
+<h3><a name="MAP_EXISTS"></a><tt>MAP_EXISTS</tt></h3>
+
+<ul>
+
+<li>Description: Checks for existence of a key in a map.</li>
+
+<li>Input:
+
+<ul>
+
+<li>key - The key to check for existence</li>
+
+<li>map - The map to check for existence of the key</li>
+ </ul></li>
+
+<li>Returns: True if the key is found in the map and false if otherwise.</li>
+</ul></div>
+<div class="section">
+<h3><a name="MAP_GET"></a><tt>MAP_GET</tt></h3>
+
+<ul>
+
+<li>Description: Gets the value associated with a key from a map</li>
+
+<li>Input:
+
+<ul>
+
+<li>key - The key</li>
+
+<li>map - The map</li>
+
+<li>default - Optionally the default value to return if the key is not in the map.</li>
+ </ul></li>
+
+<li>Returns: The object associated with the key in the map. If no value is associated with the key and default is specified, then default is returned. If no value is associated with the key or default, then null is returned.</li>
+</ul></div>
+<div class="section">
+<h3><a name="MONTH"></a><tt>MONTH</tt></h3>
+
+<ul>
+
+<li>Description: The number representing the month. The first month, January, has a value of 0.</li>
+
+<li>Input:
+
+<ul>
+
+<li>dateTime - The datetime as a long representing the milliseconds since unix epoch</li>
+ </ul></li>
+
+<li>Returns: The current month (0-based).</li>
+</ul></div>
+<div class="section">
+<h3><a name="PROFILE_GET"></a><tt>PROFILE_GET</tt></h3>
+
+<ul>
+
+<li>Description: Retrieves a series of values from a stored profile.</li>
+
+<li>Input:
+
+<ul>
+
+<li>profile - The name of the profile.</li>
+
+<li>entity - The name of the entity.</li>
+
+<li>periods - The list of profile periods to grab. These are ProfilePeriod objects.</li>
+
+<li>groups_list - Optional, must correspond to the ‘groupBy’ list used in profile creation - List (in square brackets) of groupBy values used to filter the profile. Default is the empty list, meaning groupBy was not used when creating the profile.</li>
+
+<li>config_overrides - Optional - Map (in curly braces) of name:value pairs, each overriding the global config parameter of the same name. Default is the empty Map, meaning no overrides.</li>
+ </ul></li>
+
+<li>Returns: The selected profile measurements.</li>
+</ul></div>
+<div class="section">
+<h3><a name="PROFILE_FIXED"></a><tt>PROFILE_FIXED</tt></h3>
+
+<ul>
+
+<li>Description: The profile periods associated with a fixed lookback starting from now</li>
+
+<li>Input:
+
+<ul>
+
+<li>durationAgo - How long ago should values be retrieved from?</li>
+
+<li>units - The units of ‘durationAgo’.</li>
+
+<li>config_overrides - Optional - Map (in curly braces) of name:value pairs, each overriding the global config parameter of the same name. Default is the empty Map, meaning no overrides.</li>
+ </ul></li>
+
+<li>Returns: The selected profile measurement timestamps. These are ProfilePeriod objects.</li>
+</ul></div>
+<div class="section">
+<h3><a name="PROTOCOL_TO_NAME"></a><tt>PROTOCOL_TO_NAME</tt></h3>
+
+<ul>
+
+<li>Description: Converts the IANA protocol number to the protocol name</li>
+
+<li>Input:
+
+<ul>
+
+<li>IANA Number</li>
+ </ul></li>
+
+<li>Returns: The protocol name associated with the IANA number.</li>
+</ul></div>
+<div class="section">
+<h3><a name="REGEXP_MATCH"></a><tt>REGEXP_MATCH</tt></h3>
+
+<ul>
+
+<li>Description: Determines whether a regex matches a string</li>
+
+<li>Input:
+
+<ul>
+
+<li>string - The string to test</li>
+
+<li>pattern - The proposed regex pattern</li>
+ </ul></li>
+
+<li>Returns: True if the regex pattern matches the string and false if otherwise.</li>
+</ul></div>
+<div class="section">
+<h3><a name="STRING_ENTROPY"></a><tt>STRING_ENTROPY</tt></h3>
+
+<ul>
+
+<li>Description: Computes the base-2 shannon entropy of a string.</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - String</li>
+ </ul></li>
+
+<li>Returns: The base-2 shannon entropy of the string (<a class="externalLink" href="https://en.wikipedia.org/wiki/Entropy_(information_theory)#Definition)">https://en.wikipedia.org/wiki/Entropy_(information_theory)#Definition)</a>. The unit of this is bits.</li>
+</ul></div>
+<div class="section">
+<h3><a name="SPLIT"></a><tt>SPLIT</tt></h3>
+
+<ul>
+
+<li>Description: Splits the string by the delimiter.</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - String to split</li>
+
+<li>delim - String delimiter</li>
+ </ul></li>
+
+<li>Returns: List of strings</li>
+</ul></div>
+<div class="section">
+<h3><a name="STARTS_WITH"></a><tt>STARTS_WITH</tt></h3>
+
+<ul>
+
+<li>Description: Determines whether a string starts with a prefix</li>
+
+<li>Input:
+
+<ul>
+
+<li>string - The string to test</li>
+
+<li>prefix - The proposed prefix</li>
+ </ul></li>
+
+<li>Returns: True if the string starts with the specified prefix and false if otherwise</li>
+</ul></div>
+<div class="section">
+<h3><a name="SYSTEM_ENV_GET"></a><tt>SYSTEM_ENV_GET</tt></h3>
+
+<ul>
+
+<li>Description: Returns the value associated with an environment variable</li>
+
+<li>Input:
+
+<ul>
+
+<li>env_var - Environment variable name to get the value for</li>
+ </ul></li>
+
+<li>Returns: String</li>
+</ul></div>
+<div class="section">
+<h3><a name="SYSTEM_PROPERTY_GET"></a><tt>SYSTEM_PROPERTY_GET</tt></h3>
+
+<ul>
+
+<li>Description: Returns the value associated with a Java system property</li>
+
+<li>Input:
+
+<ul>
+
+<li>key - Property to get the value for</li>
+ </ul></li>
+
+<li>Returns: String</li>
+</ul></div>
+<div class="section">
+<h3><a name="TO_DOUBLE"></a><tt>TO_DOUBLE</tt></h3>
+
+<ul>
+
+<li>Description: Transforms the first argument to a double precision number</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - Object of string or numeric type</li>
+ </ul></li>
+
+<li>Returns: Double version of the first argument</li>
+</ul></div>
+<div class="section">
+<h3><a name="TO_EPOCH_TIMESTAMP"></a><tt>TO_EPOCH_TIMESTAMP</tt></h3>
+
+<ul>
+
+<li>Description: Returns the epoch timestamp of the dateTime in the specified format. If the format does not have a timestamp and you wish to assume a given timestamp, you may specify the timezone optionally.</li>
+
+<li>Input:
+
+<ul>
+
+<li>dateTime - DateTime in String format</li>
+
+<li>format - DateTime format as a String</li>
+
+<li>timezone - Optional timezone in String format</li>
+ </ul></li>
+
+<li>Returns: Epoch timestamp</li>
+</ul></div>
+<div class="section">
+<h3><a name="TO_FOAT"></a><tt>TO_FOAT</tt></h3>
+
+<ul>
+
+<li>Description: Transforms the first argument to a float</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - Object of string or numeric type</li>
+ </ul></li>
+
+<li>Returns: Float version of the first argument</li>
+</ul></div>
+<div class="section">
+<h3><a name="TO_INTEGER"></a><tt>TO_INTEGER</tt></h3>
+
+<ul>
+
+<li>Description: Transforms the first argument to an integer</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - Object of string or numeric type</li>
+ </ul></li>
+
+<li>Returns: Integer version of the first argument</li>
+</ul></div>
+<div class="section">
+<h3><a name="TO_LONG"></a><tt>TO_LONG</tt></h3>
+
+<ul>
+
+<li>Description: Transforms the first argument to a long integer</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - Object of string or numeric type</li>
+ </ul></li>
+
+<li>Returns: Long version of the first argument</li>
+</ul></div>
+<div class="section">
+<h3><a name="TO_LOWER"></a><tt>TO_LOWER</tt></h3>
+
+<ul>
+
+<li>Description: Transforms the first argument to a lowercase string</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - String</li>
+ </ul></li>
+
+<li>Returns: Lowercase string</li>
+</ul></div>
+<div class="section">
+<h3><a name="TO_STRING"></a><tt>TO_STRING</tt></h3>
+
+<ul>
+
+<li>Description: Transforms the first argument to a string</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - Object</li>
+ </ul></li>
+
+<li>Returns: String</li>
+</ul></div>
+<div class="section">
+<h3><a name="TO_UPPER"></a><tt>TO_UPPER</tt></h3>
+
+<ul>
+
+<li>Description: Transforms the first argument to an uppercase string</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - String</li>
+ </ul></li>
+
+<li>Returns: Uppercase string</li>
+</ul></div>
+<div class="section">
+<h3><a name="TRIM"></a><tt>TRIM</tt></h3>
+
+<ul>
+
+<li>Description: Trims whitespace from both sides of a string.</li>
+
+<li>Input:
+
+<ul>
+
+<li>input - String</li>
+ </ul></li>
+
+<li>Returns: String</li>
+</ul></div>
+<div class="section">
+<h3><a name="URL_TO_HOST"></a><tt>URL_TO_HOST</tt></h3>
+
+<ul>
+
+<li>Description: Extract the hostname from a URL.</li>
+
+<li>Input:
+
+<ul>
+
+<li>url - URL in String form</li>
+ </ul></li>
+
+<li>Returns: The hostname from the URL as a String. e.g. URL_TO_HOST(’<a class="externalLink" href="http://www.yahoo.com/foo'">http://www.yahoo.com/foo'</a>) would yield ‘www.yahoo.com’</li>
+</ul></div>
+<div class="section">
+<h3><a name="URL_TO_PATH"></a><tt>URL_TO_PATH</tt></h3>
+
+<ul>
+
+<li>Description: Extract the path from a URL.</li>
+
+<li>Input:
+
+<ul>
+
+<li>url - URL in String form</li>
+ </ul></li>
+
+<li>Returns: The path from the URL as a String. e.g. URL_TO_PATH(’<a class="externalLink" href="http://www.yahoo.com/foo'">http://www.yahoo.com/foo'</a>) would yield ‘foo’</li>
+</ul></div>
+<div class="section">
+<h3><a name="URL_TO_PORT"></a><tt>URL_TO_PORT</tt></h3>
+
+<ul>
+
+<li>Description: Extract the port from a URL. If the port is not explicitly stated in the URL, then an implicit port is inferred based on the protocol.</li>
+
+<li>Input:
+
+<ul>
+
+<li>url - URL in string form</li>
+ </ul></li>
+
+<li>Returns: The port used in the URL as an integer (for example, URL_TO_PORT(’<a class="externalLink" href="http://www.yahoo.com/foo'">http://www.yahoo.com/foo'</a>) would yield 80)</li>
+</ul></div>
+<div class="section">
+<h3><a name="URL_TO_PROTOCOL"></a><tt>URL_TO_PROTOCOL</tt></h3>
+
+<ul>
+
+<li>Description: Extract the protocol from a URL.</li>
+
+<li>Input:
+
+<ul>
+
+<li>url - URL in String form</li>
+ </ul></li>
+
+<li>Returns: The protocol from the URL as a String. e.g. URL_TO_PROTOCOL(’<a class="externalLink" href="http://www.yahoo.com/foo'">http://www.yahoo.com/foo'</a>) would yield ‘http’</li>
+</ul></div>
+<div class="section">
+<h3><a name="WEEK_OF_MONTH"></a><tt>WEEK_OF_MONTH</tt></h3>
+
+<ul>
+
+<li>Description: The numbered week within the month. The first week within the month has a value of 1.</li>
+
+<li>Input:
+
+<ul>
+
+<li>dateTime - The datetime as a long representing the milliseconds since unix epoch</li>
+ </ul></li>
+
+<li>Returns: The numbered week within the month.</li>
+</ul></div>
+<div class="section">
+<h3><a name="WEEK_OF_YEAR"></a><tt>WEEK_OF_YEAR</tt></h3>
+
+<ul>
+
+<li>Description: The numbered week within the year. The first week in the year has a value of 1.</li>
+
+<li>Input:
+
+<ul>
+
+<li>dateTime - The datetime as a long representing the milliseconds since unix epoch</li>
+ </ul></li>
+
+<li>Returns: The numbered week within the year.</li>
+</ul></div>
+<div class="section">
+<h3><a name="YEAR"></a><tt>YEAR</tt></h3>
+
+<ul>
+
+<li>Description: The number representing the year.</li>
+
+<li>Input:
+
+<ul>
+
+<li>dateTime - The datetime as a long representing the milliseconds since unix epoch</li>
+ </ul></li>
+
+<li>Returns: The current year</li>
+</ul>
+<p>The following is an example query (i.e. a function which returns a boolean) which would be seen possibly in threat triage:</p>
+<p><tt>IN_SUBNET( ip, '192.168.0.0/24') or ip in [ '10.0.0.1', '10.0.0.2' ] or exists(is_local)</tt></p>
+<p>This evaluates to true precisely when one of the following is true:</p>
+
+<ul>
+
+<li>The value of the <tt>ip</tt> field is in the <tt>192.168.0.0/24</tt> subnet</li>
+
+<li>The value of the <tt>ip</tt> field is <tt>10.0.0.1</tt> or <tt>10.0.0.2</tt></li>
+
+<li>The field <tt>is_local</tt> exists</li>
+</ul>
+<p>The following is an example transformation which might be seen in a field transformation:</p>
+<p><tt>TO_EPOCH_TIMESTAMP(timestamp, 'yyyy-MM-dd HH:mm:ss', MAP_GET(dc, dc2tz, 'UTC'))</tt></p>
+<p>For a message with a <tt>timestamp</tt> and <tt>dc</tt> field, we want to set the transform the timestamp to an epoch timestamp given a timezone which we will lookup in a separate map, called <tt>dc2tz</tt>.</p>
+<p>This will convert the timestamp field to an epoch timestamp based on the </p>
+
+<ul>
+
+<li>Format <tt>yyyy-MM-dd HH:mm:ss</tt></li>
+
+<li>The value in <tt>dc2tz</tt> associated with the value associated with field <tt>dc</tt>, defaulting to <tt>UTC</tt></li>
+</ul></div></div>
+<div class="section">
+<h2><a name="Stellar_Shell"></a>Stellar Shell</h2>
+<p>A REPL (Read Eval Print Loop) for the Stellar language that helps in debugging, troubleshooting and learning Stellar. The Stellar DSL (domain specific language) is used to act upon streaming data within Apache Storm. It is difficult to troubleshoot Stellar when it can only be executed within a Storm topology. This REPL is intended to help mitigate that problem by allowing a user to replicate data encountered in production, isolate initialization errors, or understand function resolution problems.</p>
+<p>The shell supports customization via <tt>~/.inputrc</tt> as it is backed by a proper readline implementation. </p>
+<p>Shell-like operations are supported such as </p>
+
+<ul>
+
+<li>reverse search via ctrl-r</li>
+
+<li>autocomplete of Stellar functions and variables via tab
+
+<ul>
+
+<li>NOTE: Stellar functions are read via a classpath search which happens in the background. Until that happens, autocomplete will not include function names.</li>
+ </ul></li>
+
+<li>emacs or vi keybindings for edit mode</li>
+</ul>
+<div class="section">
+<h3><a name="Getting_Started"></a>Getting Started</h3>
+
+<div class="source">
+<div class="source">
+<pre>$ $METRON_HOME/bin/stellar
+
+Stellar, Go!
+{es.clustername=metron, es.ip=node1, es.port=9300, es.date.format=yyyy.MM.dd.HH}
+
+[Stellar]>>> %functions
+BLOOM_ADD, BLOOM_EXISTS, BLOOM_INIT, BLOOM_MERGE, DAY_OF_MONTH, DAY_OF_WEEK, DAY_OF_YEAR, ...
+
+[Stellar]>>> ?PROTOCOL_TO_NAME
+PROTOCOL_TO_NAME
+ desc: Convert the IANA protocol number to the protocol name
+ args: IANA Number
+ ret: The protocol name associated with the IANA number.
+
+[Stellar]>>> ip.protocol := 6
+6
+[Stellar]>>> PROTOCOL_TO_NAME(ip.protocol)
+TCP
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Command_Line_Options"></a>Command Line Options</h3>
+
+<div class="source">
+<div class="source">
+<pre>$ $METRON_HOME/bin/stellar -h
+usage: stellar
+ -h,--help Print help
+ -irc,--inputrc <arg> File containing the inputrc if not the default
+ ~/.inputrc
+ -v,--variables <arg> File containing a JSON Map of variables
+ -z,--zookeeper <arg> Zookeeper URL
+ -na,--no_ansi Make the input prompt not use ANSI colors.
+</pre></div></div>
+<div class="section">
+<h4><a name="a-v_--variables"></a><tt>-v, --variables</tt></h4>
+<p><i>Optional</i></p>
+<p>Optionally load a JSON map which contains variable assignments. This is intended to give you the ability to save off a message from Metron and work on it via the REPL.</p></div>
+<div class="section">
+<h4><a name="a-z_--zookeeper"></a><tt>-z, --zookeeper</tt></h4>
+<p><i>Optional</i></p>
+<p>Attempts to connect to Zookeeper and read the Metron global configuration. Stellar functions may require the global configuration to work properly. If found, the global configuration values are printed to the console.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ $METRON_HOME/bin/stellar -z node1:2181
+Stellar, Go!
+{es.clustername=metron, es.ip=node1, es.port=9300, es.date.format=yyyy.MM.dd.HH}
+[Stellar]>>>
+</pre></div></div></div></div>
+<div class="section">
+<h3><a name="Variable_Assignment"></a>Variable Assignment</h3>
+<p>Stellar has no concept of variable assignment. For testing and debugging purposes, it is important to be able to create variables that simulate data contained within incoming messages. The REPL has created a means for a user to perform variable assignment outside of the core Stellar language. This is done via the <tt>:=</tt> operator, such as <tt>foo := 1 + 1</tt> would assign the result of the stellar expression <tt>1 + 1</tt> to the variable <tt>foo</tt>.</p>
+
+<div class="source">
+<div class="source">
+<pre>[Stellar]>>> foo := 2 + 2
+4.0
+[Stellar]>>> 2 + 2
+4.0
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Magic_Commands"></a>Magic Commands</h3>
+<p>The REPL has a set of magic commands that provide the REPL user with information about the Stellar execution environment. The following magic commands are supported.</p>
+<div class="section">
+<h4><a name="afunctions"></a><tt>%functions</tt></h4>
+<p>This command lists all functions resolvable in the Stellar environment. Stellar searches the classpath for Stellar functions. This can make it difficult in some cases to understand which functions are resolvable. </p>
+
+<div class="source">
+<div class="source">
+<pre>[Stellar]>>> %functions
+BLOOM_ADD, BLOOM_EXISTS, BLOOM_INIT, BLOOM_MERGE, DAY_OF_MONTH, DAY_OF_WEEK, DAY_OF_YEAR,
+DOMAIN_REMOVE_SUBDOMAINS, DOMAIN_REMOVE_TLD, DOMAIN_TO_TLD, ENDS_WITH, GET, GET_FIRST,
+GET_LAST, IN_SUBNET, IS_DATE, IS_DOMAIN, IS_EMAIL, IS_EMPTY, IS_INTEGER, IS_IP, IS_URL,
+JOIN, LENGTH, MAAS_GET_ENDPOINT, MAAS_MODEL_APPLY, MAP_EXISTS, MAP_GET, MONTH, PROTOCOL_TO_NAME,
+REGEXP_MATCH, SPLIT, STARTS_WITH, STATS_ADD, STATS_COUNT, STATS_GEOMETRIC_MEAN, STATS_INIT,
+STATS_KURTOSIS, STATS_MAX, STATS_MEAN, STATS_MERGE, STATS_MIN, STATS_PERCENTILE,
+STATS_POPULATION_VARIANCE, STATS_QUADRATIC_MEAN, STATS_SD, STATS_SKEWNESS, STATS_SUM,
+STATS_SUM_LOGS, STATS_SUM_SQUARES, STATS_VARIANCE, TO_DOUBLE, TO_EPOCH_TIMESTAMP, TO_FLOAT,
+TO_INTEGER, TO_LOWER, TO_STRING, TO_UPPER, TRIM, URL_TO_HOST, URL_TO_PATH, URL_TO_PORT,
+URL_TO_PROTOCOL, WEEK_OF_MONTH, WEEK_OF_YEAR, YEAR
+[Stellar]>>>
+</pre></div></div></div>
+<div class="section">
+<h4><a name="avars"></a><tt>%vars</tt></h4>
+<p>Lists all variables in the Stellar environment.</p>
+
+<div class="source">
+<div class="source">
+<pre>Stellar, Go!
+{es.clustername=metron, es.ip=node1, es.port=9300, es.date.format=yyyy.MM.dd.HH}
+[Stellar]>>> %vars
+[Stellar]>>> foo := 2 + 2
+4.0
+[Stellar]>>> %vars
+foo = 4.0
+</pre></div></div></div>
+<div class="section">
+<h4><a name="afunction"></a><tt>?<function></tt></h4>
+<p>Returns formatted documentation of the Stellar function. Provides the description of the function along with the expected arguments.</p>
+
+<div class="source">
+<div class="source">
+<pre>[Stellar]>>> ?BLOOM_ADD
+BLOOM_ADD
+ desc: Adds an element to the bloom filter passed in
+ args: bloom - The bloom filter, value* - The values to add
+ ret: Bloom Filter
+[Stellar]>>> ?IS_EMAIL
+IS_EMAIL
+ desc: Tests if a string is a valid email address
+ args: address - The String to test
+ ret: True if the string is a valid email address and false otherwise.
+[Stellar]>>>
+</pre></div></div>
+<p><a name="Global_Configuration"></a></p>
+<h1>Global Configuration</h1>
+<p>The format of the global enrichment is a JSON String to Object map. This is intended for configuration which is non sensor specific configuration.</p>
+<p>This configuration is stored in zookeeper, but looks something like</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "es.clustername": "metron",
+ "es.ip": "node1",
+ "es.port": "9300",
+ "es.date.format": "yyyy.MM.dd.HH",
+ "fieldValidations" : [
+ {
+ "input" : [ "ip_src_addr", "ip_dst_addr" ],
+ "validation" : "IP",
+ "config" : {
+ "type" : "IPV4"
+ }
+ }
+ ]
+}
+</pre></div></div></div></div></div>
+<div class="section">
+<h2><a name="Validation_Framework"></a>Validation Framework</h2>
+<p>Inside of the global configuration, there is a validation framework in place that enables the validation that messages coming from all parsers are valid. This is done in the form of validation plugins where assertions about fields or whole messages can be made. </p>
+<p>The format for this is a <tt>fieldValidations</tt> field inside of global config. This is associated with an array of field validation objects structured like so:</p>
+
+<ul>
+
+<li><tt>input</tt> : An array of input fields or a single field. If this is omitted, then the whole messages is passed to the validator.</li>
+
+<li><tt>config</tt> : A String to Object map for validation configuration. This is optional if the validation function requires no configuration.</li>
+
+<li><tt>validation</tt> : The validation function to be used. This is one of
+
+<ul>
+
+<li><tt>STELLAR</tt> : Execute a Stellar Language statement. Expects the query string in the <tt>condition</tt> field of the config.</li>
+
+<li><tt>IP</tt> : Validates that the input fields are an IP address. By default, if no configuration is set, it assumes <tt>IPV4</tt>, but you can specify the type by passing in the config by passing in <tt>type</tt> with either <tt>IPV6</tt> or <tt>IPV4</tt> or by passing in a list [<tt>IPV4</tt>,<tt>IPV6</tt>] in which case the input(s) will be validated against both.</li>
+
+<li><tt>DOMAIN</tt> : Validates that the fields are all domains.</li>
+
+<li><tt>EMAIL</tt> : Validates that the fields are all email addresses</li>
+
+<li><tt>URL</tt> : Validates that the fields are all URLs</li>
+
+<li><tt>DATE</tt> : Validates that the fields are a date. Expects <tt>format</tt> in the config.</li>
+
+<li><tt>INTEGER</tt> : Validates that the fields are an integer. String representation of an integer is allowed.</li>
+
+<li><tt>REGEX_MATCH</tt> : Validates that the fields match a regex. Expects <tt>pattern</tt> in the config.</li>
+
+<li><tt>NOT_EMPTY</tt> : Validates that the fields exist and are not empty (after trimming.)</li>
+ </ul></li>
+</ul>
+<p><a name="Management_Utility"></a></p>
+<h1>Management Utility</h1>
+<p>Configurations should be stored on disk in the following structure starting at <tt>$BASE_DIR</tt>:</p>
+
+<ul>
+
+<li>global.json : The global config</li>
+
+<li><tt>sensors</tt> : The subdirectory containing sensor enrichment configuration JSON (e.g. <tt>snort.json</tt>, <tt>bro.json</tt>)</li>
+</ul>
+<p>By default, this directory as deployed by the ansible infrastructure is at <tt>$METRON_HOME/config/zookeeper</tt></p>
+<p>While the configs are stored on disk, they must be loaded into Zookeeper to be used. To this end, there is a utility program to assist in this called <tt>$METRON_HOME/bin/zk_load_config.sh</tt></p>
+<p>This has the following options:</p>
+
+<div class="source">
+<div class="source">
+<pre> -f,--force Force operation
+ -h,--help Generate Help screen
+ -i,--input_dir <DIR> The input directory containing
+ the configuration files named
+ like "$source.json"
+ -m,--mode <MODE> The mode of operation: DUMP,
+ PULL, PUSH
+ -o,--output_dir <DIR> The output directory which will
+ store the JSON configuration
+ from Zookeeper
+ -z,--zk_quorum <host:port,[host:port]*> Zookeeper Quorum URL
+ (zk1:port,zk2:port,...)
+</pre></div></div>
+<p>Usage examples:</p>
+
+<ul>
+
+<li>To dump the existing configs from zookeeper on the singlenode vagrant machine: <tt>$METRON_HOME/bin/zk_load_configs.sh -z node1:2181 -m DUMP</tt></li>
+
+<li>To push the configs into zookeeper on the singlenode vagrant machine: <tt>$METRON_HOME/bin/zk_load_configs.sh -z node1:2181 -m PUSH -i $METRON_HOME/config/zookeeper</tt></li>
+
+<li>To pull the configs from zookeeper to the singlenode vagrant machine disk: <tt>$METRON_HOME/bin/zk_load_configs.sh -z node1:2181 -m PULL -o $METRON_HOME/config/zookeeper -f</tt></li>
+</ul></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
[14/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
METRON-766: Release 0.3.1 closes apache/incubator-metron#477
Project: http://git-wip-us.apache.org/repos/asf/incubator-metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-metron/commit/39059e5b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-metron/tree/39059e5b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-metron/diff/39059e5b
Branch: refs/heads/master
Commit: 39059e5be07796550e8ff7dd054e1e1d2ac93e75
Parents: f1435a6
Author: cstella <ce...@gmail.com>
Authored: Fri Mar 17 10:46:59 2017 -0400
Committer: cstella <ce...@gmail.com>
Committed: Fri Mar 17 10:46:59 2017 -0400
----------------------------------------------------------------------
site/current-book/Upgrading.html | 293 +
.../css/apache-maven-fluido-1.3.0.min.css | 9 +
site/current-book/css/print.css | 23 +
site/current-book/css/site.css | 1 +
.../images/ApacheIncubating_Logo.png | Bin 0 -> 11294 bytes
.../images/accessories-text-editor.png | Bin 0 -> 746 bytes
site/current-book/images/add.gif | Bin 0 -> 397 bytes
.../images/apache-maven-project-2.png | Bin 0 -> 33442 bytes
.../images/application-certificate.png | Bin 0 -> 923 bytes
site/current-book/images/contact-new.png | Bin 0 -> 736 bytes
.../current-book/images/document-properties.png | Bin 0 -> 577 bytes
site/current-book/images/drive-harddisk.png | Bin 0 -> 700 bytes
site/current-book/images/enrichment_arch.png | Bin 0 -> 113606 bytes
site/current-book/images/fix.gif | Bin 0 -> 366 bytes
site/current-book/images/icon_error_sml.gif | Bin 0 -> 633 bytes
site/current-book/images/icon_help_sml.gif | Bin 0 -> 1072 bytes
site/current-book/images/icon_info_sml.gif | Bin 0 -> 638 bytes
site/current-book/images/icon_success_sml.gif | Bin 0 -> 604 bytes
site/current-book/images/icon_warning_sml.gif | Bin 0 -> 625 bytes
site/current-book/images/image-x-generic.png | Bin 0 -> 662 bytes
site/current-book/images/indexing_arch.png | Bin 0 -> 86115 bytes
.../images/internet-web-browser.png | Bin 0 -> 1017 bytes
.../images/logos/build-by-maven-black.png | Bin 0 -> 2294 bytes
.../images/logos/build-by-maven-white.png | Bin 0 -> 2260 bytes
.../current-book/images/logos/maven-feather.png | Bin 0 -> 3330 bytes
site/current-book/images/maas_arch.png | Bin 0 -> 106286 bytes
site/current-book/images/metron-logo.png | Bin 0 -> 21186 bytes
site/current-book/images/network-server.png | Bin 0 -> 536 bytes
site/current-book/images/package-x-generic.png | Bin 0 -> 717 bytes
site/current-book/images/parser_arch.png | Bin 0 -> 81891 bytes
.../images/profiles/pre-release.png | Bin 0 -> 32607 bytes
site/current-book/images/profiles/retired.png | Bin 0 -> 22003 bytes
site/current-book/images/profiles/sandbox.png | Bin 0 -> 33010 bytes
site/current-book/images/remove.gif | Bin 0 -> 607 bytes
site/current-book/images/rss.png | Bin 0 -> 474 bytes
site/current-book/images/update.gif | Bin 0 -> 1090 bytes
site/current-book/images/window-new.png | Bin 0 -> 583 bytes
.../img/glyphicons-halflings-white.png | Bin 0 -> 8777 bytes
site/current-book/img/glyphicons-halflings.png | Bin 0 -> 12799 bytes
site/current-book/index.html | 257 +
.../js/apache-maven-fluido-1.3.0.min.js | 21 +
site/current-book/metron-analytics/index.html | 210 +
.../metron-maas-service/index.html | 474 ++
.../metron-profiler-client/index.html | 427 +
.../metron-analytics/metron-profiler/index.html | 779 ++
.../metron-statistics/HLLP.html | 7587 ++++++++++++++++++
.../metron-statistics/index.html | 916 +++
.../metron-deployment/amazon-ec2/index.html | 521 ++
site/current-book/metron-deployment/index.html | 434 +
.../packaging/docker/ansible-docker/index.html | 239 +
.../packaging/docker/rpm-docker/index.html | 226 +
.../metron-deployment/packer-build/index.html | 274 +
.../metron-deployment/roles/index.html | 271 +
.../metron-deployment/roles/kibana/index.html | 285 +
.../metron-deployment/roles/monit/index.html | 327 +
.../roles/opentaxii/index.html | 419 +
.../roles/pcap_replay/index.html | 294 +
.../roles/sensor-stubs/index.html | 340 +
.../roles/sensor-test-mode/index.html | 284 +
.../vagrant/codelab-platform/index.html | 315 +
.../vagrant/fastcapa-test-platform/index.html | 313 +
.../vagrant/full-dev-platform/index.html | 374 +
.../metron-deployment/vagrant/index.html | 255 +
.../vagrant/quick-dev-platform/index.html | 327 +
site/current-book/metron-docker/index.html | 470 ++
site/current-book/metron-platform/index.html | 249 +
.../metron-platform/metron-api/index.html | 285 +
.../metron-platform/metron-common/index.html | 2233 ++++++
.../metron-data-management/index.html | 1014 +++
.../metron-enrichment/index.html | 670 ++
.../metron-platform/metron-indexing/index.html | 428 +
.../metron-management/index.html | 1602 ++++
.../metron-platform/metron-parsers/index.html | 691 ++
.../metron-pcap-backend/index.html | 376 +
.../metron-sensors/fastcapa/index.html | 195 +
site/current-book/metron-sensors/index.html | 193 +
.../metron-sensors/pycapa/index.html | 262 +
site/documentation/index.md | 24 +-
78 files changed, 25180 insertions(+), 7 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/Upgrading.html
----------------------------------------------------------------------
diff --git a/site/current-book/Upgrading.html b/site/current-book/Upgrading.html
new file mode 100644
index 0000000..0e083cf
--- /dev/null
+++ b/site/current-book/Upgrading.html
@@ -0,0 +1,293 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Upgrading</title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Upgrading</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Upgrading</h1>
+<p><a name="Upgrading"></a></p>
+<p>This document constitutes a per-version listing of changes of configuration which are non-backwards compatible.</p>
+<div class="section">
+<h2><a name="a0.2.0BETA_to_0.3.0"></a>0.2.0BETA to 0.3.0</h2>
+<div class="section">
+<h3><a name="METRON-447:_Monit_fails_to_reload_when_upgrading_from_0.2.0BETA_to_master"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-447">METRON-447: Monit fails to reload when upgrading from 0.2.0BETA to master</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p><tt>/etc/monit.d/enrichment-elasticsearch.monit</tt> was renamed to <tt>/etc/monit.d/indexing-elasticsearch.monit</tt>, however the old file isn’t removed via ansible, which causes the below error during an upgrade: <tt>Starting monit: /etc/monit.d/enrichment-elasticsearch.monit:18: Service
+name conflict, enrichment already defined
+'/usr/local/monit/status_enrichment_topology.sh'</tt></p></div></div>
+<div class="section">
+<h3><a name="METRON-448:Upgrading_via_Ansible_deployment_does_not_add_topology.classpath"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-448">METRON-448:Upgrading via Ansible deployment does not add topology.classpath </a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>When using Ansible to deploy the latest Metron bits to an existing installation, storm-site is not being updated with the new 0.2.1BETA parameter <tt>topology.classpath</tt>. Topologies are unable to find the client configs as a result.</p></div>
+<div class="section">
+<h4><a name="Workaround"></a>Workaround</h4>
+<p>Set the <tt>topology.classpath</tt> property for storm in Ambari to <tt>/etc/hbase/conf:/etc/hadoop/conf</tt></p></div></div></div>
+<div class="section">
+<h2><a name="a0.3.0_to_0.3.1"></a>0.3.0 to 0.3.1</h2>
+<div class="section">
+<h3><a name="METRON-664:_Make_the_index_configuration_per-writer_with_enableddisabled"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-664">METRON-664: Make the index configuration per-writer with enabled/disabled</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>As of 0.3.0 the indexing configuration</p>
+
+<ul>
+
+<li>Is held in the enrichment configuration for a sensor</li>
+
+<li>Has properties which control every writers (i.e. HDFS, solr or elasticsearch).</li>
+</ul>
+<p>In the 0.3.1 release, this configuration has been broken out and control for individual writers are separated.</p>
+<p>Please see the description of the configurations in the indexing <a class="externalLink" href="https://github.com/apache/incubator-metron/tree/Metron_0.3.1/metron-platform/metron-indexing#sensor-indexing-configuration">README</a></p></div>
+<div class="section">
+<h4><a name="Migration"></a>Migration</h4>
+<p>Migrate the configurations from each sensor enrichment configuration and create appropriate configurations for indexing.</p>
+<p>For instance, if a sensor enrichment config for sensor <tt>foo</tt> is in <tt>$METRON_HOME/config/zookeeper/enrichments/foo.json</tt> and looks like</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "index" : "foo",
+ "batchSize" : 100
+}
+</pre></div></div>
+<p>You would create a file to configure each writer for sensor <tt>foo</tt> called <tt>$METRON_HOME/config/zookeeper/indexing/foo.json</tt> with the contents</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "elasticsearch" : {
+ "index" : "foo",
+ "batchSize" : 100,
+ "enabled" : true
+ },
+ "hdfs" : {
+ "index" : "foo",
+ "batchSize" : 100,
+ "enabled" : true
+ }
+}
+</pre></div></div></div></div>
+<div class="section">
+<h3><a name="METRON-675:_Make_Threat_Triage_rules_able_to_be_assigned_names_and_comments"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-675">METRON-675: Make Threat Triage rules able to be assigned names and comments</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>As of 0.3.0, threat triage rules were defined as a simple Map associating a Stellar expression with a score. As of 0.3.1, due to the fact that there may be many threat triage rules, we have made the rules more complex. To help organize these, we have made the threat triage objects in their own right that contain optional name and optional comment fields.</p>
+<p>This essentially makes the risk level rules slightly more complex. The format goes from:</p>
+
+<div class="source">
+<div class="source">
+<pre>"riskLevelRules" : {
+ "stellar expression" : numeric score
+}
+</pre></div></div>
+<p>to:</p>
+
+<div class="source">
+<div class="source">
+<pre>"riskLevelRules" : [
+ {
+ "name" : "optional name",
+ "comment" : "optional comment",
+ "rule" : "stellar expression",
+ "score" : numeric score
+ }
+]
+</pre></div></div></div>
+<div class="section">
+<h4><a name="Migration"></a>Migration</h4>
+<p>For every sensor enrichment configuration, you will need to migrate the <tt>riskLevelRules</tt> section to move from a map to a list of risk level rule objects.</p></div></div>
+<div class="section">
+<h3><a name="METRON-283:_Migrate_Geo_Enrichment_outside_of_MySQL"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-283">METRON-283: Migrate Geo Enrichment outside of MySQL</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>As of 0.3.0, a MySQL database was used for storage and retrieval of GeoIP information during enrichment. As of 0.3.1, the MySQL database is removed in favor of using MaxMind’s binary GeoIP files and stored on HDFS</p>
+<p>After initial setup, this change is transparent and existing enrichment definitions will run as-is.</p></div>
+<div class="section">
+<h4><a name="Migration"></a>Migration</h4>
+<p>While new installs will not require any additional steps, in an existing install a script must be run to retrieve and load the initial data.</p>
+<p>The shell script <tt>geo_enrichment_load.sh</tt> will retrieve MaxMind GeoLite2 data and load data into HDFS, and update the configuration to point to this data. In most cases the following usage will grab the data appropriately:</p>
+
+<div class="source">
+<div class="source">
+<pre>$METRON_HOME/bin/geo_enrichment_load.sh -z <zk_server>:<zk_port>
+</pre></div></div>
+<p>Additional options, including changing the source file location (which can be a file:// location if the GeoIP data is already downloaded), are available with the -h flag and are also detailed in the metron-data-management README.me file.</p>
+<p>One caveat is that this script will NOT update on disk config files. It is recommended to retrieve the configuration using</p>
+
+<div class="source">
+<div class="source">
+<pre>$METRON_HOME/bin/zk_load_configs.sh -z <zk_server>:<zk_port> -m DUMP
+</pre></div></div>
+<p>The new config will be <tt>geo.hdfs.file</tt> in the global section of the configuration. Append this key-value into the global.json in the config directory. A PUSH is unnecessary</p></div></div>
+<div class="section">
+<h3><a name="METRON-684:_Decouple_Timestamp_calculation_from_PROFILE_GET"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-684">METRON-684: Decouple Timestamp calculation from PROFILE_GET</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>During 0.3.1 we decoupled specifying durations for calls to the profiler into a separate function. The consequence is that existing calls to <tt>PROFILE_GET</tt> will need to migrate.</p></div>
+<div class="section">
+<h4><a name="Migration"></a>Migration</h4>
+<p>Existing calls to <tt>PROFILE_GET</tt> will need to change from <tt>PROFILE_GET('profile', 'entity', duration, 'durationUnits')</tt> to <tt>PROFILE_GET('profile', 'entity', PROFILE_FIXED(duration, 'durationUnits'))</tt></p></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
[02/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-platform/metron-management/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-management/index.html b/site/current-book/metron-platform/metron-management/index.html
new file mode 100644
index 0000000..c8d1d13
--- /dev/null
+++ b/site/current-book/metron-platform/metron-management/index.html
@@ -0,0 +1,1602 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Stellar REPL Management Utilities</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Stellar REPL Management Utilities</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-down"></i>
+ Platform</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-platform/metron-api/index.html" title="Api">
+ <i class="none"></i>
+ Api</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-common/index.html" title="Common">
+ <i class="none"></i>
+ Common</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
+ <i class="none"></i>
+ Data-management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
+ <i class="none"></i>
+ Enrichment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
+ <i class="none"></i>
+ Indexing</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
+ <i class="none"></i>
+ Parsers</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
+ <i class="none"></i>
+ Pcap-backend</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Stellar REPL Management Utilities</h1>
+<p><a name="Stellar_REPL_Management_Utilities"></a></p>
+<p>In order to augment the functionality of the Stellar REPL, a few management functions surrounding the management of the configurations and the management of Stellar transformations in the following areas have been added:</p>
+
+<ul>
+
+<li>Stellar field transformations in the Parsers</li>
+
+<li>Stellar enrichments in the Enrichment topology</li>
+
+<li>Stellar threat triage rules</li>
+</ul>
+<p>Additionally, some shell functions have been added to </p>
+
+<ul>
+
+<li>provide the ability to refer to the Stellar expression used to create a variable</li>
+
+<li>print structured data in a way that is easier to view (i.e. tabular)</li>
+</ul>
+<p>This functionality is exposed as a pack of Stellar functions in this project.</p>
+<div class="section">
+<h2><a name="Function_Details"></a>Function Details</h2>
+<p>The functions are split roughly into a few sections:</p>
+
+<ul>
+
+<li>Shell functions - Functions surrounding interacting with the shell in either a nicer way or a more functional way.</li>
+
+<li>Grok Functions - Functions that allow you to evaluate grok expressions.</li>
+
+<li>File functions - Functions around interacting with local or HDFS files</li>
+
+<li>Configuration functions - Functions surrounding pulling and pushing configs from zookeeper</li>
+
+<li>Parser functions - Functions surrounding adding, viewing, and removing Parser functions.</li>
+
+<li>Enrichment functions - Functions surrounding adding, viewing and removing Stellar enrichments as well as managing batch size and index names for the enrichment topology configuration</li>
+
+<li>Threat Triage functions - Functions surrounding adding, viewing and removing threat triage functions.</li>
+</ul>
+<div class="section">
+<h3><a name="Grok_Functions"></a>Grok Functions</h3>
+
+<ul>
+
+<li><tt>GROK_EVAL</tt>
+
+<ul>
+
+<li>Description: Evaluate a grok expression for a statement.</li>
+
+<li>Input:
+
+<ul>
+
+<li>grokExpression - The grok expression to evaluate</li>
+
+<li>data - Either a data message or a list of data messages to evaluate using the grokExpression</li>
+ </ul></li>
+
+<li>Returns: The Map associated with the grok expression being evaluated on the list of messages.</li>
+ </ul></li>
+
+<li><tt>GROK_PREDICT</tt>
+
+<ul>
+
+<li>Description: Discover a grok statement for an input doc</li>
+
+<li>Input:
+
+<ul>
+
+<li>data - The data to discover a grok expression from</li>
+ </ul></li>
+
+<li>Returns: A grok expression that should match the data.</li>
+ </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="File_Functions"></a>File Functions</h3>
+
+<ul>
+
+<li>Local Files
+
+<ul>
+
+<li><tt>LOCAL_LS</tt>
+
+<ul>
+
+<li>Description: Lists the contents of a directory.</li>
+
+<li>Input:
+
+<ul>
+
+<li>path - The path of the file</li>
+ </ul></li>
+
+<li>Returns: The contents of the directory in tabular form sorted by last modification date.</li>
+ </ul></li>
+
+<li><tt>LOCAL_RM</tt>
+
+<ul>
+
+<li>Description: Removes the path</li>
+
+<li>Input:
+
+<ul>
+
+<li>path - The path of the file or directory.</li>
+
+<li>recursive - Recursively remove or not (optional and defaulted to false)</li>
+ </ul></li>
+
+<li>Returns: boolean - true if successful, false otherwise</li>
+ </ul></li>
+
+<li><tt>LOCAL_READ</tt>
+
+<ul>
+
+<li>Description: Retrieves the contents as a string of a file.</li>
+
+<li>Input:
+
+<ul>
+
+<li>path - The path of the file</li>
+ </ul></li>
+
+<li>Returns: The contents of the file and null otherwise.</li>
+ </ul></li>
+
+<li><tt>LOCAL_READ_LINES</tt>
+
+<ul>
+
+<li>Description: Retrieves the contents of a file as a list of strings.</li>
+
+<li>Input:
+
+<ul>
+
+<li>path - The path of the file</li>
+ </ul></li>
+
+<li>Returns: A list of lines</li>
+ </ul></li>
+
+<li><tt>LOCAL_WRITE</tt>
+
+<ul>
+
+<li>Description: Writes the contents of a string to a local file</li>
+
+<li>Input:
+
+<ul>
+
+<li>content - The content to write out</li>
+
+<li>path - The path of the file</li>
+ </ul></li>
+
+<li>Returns: true if the file was written and false otherwise.</li>
+ </ul></li>
+ </ul></li>
+
+<li>HDFS Files
+
+<ul>
+
+<li><tt>HDFS_LS</tt>
+
+<ul>
+
+<li>Description: Lists the contents of a directory in HDFS.</li>
+
+<li>Input:
+
+<ul>
+
+<li>path - The path of the file</li>
+ </ul></li>
+
+<li>Returns: The contents of the directory in tabular form sorted by last modification date.</li>
+ </ul></li>
+
+<li><tt>HDFS_RM</tt>
+
+<ul>
+
+<li>Description: Removes the path in HDFS.</li>
+
+<li>Input:
+
+<ul>
+
+<li>path - The path of the file or directory.</li>
+
+<li>recursive - Recursively remove or not (optional and defaulted to false)</li>
+ </ul></li>
+
+<li>Returns: boolean - true if successful, false otherwise</li>
+ </ul></li>
+
+<li><tt>HDFS_READ</tt>
+
+<ul>
+
+<li>Description: Retrieves the contents as a string of a file in HDFS.</li>
+
+<li>Input:
+
+<ul>
+
+<li>path - The path of the file</li>
+ </ul></li>
+
+<li>Returns: The contents of the file and null otherwise.</li>
+ </ul></li>
+
+<li><tt>HDFS_READ_LINES</tt>
+
+<ul>
+
+<li>Description: Retrieves the contents of a HDFS file as a list of strings.</li>
+
+<li>Input:
+
+<ul>
+
+<li>path - The path of the file</li>
+ </ul></li>
+
+<li>Returns: A list of lines</li>
+ </ul></li>
+
+<li><tt>HDFS_WRITE</tt>
+
+<ul>
+
+<li>Description: Writes the contents of a string to a HDFS file</li>
+
+<li>Input:
+
+<ul>
+
+<li>content - The content to write out</li>
+
+<li>path - The path of the file</li>
+ </ul></li>
+
+<li>Returns: true if the file was written and false otherwise.</li>
+ </ul></li>
+ </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="Shell_Functions"></a>Shell Functions</h3>
+
+<ul>
+
+<li><tt>SHELL_EDIT</tt>
+
+<ul>
+
+<li>Description: Open an editor (optionally initialized with text) and return whatever is saved from the editor. The editor to use is pulled from <tt>EDITOR</tt> or <tt>VISUAL</tt> environment variable.</li>
+
+<li>Input:
+
+<ul>
+
+<li>string - (Optional) A string whose content is used to initialize the editor.</li>
+ </ul></li>
+
+<li>Returns: The content that the editor saved after editor exit.</li>
+ </ul></li>
+
+<li><tt>SHELL_GET_EXPRESSION</tt>
+
+<ul>
+
+<li>Description: Get a stellar expression from a variable</li>
+
+<li>Input:
+
+<ul>
+
+<li>variable - variable name</li>
+ </ul></li>
+
+<li>Returns: The stellar expression associated with the variable.</li>
+ </ul></li>
+
+<li><tt>SHELL_LIST_VARS</tt>
+
+<ul>
+
+<li>Description: Return the variables in a tabular form</li>
+
+<li>Input:
+
+<ul>
+
+<li>wrap : Length of string to wrap the columns</li>
+ </ul></li>
+
+<li>Returns: A tabular representation of the variables.</li>
+ </ul></li>
+
+<li><tt>SHELL_MAP2TABLE</tt>
+
+<ul>
+
+<li>Description: Take a map and return a table</li>
+
+<li>Input:
+
+<ul>
+
+<li>map - Map</li>
+ </ul></li>
+
+<li>Returns: The map in table form</li>
+ </ul></li>
+
+<li><tt>SHELL_VARS2MAP</tt>
+
+<ul>
+
+<li>Description: Take a set of variables and return a map</li>
+
+<li>Input:
+
+<ul>
+
+<li>variables* - variable names to use to create map</li>
+ </ul></li>
+
+<li>Returns: A map associating the variable name with the stellar expression.</li>
+ </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="Configuration_Functions"></a>Configuration Functions</h3>
+
+<ul>
+
+<li><tt>CONFIG_GET</tt>
+
+<ul>
+
+<li>Description: Retrieve a Metron configuration from zookeeper.</li>
+
+<li>Input:
+
+<ul>
+
+<li>type - One of ENRICHMENT, INDEXING, PARSER, GLOBAL, PROFILER</li>
+
+<li>sensor - Sensor to retrieve (required for enrichment and parser, not used for profiler and global)</li>
+
+<li>emptyIfNotPresent - If true, then return an empty, minimally viable config</li>
+ </ul></li>
+
+<li>Returns: The String representation of the config in zookeeper</li>
+ </ul></li>
+
+<li><tt>CONFIG_PUT</tt>
+
+<ul>
+
+<li>Description: Updates a Metron config to Zookeeper.</li>
+
+<li>Input:
+
+<ul>
+
+<li>type - One of ENRICHMENT, INDEXING, PARSER, GLOBAL, PROFILER</li>
+
+<li>config - The config (a string in JSON form) to update</li>
+
+<li>sensor - Sensor to retrieve (required for enrichment and parser, not used for profiler and global)</li>
+ </ul></li>
+
+<li>Returns: The String representation of the config in zookeeper</li>
+ </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="Parser_Functions"></a>Parser Functions</h3>
+
+<ul>
+
+<li><tt>PARSER_STELLAR_TRANSFORM_ADD</tt>
+
+<ul>
+
+<li>Description: Add stellar field transformation.</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>stellarTransforms - A Map associating fields to stellar expressions</li>
+ </ul></li>
+
+<li>Returns: The String representation of the config in zookeeper</li>
+ </ul></li>
+
+<li><tt>PARSER_STELLAR_TRANSFORM_PRINT</tt>
+
+<ul>
+
+<li>Description: Retrieve stellar field transformations.</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+ </ul></li>
+
+<li>Returns: The String representation of the transformations</li>
+ </ul></li>
+
+<li><tt>PARSER_STELLAR_TRANSFORM_REMOVE</tt>
+
+<ul>
+
+<li>Description: Remove stellar field transformation.</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>stellarTransforms - A list of stellar transforms to remove</li>
+ </ul></li>
+
+<li>Returns: The String representation of the config in zookeeper</li>
+ </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="Indexing_Functions"></a>Indexing Functions</h3>
+
+<ul>
+
+<li><tt>INDEXING_SET_BATCH</tt>
+
+<ul>
+
+<li>Description: Set batch size</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>writer - The writer to update (e.g. elasticsearch, solr or hdfs)</li>
+
+<li>size - batch size (integer)</li>
+ </ul></li>
+
+<li>Returns: The String representation of the config in zookeeper</li>
+ </ul></li>
+
+<li><tt>INDEXING_SET_ENABLED</tt>
+
+<ul>
+
+<li>Description: Enable or disable an indexing writer for a sensor.</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>writer - The writer to update (e.g. elasticsearch, solr or hdfs)</li>
+
+<li>enabled? - boolean indicating whether the writer is enabled. If omitted, then it will set enabled.</li>
+ </ul></li>
+
+<li>Returns: The String representation of the config in zookeeper</li>
+ </ul></li>
+
+<li><tt>INDEXING_SET_INDEX</tt>
+
+<ul>
+
+<li>Description: Set the index for the sensor</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>writer - The writer to update (e.g. elasticsearch, solr or hdfs)</li>
+
+<li>sensor - sensor name</li>
+ </ul></li>
+
+<li>Returns: The String representation of the config in zookeeper</li>
+ </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="Enrichment_Functions"></a>Enrichment Functions</h3>
+
+<ul>
+
+<li><tt>ENRICHMENT_STELLAR_TRANSFORM_ADD</tt>
+
+<ul>
+
+<li>Description: Add stellar field transformation.</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>type - ENRICHMENT or THREAT_INTEL</li>
+
+<li>stellarTransforms - A Map associating fields to stellar expressions</li>
+
+<li>group - Group to add to (optional)</li>
+ </ul></li>
+
+<li>Returns: The String representation of the config in zookeeper</li>
+ </ul></li>
+
+<li><tt>ENRICHMENT_STELLAR_TRANSFORM_PRINT</tt>
+
+<ul>
+
+<li>Description: Retrieve stellar enrichment transformations.</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>type - ENRICHMENT or THREAT_INTEL</li>
+ </ul></li>
+
+<li>Returns: The String representation of the transformations</li>
+ </ul></li>
+
+<li><tt>ENRICHMENT_STELLAR_TRANSFORM_REMOVE</tt>
+
+<ul>
+
+<li>Description: Remove one or more stellar field transformations.</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>type - ENRICHMENT or THREAT_INTEL</li>
+
+<li>stellarTransforms - A list of removals</li>
+
+<li>group - Group to remove from (optional)</li>
+ </ul></li>
+
+<li>Returns: The String representation of the config in zookeeper</li>
+ </ul></li>
+</ul></div>
+<div class="section">
+<h3><a name="Threat_Triage_Functions"></a>Threat Triage Functions</h3>
+
+<ul>
+
+<li><tt>THREAT_TRIAGE_ADD</tt>
+
+<ul>
+
+<li>Description: Add a threat triage rule.</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>stellarTransforms - A Map associating stellar rules to scores</li>
+
+<li>triageRules - Map (or list of Maps) representing a triage rule. It must contain ‘rule’ and ‘score’ keys, the stellar expression for the rule and triage score respectively. It may contain ‘name’ and ‘comment’, the name of the rule and comment associated with the rule respectively."</li>
+ </ul></li>
+
+<li>Returns: The String representation of the threat triage rules</li>
+ </ul></li>
+
+<li><tt>THREAT_TRIAGE_PRINT</tt>
+
+<ul>
+
+<li>Description: Retrieve stellar enrichment transformations.</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+ </ul></li>
+
+<li>Returns: The String representation of the threat triage rules</li>
+ </ul></li>
+
+<li><tt>THREAT_TRIAGE_REMOVE</tt>
+
+<ul>
+
+<li>Description: Remove stellar threat triage rule(s).</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>rules - A list of stellar rules or rule names to remove</li>
+ </ul></li>
+
+<li>Returns: The String representation of the enrichment config</li>
+ </ul></li>
+
+<li><tt>THREAT_TRIAGE_SET_AGGREGATOR</tt>
+
+<ul>
+
+<li>Description: Set the threat triage aggregator.</li>
+
+<li>Input:
+
+<ul>
+
+<li>sensorConfig - Sensor config to add transformation to.</li>
+
+<li>aggregator - Aggregator to use. One of MIN, MAX, MEAN, SUM, POSITIVE_MEAN</li>
+
+<li>aggregatorConfig - Optional config for aggregator</li>
+ </ul></li>
+
+<li>Returns: The String representation of the enrichment config</li>
+ </ul></li>
+</ul></div></div>
+<div class="section">
+<h2><a name="Deployment_Instructions"></a>Deployment Instructions</h2>
+<p>Deployment is as simple as dropping the jar created by this project into <tt>$METRON_HOME/lib</tt> and starting the Stellar shell via <tt>$METRON_HOME/bin/stellar</tt></p></div>
+<div class="section">
+<h2><a name="Examples"></a>Examples</h2>
+<p>Included for description and education purposes are a couple example Stellar REPL transcripts with helpful comments to illustrate some common operations.</p>
+<div class="section">
+<h3><a name="Iterate_in_finding_a_valid_Grok_pattern"></a>Iterate in finding a valid Grok pattern</h3>
+
+<div class="source">
+<div class="source">
+<pre>Stellar, Go!
+Please note that functions are loading lazily in the background and will be unavailable until loaded fully.
+[Stellar]>>> # We are going to debug a squid grok statement with a bug in it
+[Stellar]>>> squid_grok_orig := '%{NUMBER:timestamp} %{SPACE:UNWANTED} %{INT:elapsed} %{IP:ip_src_addr} %{WORD:action}/%{NUMBER:code} %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url}
+ - %{WORD:UNWANTED}/%{IP:ip_dst_addr} %{WORD:UNWANTED}/%{WORD:UNWANTED}'
+[Stellar]>>> # We have gone ot a couple of domains in squid:
+[Stellar]>>> # 1475022887.362 256 127.0.0.1 TCP_MISS/301 803 GET http://www.youtube.com/ - DIRECT/216.58.216.238 text/html
+[Stellar]>>> # 1475022915.731 1 127.0.0.1 NONE/400 3520 GET flimflammadeupdomain.com - NONE/- text/html
+[Stellar]>>> # 1475022938.661 0 127.0.0.1 NONE/400 3500 GET www.google.com - NONE/- text/html
+[Stellar]>>> # Note that flimflammadeupdomain.com and www.google.com did not resolve to IPs
+[Stellar]>>> # We can load up these messages from disk into a list of messages
+[Stellar]>>> messages := LOCAL_READ_LINES( '/var/log/squid/access.log')
+27687 [Thread-1] INFO o.r.Reflections - Reflections took 26542 ms to scan 22 urls, producing 17906 keys and 121560 values
+27837 [Thread-1] INFO o.a.m.c.d.FunctionResolverSingleton - Found 97 Stellar Functions...
+Functions loaded, you may refer to functions now...
+[Stellar]>>> # and evaluate the messages against our grok statement
+[Stellar]>>> GROK_EVAL(squid_grok_orig, messages)
+╔══════════╤═════════╤═════════╤═════════╤════════════════╤═════════════╤═════════╤════════════════╤════════════════════════%
50;╗
+║ action │ bytes │ code │ elapsed │ ip_dst_addr │ ip_src_addr │ method │ timestamp │ url ║
+╠══════════╪═════════╪═════════╪═════════╪════════════════╪═════════════╪═════════╪════════════════╪════════════════════════%
50;╣
+║ TCP_MISS │ 803 │ 301 │ 256 │ 216.58.216.238 │ 127.0.0.1 │ GET │ 1475022887.362 │ http://www.youtube.com/ ║
+╟──────────┼─────────┼─────────┼─────────┼────────────────┼─────────────┼─────────┼────────────────┼────────────────────────%
00;╢
+║ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING ║
+╟──────────┼─────────┼─────────┼─────────┼────────────────┼─────────────┼─────────┼────────────────┼────────────────────────%
00;╢
+║ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING ║
+╚══════════╧═════════╧═════════╧═════════╧════════════════╧═════════════╧═════════╧════════════════╧════════════════════════%
50;╝
+
+[Stellar]>>> # Uh oh, looks like the messages without destination IPs do not parse
+[Stellar]>>> # We can start peeling off groups from the end of the message until things parse
+[Stellar]>>> squid_grok := '%{NUMBER:timestamp} %{SPACE:UNWANTED} %{INT:elapsed} %{IP:ip_src_addr} %{WORD:action}/%{NUMBER:code} %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} - %{
+WORD:UNWANTED}/%{IP:ip_dst_addr}'
+[Stellar]>>> GROK_EVAL(squid_grok, messages)
+╔══════════╤═════════╤═════════╤═════════╤════════════════╤═════════════╤═════════╤════════════════╤════════════════════════%
50;╗
+║ action │ bytes │ code │ elapsed │ ip_dst_addr │ ip_src_addr │ method │ timestamp │ url ║
+╠══════════╪═════════╪═════════╪═════════╪════════════════╪═════════════╪═════════╪════════════════╪════════════════════════%
50;╣
+║ TCP_MISS │ 803 │ 301 │ 256 │ 216.58.216.238 │ 127.0.0.1 │ GET │ 1475022887.362 │ http://www.youtube.com/ ║
+╟──────────┼─────────┼─────────┼─────────┼────────────────┼─────────────┼─────────┼────────────────┼────────────────────────%
00;╢
+║ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING ║
+╟──────────┼─────────┼─────────┼─────────┼────────────────┼─────────────┼─────────┼────────────────┼────────────────────────%
00;╢
+║ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING ║
+╚══════════╧═════════╧═════════╧═════════╧════════════════╧═════════════╧═════════╧════════════════╧════════════════════════%
50;╝
+
+[Stellar]>>> # Still looks like it is having issues...
+[Stellar]>>> squid_grok := '%{NUMBER:timestamp} %{SPACE:UNWANTED} %{INT:elapsed} %{IP:ip_src_addr} %{WORD:action}/%{NUMBER:code} %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} - %{
+WORD:UNWANTED}/%{IP:ip_dst_addr}'
+[Stellar]>>> GROK_EVAL(squid_grok, messages)
+╔══════════╤═════════╤═════════╤═════════╤════════════════╤═════════════╤═════════╤════════════════╤════════════════════════%
50;╗
+║ action │ bytes │ code │ elapsed │ ip_dst_addr │ ip_src_addr │ method │ timestamp │ url ║
+╠══════════╪═════════╪═════════╪═════════╪════════════════╪═════════════╪═════════╪════════════════╪════════════════════════%
50;╣
+║ TCP_MISS │ 803 │ 301 │ 256 │ 216.58.216.238 │ 127.0.0.1 │ GET │ 1475022887.362 │ http://www.youtube.com/ ║
+╟──────────┼─────────┼─────────┼─────────┼────────────────┼─────────────┼─────────┼────────────────┼────────────────────────%
00;╢
+║ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING ║
+╟──────────┼─────────┼─────────┼─────────┼────────────────┼─────────────┼─────────┼────────────────┼────────────────────────%
00;╢
+║ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING │ MISSING ║
+╚══════════╧═════════╧═════════╧═════════╧════════════════╧═════════════╧═════════╧════════════════╧════════════════════════%
50;╝
+
+[Stellar]>>> # Still looks wrong. Hmm, I bet it is due to that dst_addr not being there; we can make it optional
+[Stellar]>>> squid_grok := '%{NUMBER:timestamp} %{SPACE:UNWANTED} %{INT:elapsed} %{IP:ip_src_addr} %{WORD:action}/%{NUMBER:code} %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} - %{
+WORD:UNWANTED}/(%{IP:ip_dst_addr})?'
+[Stellar]>>> GROK_EVAL(squid_grok, messages)
+╔══════════╤═══════╤══════╤═════════╤════════════════╤═════════════╤════════╤════════════════╤══════════════════════════╗
+║ action │ bytes │ code │ elapsed │ ip_dst_addr │ ip_src_addr │ method │ timestamp │ url ║
+╠══════════╪═══════╪══════╪═════════╪════════════════╪═════════════╪════════╪════════════════╪══════════════════════════╣
+║ TCP_MISS │ 803 │ 301 │ 256 │ 216.58.216.238 │ 127.0.0.1 │ GET │ 1475022887.362 │ http://www.youtube.com/ ║
+╟──────────┼───────┼──────┼─────────┼────────────────┼─────────────┼────────┼────────────────┼──────────────────────────╢
+║ NONE │ 3520 │ 400 │ 1 │ null │ 127.0.0.1 │ GET │ 1475022915.731 │ flimflammadeupdomain.com ║
+╟──────────┼───────┼──────┼─────────┼────────────────┼─────────────┼────────┼────────────────┼──────────────────────────╢
+║ NONE │ 3500 │ 400 │ 0 │ null │ 127.0.0.1 │ GET │ 1475022938.661 │ www.google.com ║
+╚══════════╧═══════╧══════╧═════════╧════════════════╧═════════════╧════════╧════════════════╧══════════════════════════╝
+
+[Stellar]>>> # Ahh, that is much better.
+[Stellar]>>>
+[Stellar]>>>
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Manage_Stellar_Field_Transformations"></a>Manage Stellar Field Transformations</h3>
+
+<div class="source">
+<div class="source">
+<pre>964 [main] INFO o.a.c.f.i.CuratorFrameworkImpl - Starting
+1025 [main-EventThread] INFO o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
+Stellar, Go!
+Please note that functions are loading lazily in the background and will be unavailable until loaded fully.
+{es.clustername=metron, es.ip=node1, es.port=9300, es.date.format=yyyy.MM.dd.HH}
+[Stellar]>>> # First we get the squid parser config from zookeeper
+[Stellar]>>> squid_parser_config := CONFIG_GET('PARSER', 'squid')
+29089 [Thread-1] INFO o.r.Reflections - Reflections took 26765 ms to scan 22 urls, producing 17898 keys and 121518 values
+29177 [Thread-1] INFO o.a.m.c.d.FunctionResolverSingleton - Found 83 Stellar Functions...
+Functions loaded, you may refer to functions now...
+[Stellar]>>> # See what kind of transformations it already has
+[Stellar]>>> PARSER_STELLAR_TRANSFORM_PRINT(squid_parser_config)
+╔═══════════════════════════╤═════════════════════════════════════════╗
+║ Field │ Transformation ║
+╠═══════════════════════════╪═════════════════════════════════════════╣
+║ full_hostname │ URL_TO_HOST(url) ║
+╟───────────────────────────┼─────────────────────────────────────────╢
+║ domain_without_subdomains │ DOMAIN_REMOVE_SUBDOMAINS(full_hostname) ║
+╚═══════════════════════════╧═════════════════════════════════════════╝
+
+[Stellar]>>> #Just to make sure it looks right, we can view the JSON
+[Stellar]>>> squid_parser_config
+{
+ "parserClassName": "org.apache.metron.parsers.GrokParser",
+ "sensorTopic": "squid",
+ "parserConfig": {
+ "grokPath": "/patterns/squid",
+ "patternLabel": "SQUID_DELIMITED",
+ "timestampField": "timestamp"
+ },
+ "fieldTransformations" : [
+ {
+ "transformation" : "STELLAR"
+ ,"output" : [ "full_hostname", "domain_without_subdomains" ]
+ ,"config" : {
+ "full_hostname" : "URL_TO_HOST(url)"
+ ,"domain_without_subdomains" : "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)"
+ }
+ }
+ ]
+}
+
+[Stellar]>>> # Add another transformation in there
+[Stellar]>>> domain_without_subdomains := 'cnn.com'
+[Stellar]>>> upper_domain := TO_UPPER(domain_without_subdomains)
+[Stellar]>>> # Now we can look at our variables and see what expressions created them
+[Stellar]>>> # NOTE: the 40 is the max char for a word
+[Stellar]>>> SHELL_LIST_VARS( 40 )
+╔═══════════════════════════╤═══════════════════════════════════════════╤═════════════════════════════════════╗
+║ VARIABLE │ VALUE │ EXPRESSION ║
+╠═══════════════════════════╪═══════════════════════════════════════════╪═════════════════════════════════════╣
+║ squid_parser_config │ { │ CONFIG_GET('PARSER', 'squid') ║
+║ │ "parserClassName": │ ║
+║ │ "org.apache.metron.parsers.GrokParser", │ ║
+║ │ │ ║
+║ │ "sensorTopic": "squid", │ ║
+║ │ │ ║
+║ │ "parserConfig": { │ ║
+║ │ "grokPath": │ ║
+║ │ "/patterns/squid", │ ║
+║ │ "patternLabel": │ ║
+║ │ "SQUID_DELIMITED", │ ║
+║ │ "timestampField": │ ║
+║ │ "timestamp" │ ║
+║ │ }, │ ║
+║ │ │ ║
+║ │ "fieldTransformations" : [ │ ║
+║ │ { │ ║
+║ │ │ ║
+║ │ "transformation" : "STELLAR" │ ║
+║ │ │ ║
+║ │ ,"output" : [ "full_hostname", │ ║
+║ │ "domain_without_subdomains" ] │ ║
+║ │ │ ║
+║ │ ,"config" : { │ ║
+║ │ "full_hostname" : │ ║
+║ │ "URL_TO_HOST(url)" │ ║
+║ │ │ ║
+║ │ ,"domain_without_subdomains" : │ ║
+║ │ "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)" │ ║
+║ │ │ ║
+║ │ } │ ║
+║ │ } │ ║
+║ │ ] │ ║
+║ │ } │ ║
+╟───────────────────────────┼───────────────────────────────────────────┼─────────────────────────────────────╢
+║ domain_without_subdomains │ cnn.com │ 'cnn.com' ║
+╟───────────────────────────┼───────────────────────────────────────────┼─────────────────────────────────────╢
+║ upper_domain │ CNN.COM │ TO_UPPER(domain_without_subdomains) ║
+╚═══════════════════════════╧═══════════════════════════════════════════╧═════════════════════════════════════╝
+
+[Stellar]>>> # We can add upper_domain as a transformation to the parser now by
+[Stellar]>>> ?PARSER_STELLAR_TRANSFORM_ADD
+PARSER_STELLAR_TRANSFORM_ADD
+Description: Add stellar field transformation.
+
+Arguments:
+ sensorConfig - Sensor config to add transformation to.
+ stellarTransforms - A Map associating fields to stellar expressions
+
+Returns: The String representation of the config in zookeeper
+[Stellar]>>> # We will use the SHELL_VARS2MAP to construct our map associating the field name with the expression
+[Stellar]>>> ?SHELL_VARS2MAP
+SHELL_VARS2MAP
+Description: Take a set of variables and return a map
+
+Arguments:
+ variables* - variable names to use to create map
+
+Returns: A map associating the variable name with the stellar expression.
+[Stellar]>>> squid_parser_config_new := PARSER_STELLAR_TRANSFORM_ADD( squid_parser_config, SHELL_VARS2MAP('upper_domain') )
+[Stellar]>>> #Now we can make sure that we have the transformation added
+[Stellar]>>> PARSER_STELLAR_TRANSFORM_PRINT(squid_parser_config_new)
+╔═══════════════════════════╤═════════════════════════════════════════╗
+║ Field │ Transformation ║
+╠═══════════════════════════╪═════════════════════════════════════════╣
+║ full_hostname │ URL_TO_HOST(url) ║
+╟───────────────────────────┼─────────────────────────────────────────╢
+║ domain_without_subdomains │ DOMAIN_REMOVE_SUBDOMAINS(full_hostname) ║
+╟───────────────────────────┼─────────────────────────────────────────╢
+║ upper_domain │ TO_UPPER(domain_without_subdomains) ║
+╚═══════════════════════════╧═════════════════════════════════════════╝
+
+[Stellar]>>> #And finally, we push the configs back to zookeeper
+[Stellar]>>> CONFIG_PUT('PARSER', squid_parser_config_new, 'squid')
+[Stellar]>>> #Now we can make sure that we have the transformation added into zookeeper
+[Stellar]>>> PARSER_STELLAR_TRANSFORM_PRINT(CONFIG_GET('PARSER', 'squid'))
+╔═══════════════════════════╤═════════════════════════════════════════╗
+║ Field │ Transformation ║
+╠═══════════════════════════╪═════════════════════════════════════════╣
+║ full_hostname │ URL_TO_HOST(url) ║
+╟───────────────────────────┼─────────────────────────────────────────╢
+║ domain_without_subdomains │ DOMAIN_REMOVE_SUBDOMAINS(full_hostname) ║
+╟───────────────────────────┼─────────────────────────────────────────╢
+║ upper_domain │ TO_UPPER(domain_without_subdomains) ║
+╚═══════════════════════════╧═════════════════════════════════════════╝
+
+[Stellar]>>> #Now that we have added it, we can change our mind and remove it
+[Stellar]>>> ?PARSER_STELLAR_TRANSFORM_REMOVE
+PARSER_STELLAR_TRANSFORM_REMOVE
+Description: Remove stellar field transformation.
+
+Arguments:
+ sensorConfig - Sensor config to add transformation to.
+ stellarTransforms - A list of stellar transforms to remove
+
+Returns: The String representation of the config in zookeeper
+[Stellar]>>> squid_parser_config_new := PARSER_STELLAR_TRANSFORM_REMOVE( CONFIG_GET('PARSER', 'squid'), [ 'upper_domain' ] )
+[Stellar]>>> PARSER_STELLAR_TRANSFORM_PRINT(squid_parser_config_new)
+╔═══════════════════════════╤═════════════════════════════════════════╗
+║ Field │ Transformation ║
+╠═══════════════════════════╪═════════════════════════════════════════╣
+║ full_hostname │ URL_TO_HOST(url) ║
+╟───────────────────────────┼─────────────────────────────────────────╢
+║ domain_without_subdomains │ DOMAIN_REMOVE_SUBDOMAINS(full_hostname) ║
+╚═══════════════════════════╧═════════════════════════════════════════╝
+
+[Stellar]>>> #We can now push up the config to zookeeper
+[Stellar]>>> CONFIG_PUT('PARSER', squid_parser_config_new, 'squid')
+[Stellar]>>> #It should be just as we started the exercise
+[Stellar]>>> CONFIG_GET('PARSER', 'squid')
+{
+ "parserClassName" : "org.apache.metron.parsers.GrokParser",
+ "sensorTopic" : "squid",
+ "parserConfig" : {
+ "grokPath" : "/patterns/squid",
+ "patternLabel" : "SQUID_DELIMITED",
+ "timestampField" : "timestamp"
+ },
+ "fieldTransformations" : [ {
+ "input" : [ ],
+ "output" : [ "full_hostname", "domain_without_subdomains" ],
+ "transformation" : "STELLAR",
+ "config" : {
+ "full_hostname" : "URL_TO_HOST(url)",
+ "domain_without_subdomains" : "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)"
+ }
+ } ]
+}
+[Stellar]>>> #And quit the REPL
+[Stellar]>>> quit
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Manage_Stellar_Enrichments"></a>Manage Stellar Enrichments</h3>
+
+<div class="source">
+<div class="source">
+<pre>1010 [main] INFO o.a.c.f.i.CuratorFrameworkImpl - Starting
+1077 [main-EventThread] INFO o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
+Stellar, Go!
+Please note that functions are loading lazily in the background and will be unavailable until loaded fully.
+{es.clustername=metron, es.ip=node1, es.port=9300, es.date.format=yyyy.MM.dd.HH}
+[Stellar]>>> # First we get the squid enrichment config from zookeeper.
+[Stellar]>>> # If it is not there, which it is not by default, a suitable default
+[Stellar]>>> # config will be specified.
+[Stellar]>>> squid_enrichment_config := CONFIG_GET('ENRICHMENT', 'squid')
+26307 [Thread-1] INFO o.r.Reflections - Reflections took 24845 ms to scan 22 urls, producing 17898 keys and 121520 values
+26389 [Thread-1] INFO o.a.m.c.d.FunctionResolverSingleton - Found 84 Stellar Functions...
+Functions loaded, you may refer to functions now...
+[Stellar]>>> # Just to make sure it looks right, we can view the JSON
+[Stellar]>>> squid_enrichment_config
+{
+ "enrichment" : {
+ "fieldMap" : { },
+ "fieldToTypeMap" : { },
+ "config" : { }
+ },
+ "threatIntel" : {
+ "fieldMap" : { },
+ "fieldToTypeMap" : { },
+ "config" : { },
+ "triageConfig" : {
+ "riskLevelRules" : [ ],
+ "aggregator" : "MAX",
+ "aggregationConfig" : { }
+ }
+ },
+ "configuration" : { }
+}
+[Stellar]>>> # Now that we have a config, we can add an enrichment to the Stellar adapter
+[Stellar]>>> # We should make sure that the current enrichment does not have any already
+[Stellar]>>> ?ENRICHMENT_STELLAR_TRANSFORM_PRINT
+ENRICHMENT_STELLAR_TRANSFORM_PRINT
+Description: Retrieve stellar enrichment transformations.
+
+Arguments:
+ sensorConfig - Sensor config to add transformation to.
+ type - ENRICHMENT or THREAT_INTEL
+
+Returns: The String representation of the transformations
+[Stellar]>>> # Since there are two places we can add enrichments, we should check both
+[Stellar]>>> ENRICHMENT_STELLAR_TRANSFORM_PRINT(squid_enrichment_config, 'ENRICHMENT')
+╔═══════╤═══════╤════════════════╗
+║ Group │ Field │ Transformation ║
+╠═══════╧═══════╧════════════════╣
+║ (empty) ║
+╚════════════════════════════════╝
+
+[Stellar]>>> ENRICHMENT_STELLAR_TRANSFORM_PRINT(squid_enrichment_config, 'THREAT_INTEL')
+╔═══════╤═══════╤════════════════╗
+║ Group │ Field │ Transformation ║
+╠═══════╧═══════╧════════════════╣
+║ (empty) ║
+╚════════════════════════════════╝
+
+[Stellar]>>> # For my enrichment, I want to add a field indicating that the src address is local or not
+[Stellar]>>> # I will define local as part of '192.168.0.0/21'
+[Stellar]>>> # We must be careful about the ip_src_addr, what if it is malformed?
+[Stellar]>>> ip_src_addr := '200.20.10.1'
+[Stellar]>>> IN_SUBNET( ip_src_addr, '192.168.0.0/21')
+false
+[Stellar]>>> # Just as we expected. Now we can try a local address
+[Stellar]>>> ip_src_addr := '192.168.0.1'
+[Stellar]>>> IN_SUBNET( ip_src_addr, '192.168.0.0/21')
+true
+[Stellar]>>> # Just as we expected. Now we can try some malformed ones
+[Stellar]>>> ip_src_addr := NULL
+[Stellar]>>> IN_SUBNET( ip_src_addr, '192.168.0.0/21')
+false
+[Stellar]>>> # So far, so good
+[Stellar]>>> ip_src_addr := 'foo bar'
+[Stellar]>>> IN_SUBNET( ip_src_addr, '192.168.0.0/21')
+[!] Unable to execute: Could not parse [foo bar]
+[Stellar]>>> # uh oh, that was terrible, we will have to adjust and be a bit more defensive
+[Stellar]>>> IN_SUBNET( if IS_IP(ip_src_addr) then ip_src_addr else NULL, '192.168.0.0/21')
+false
+[Stellar]>>> ip_src_addr := NULL
+[Stellar]>>> IN_SUBNET( if IS_IP(ip_src_addr) then ip_src_addr else NULL, '192.168.0.0/21')
+false
+[Stellar]>>> ip_src_addr := '192.168.0.1'
+[Stellar]>>> IN_SUBNET( if IS_IP(ip_src_addr) then ip_src_addr else NULL, '192.168.0.0/21')
+true
+[Stellar]>>> ip_src_addr := '200.20.10.1'
+[Stellar]>>> IN_SUBNET( if IS_IP(ip_src_addr) then ip_src_addr else NULL, '192.168.0.0/21')
+false
+[Stellar]>>> # I think we are ready, we will call it is_local
+[Stellar]>>> is_local := IN_SUBNET( if IS_IP(ip_src_addr) then ip_src_addr else NULL, '192.168.0.0/21')
+[Stellar]>>> # Now we can add the function to the ENRICHMENT phase of the enrichment topology
+[Stellar]>>> ?ENRICHMENT_STELLAR_TRANSFORM_ADD
+ENRICHMENT_STELLAR_TRANSFORM_ADD
+Description: Add stellar field transformation.
+
+Arguments:
+ sensorConfig - Sensor config to add transformation to.
+ type - ENRICHMENT or THREAT_INTEL
+ stellarTransforms - A Map associating fields to stellar expressions
+ group - Group to add to (optional)
+
+Returns: The String representation of the config in zookeeper
+[Stellar]>>> squid_enrichment_config_new := ENRICHMENT_STELLAR_TRANSFORM_ADD( squid_enrichment_config, 'ENRICHMENT', SHELL_VARS2MAP( 'is_local' ) )
+[Stellar]>>> # Make sure that it is really there
+[Stellar]>>> ENRICHMENT_STELLAR_TRANSFORM_PRINT(squid_enrichment_config_new, 'ENRICHMENT')
+╔═══════════╤══════════╤════════════════════════════════════════════════════════════════════════════════╗
+║ Group │ Field │ Transformation ║
+╠═══════════╪══════════╪════════════════════════════════════════════════════════════════════════════════╣
+║ (default) │ is_local │ IN_SUBNET( if IS_IP(ip_src_addr) then ip_src_addr else NULL, '192.168.0.0/21') ║
+╚═══════════╧══════════╧════════════════════════════════════════════════════════════════════════════════╝
+
+[Stellar]>>> # and not in threat intel
+[Stellar]>>> ENRICHMENT_STELLAR_TRANSFORM_PRINT(squid_enrichment_config_new, 'THREAT_INTEL')
+╔═══════╤═══════╤════════════════╗
<TRUNCATED>
[09/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-analytics/metron-statistics/HLLP.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/metron-statistics/HLLP.html b/site/current-book/metron-analytics/metron-statistics/HLLP.html
new file mode 100644
index 0000000..7d389ab
--- /dev/null
+++ b/site/current-book/metron-analytics/metron-statistics/HLLP.html
@@ -0,0 +1,7587 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – HyperLogLogPlus</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">HyperLogLogPlus</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-down"></i>
+ Analytics</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service">
+ <i class="none"></i>
+ Maas-service</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-profiler/index.html" title="Profiler">
+ <i class="none"></i>
+ Profiler</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client">
+ <i class="none"></i>
+ Profiler-client</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/metron-statistics/index.html" title="Statistics">
+ <i class="icon-chevron-down"></i>
+ Statistics</a>
+ <ul class="nav nav-list">
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>HLLP</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>HyperLogLogPlus</h1>
+<p><a name="HyperLogLogPlus"></a></p>
+<div class="section">
+<h2><a name="Preliminaries_-_Distinct_Value_cardinality_Estimation"></a>Preliminaries - Distinct Value (cardinality) Estimation</h2>
+<p>Calculating the number of distinct values (DV) in a specific finite-sized list of values using brute force (e.g. a HashSet) techniques is fairly straight forward provided that linear scaling to your number of values fits within your memory constraints. For many cases, the simple calculation serves just fine. However, in many other instances the simple linear approach simply will not scale as well as we would like, and a more robust algorithm is desirable. Enter probabilistic distinct value estimation algorithms: k-minimum-values (KMV), LogLog, HyperLogLog (HLL), and most recently HyperLogLog++ (HLLP). As data volume grows it becomes impractical to store all values in memory in order to reasonably calculate set cardinality. This is where probabilistic estimation algorithms become extremely useful. Often, we are happy to trade a handful of memory and better performance in exchange for an inexact, yet reasonably close approximation of the actual cardinality.</p></div>
+<div class="section">
+<h2><a name="The_Case_For_An_Improvement"></a>The Case For An Improvement</h2>
+<p>HyperLogLogPlus is an algorithm introduced by Google that builds on the DV estimation capabilities of HLL and improves on accuracy as well as scalability. HLLP not only works well with <i>extremely</i> large data sets (think <i>trillion</i>), but with smaller cardinality sets as well. HyperLogLog is simply not as accurate on the lower end of the cardinality spectrum. In addition, HLLP provides the ability to easily merge estimation sets. Although a data stream may not initially require high cardinality estimates as afforded by HLLP, it will still work accurately and efficiently on the smaller volumes without the need to rewrite code if data volume begins to surpass memory constraints. The paper describing the algorithm can be found here - <a class="externalLink" href="https://research.google.com/pubs/pub40671.html">https://research.google.com/pubs/pub40671.html</a></p></div>
+<div class="section">
+<h2><a name="Performance_Metrics"></a>Performance Metrics</h2>
+<div class="section">
+<h3><a name="Overview"></a>Overview</h3>
+<p>Below is a table detailing the performance characteristics when running HLLP over multiple data set sizes. There are three primary variables manipulated during the test: cardinality, sparse set precision, and normal set precision. Each of these values will have an impact on error rates, execution time, and total memory consumption.</p></div>
+<div class="section">
+<h3><a name="Choosing_p_sp"></a>Choosing p, sp</h3>
+<p>There is a tradeoff when choosing values for p and sp. A higher value for sp means a lower error rate for instances when the sparse set is being used. However, a higher value for sp also requires more memory, which means the algorithm will cut over to the dense representation more quickly. The Google paper (listed above) recommends settings of p=25, sp=14 for optimal memory use and algorithm accuracy, and these are the defaults provided by the Apache Metron (Incubating) implementation in Stellar. You’ll notice in the tests below, memory consumption tops out around 12KB for this setting, regardless of cardinality size.</p></div>
+<div class="section">
+<h3><a name="Key"></a>Key</h3>
+<hr />
+
+<div class="source">
+<div class="source">
+<pre>card: cardinality<br>
+sp: sparse precision value<br>
+p: normal (dense) precision value<br>
+err: error as a percent of the expected cardinality<br>
+time: total time to add values to the hllp estimator and calculate a cardinality estimate<br>
+size: size of the hllp set in bytes once all values have been added for the specified cardinality<br>
+l=low, m=mid(based on percentile chosen), h=high, std=standard deviation<br>
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Table_Cardinality_200-1000_step_by_200"></a>Table Cardinality 200-1000, step by 200</h3>
+<hr />
+<div class="section">
+<h4><a name="Options_Used"></a>Options Used</h4>
+
+<div class="source">
+<div class="source">
+<pre>num trials: 5000
+card min: 200
+card max: 1000
+card step: 200
+card start: 200
+sp min: 4
+sp max: 32
+sp step: 4
+error percentile: 50.0
+time percentile: 50.0
+size percentile: 50.0
+format err as %: true
+</pre></div></div>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>card </th>
+
+<th>sp </th>
+
+<th>p </th>
+
+<th>err l/m/h/std (% of actual) </th>
+
+<th>time l/m/h/std (ms) </th>
+
+<th>size l/m/h/std (b) </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>4 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.000 / 180.000 / 17.470 </td>
+
+<td>0.070 / 0.118 / 10.092 / 0.204 </td>
+
+<td>218 / 220 / 220 / 1 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>8 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.000 / 209.500 / 18.060 </td>
+
+<td>0.071 / 0.085 / 8.786 / 0.128 </td>
+
+<td>219 / 221 / 221 / 1 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>8 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.500 / 21.500 / 3.185 </td>
+
+<td>0.082 / 0.085 / 8.375 / 0.119 </td>
+
+<td>531 / 587 / 637 / 14 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>12 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.000 / 174.000 / 17.376 </td>
+
+<td>0.070 / 0.085 / 8.708 / 0.128 </td>
+
+<td>219 / 221 / 221 / 1 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>12 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.000 / 25.500 / 2.847 </td>
+
+<td>0.086 / 0.119 / 8.984 / 0.151 </td>
+
+<td>353 / 372 / 627 / 74 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>12 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.500 / 5.500 / 0.659 </td>
+
+<td>0.082 / 0.096 / 17.208 / 0.243 </td>
+
+<td>775 / 816 / 834 / 9 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>16 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.500 / 172.000 / 16.622 </td>
+
+<td>0.068 / 0.075 / 9.553 / 0.137 </td>
+
+<td>220 / 222 / 222 / 1 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>16 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.500 / 18.000 / 2.899 </td>
+
+<td>0.095 / 0.118 / 0.663 / 0.031 </td>
+
+<td>354 / 373 / 390 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>16 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 1.500 / 0.276 </td>
+
+<td>0.085 / 0.108 / 13.977 / 0.199 </td>
+
+<td>802 / 833 / 852 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>16 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 2.500 / 0.279 </td>
+
+<td>0.079 / 0.085 / 0.271 / 0.017 </td>
+
+<td>990 / 1027 / 1044 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>20 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.000 / 209.500 / 17.424 </td>
+
+<td>0.068 / 0.072 / 0.250 / 0.014 </td>
+
+<td>221 / 223 / 223 / 1 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>20 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.500 / 18.000 / 2.944 </td>
+
+<td>0.096 / 0.106 / 13.581 / 0.192 </td>
+
+<td>357 / 374 / 388 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>20 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 1.000 / 0.066 </td>
+
+<td>0.084 / 0.088 / 1.096 / 0.019 </td>
+
+<td>915 / 945 / 972 / 8 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>20 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.070 </td>
+
+<td>0.083 / 0.087 / 10.272 / 0.145 </td>
+
+<td>912 / 953 / 982 / 8 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>20 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 1.000 / 0.069 </td>
+
+<td>0.079 / 0.081 / 0.210 / 0.017 </td>
+
+<td>1028 / 1056 / 1058 / 2 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>24 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.500 / 136.000 / 16.589 </td>
+
+<td>0.068 / 0.071 / 0.952 / 0.025 </td>
+
+<td>221 / 223 / 223 / 1 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>24 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.500 / 18.000 / 2.955 </td>
+
+<td>0.095 / 0.106 / 10.387 / 0.148 </td>
+
+<td>353 / 373 / 390 / 5 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>24 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.016 </td>
+
+<td>0.082 / 0.083 / 0.235 / 0.014 </td>
+
+<td>1035 / 1048 / 1056 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>24 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.012 </td>
+
+<td>0.082 / 0.084 / 10.904 / 0.154 </td>
+
+<td>1038 / 1051 / 1059 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>24 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.010 </td>
+
+<td>0.082 / 0.085 / 0.410 / 0.019 </td>
+
+<td>1049 / 1064 / 1082 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>24 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.023 </td>
+
+<td>0.079 / 0.084 / 11.926 / 0.168 </td>
+
+<td>1247 / 1264 / 1275 / 4 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>28 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.500 / 229.000 / 17.294 </td>
+
+<td>0.068 / 0.072 / 0.369 / 0.015 </td>
+
+<td>222 / 224 / 224 / 1 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>28 </td>
+
+<td>8 </td>
+
+<td>83.000 / 83.000 / 84.000 / 0.118 </td>
+
+<td>0.093 / 0.098 / 10.918 / 0.157 </td>
+
+<td>272 / 276 / 278 / 1 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>28 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.016 </td>
+
+<td>0.082 / 0.083 / 10.157 / 0.143 </td>
+
+<td>1032 / 1067 / 1075 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>28 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.007 </td>
+
+<td>0.082 / 0.083 / 0.617 / 0.020 </td>
+
+<td>1058 / 1069 / 1077 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>28 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.010 </td>
+
+<td>0.082 / 0.085 / 11.619 / 0.165 </td>
+
+<td>1060 / 1071 / 1080 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>28 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.007 </td>
+
+<td>0.083 / 0.085 / 0.516 / 0.022 </td>
+
+<td>1065 / 1083 / 1100 / 4 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>28 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.019 </td>
+
+<td>0.080 / 0.087 / 11.278 / 0.160 </td>
+
+<td>1250 / 1267 / 1278 / 4 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>32 </td>
+
+<td>4 </td>
+
+<td>0.000 / 18.000 / 175.500 / 17.157 </td>
+
+<td>0.068 / 0.078 / 0.223 / 0.019 </td>
+
+<td>222 / 224 / 224 / 1 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>32 </td>
+
+<td>8 </td>
+
+<td>99.000 / 99.000 / 99.000 / 0.000 </td>
+
+<td>0.093 / 0.107 / 11.695 / 0.167 </td>
+
+<td>258 / 258 / 258 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>32 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.014 </td>
+
+<td>0.081 / 0.084 / 8.906 / 0.126 </td>
+
+<td>1060 / 1071 / 1075 / 2 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>32 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.010 </td>
+
+<td>0.082 / 0.085 / 1.458 / 0.027 </td>
+
+<td>1061 / 1073 / 1077 / 2 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>32 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.012 </td>
+
+<td>0.082 / 0.084 / 10.499 / 0.148 </td>
+
+<td>1065 / 1074 / 1079 / 2 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>32 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.012 </td>
+
+<td>0.082 / 0.085 / 0.420 / 0.020 </td>
+
+<td>1066 / 1075 / 1082 / 2 </td>
+ </tr>
+
+<tr class="b">
+
+<td>200 </td>
+
+<td>32 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.016 </td>
+
+<td>0.082 / 0.086 / 8.753 / 0.124 </td>
+
+<td>1076 / 1088 / 1102 / 4 </td>
+ </tr>
+
+<tr class="a">
+
+<td>200 </td>
+
+<td>32 </td>
+
+<td>32 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.010 </td>
+
+<td>0.080 / 0.081 / 13.248 / 0.188 </td>
+
+<td>1249 / 1268 / 1279 / 4 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>4 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.500 / 175.250 / 17.530 </td>
+
+<td>0.136 / 0.142 / 0.591 / 0.035 </td>
+
+<td>218 / 220 / 220 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>8 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.000 / 148.500 / 17.174 </td>
+
+<td>0.136 / 0.156 / 12.882 / 0.241 </td>
+
+<td>219 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>8 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.500 / 21.250 / 3.215 </td>
+
+<td>0.159 / 0.186 / 10.396 / 0.151 </td>
+
+<td>375 / 387 / 802 / 62 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>12 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.750 / 166.000 / 17.363 </td>
+
+<td>0.135 / 0.142 / 11.607 / 0.165 </td>
+
+<td>219 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>12 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.250 / 23.750 / 3.133 </td>
+
+<td>0.166 / 0.179 / 8.713 / 0.130 </td>
+
+<td>376 / 387 / 401 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>12 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.750 / 4.250 / 0.706 </td>
+
+<td>0.159 / 0.163 / 10.537 / 0.189 </td>
+
+<td>1334 / 1387 / 1434 / 14 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>16 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.125 / 163.250 / 17.208 </td>
+
+<td>0.135 / 0.138 / 12.403 / 0.178 </td>
+
+<td>220 / 222 / 222 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>16 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.250 / 23.500 / 3.126 </td>
+
+<td>0.165 / 0.179 / 9.985 / 0.147 </td>
+
+<td>378 / 388 / 401 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>16 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.250 / 1.500 / 0.195 </td>
+
+<td>0.166 / 0.173 / 10.434 / 0.150 </td>
+
+<td>1414 / 1446 / 1475 / 8 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>16 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.250 / 1.750 / 0.194 </td>
+
+<td>0.160 / 0.177 / 1.300 / 0.048 </td>
+
+<td>1761 / 1798 / 1825 / 8 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>20 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.750 / 270.750 / 17.592 </td>
+
+<td>0.135 / 0.149 / 8.814 / 0.128 </td>
+
+<td>221 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>20 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.500 / 23.000 / 3.103 </td>
+
+<td>0.165 / 0.183 / 9.271 / 0.139 </td>
+
+<td>379 / 389 / 398 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>20 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.066 </td>
+
+<td>0.164 / 0.182 / 11.093 / 0.162 </td>
+
+<td>1617 / 1652 / 1683 / 10 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>20 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.070 </td>
+
+<td>0.166 / 0.170 / 8.676 / 0.128 </td>
+
+<td>1624 / 1665 / 1706 / 10 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>20 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.750 / 0.069 </td>
+
+<td>0.159 / 0.162 / 12.846 / 0.183 </td>
+
+<td>1840 / 1853 / 1856 / 2 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>24 </td>
+
+<td>4 </td>
+
+<td>0.000 / 16.750 / 174.750 / 16.786 </td>
+
+<td>0.135 / 0.145 / 8.586 / 0.128 </td>
+
+<td>221 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>24 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.500 / 24.500 / 3.090 </td>
+
+<td>0.165 / 0.179 / 11.344 / 0.164 </td>
+
+<td>377 / 389 / 399 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>24 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.016 </td>
+
+<td>0.164 / 0.183 / 10.708 / 0.159 </td>
+
+<td>1828 / 1841 / 1852 / 4 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>24 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.017 </td>
+
+<td>0.164 / 0.172 / 11.180 / 0.161 </td>
+
+<td>1830 / 1844 / 1855 / 4 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>24 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.015 </td>
+
+<td>0.166 / 0.170 / 9.589 / 0.140 </td>
+
+<td>1843 / 1868 / 1890 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>24 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.022 </td>
+
+<td>0.160 / 0.162 / 9.137 / 0.134 </td>
+
+<td>2213 / 2232 / 2247 / 5 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>28 </td>
+
+<td>4 </td>
+
+<td>0.000 / 18.250 / 164.000 / 16.587 </td>
+
+<td>0.135 / 0.144 / 14.520 / 0.209 </td>
+
+<td>222 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>28 </td>
+
+<td>8 </td>
+
+<td>33.000 / 47.750 / 56.750 / 3.588 </td>
+
+<td>0.164 / 0.168 / 1.305 / 0.052 </td>
+
+<td>355 / 375 / 394 / 5 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>28 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.006 </td>
+
+<td>0.165 / 0.178 / 11.264 / 0.167 </td>
+
+<td>1837 / 1848 / 1854 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>28 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.014 </td>
+
+<td>0.164 / 0.167 / 8.689 / 0.130 </td>
+
+<td>1838 / 1850 / 1857 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>28 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.011 </td>
+
+<td>0.164 / 0.167 / 10.679 / 0.159 </td>
+
+<td>1839 / 1852 / 1863 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>28 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.011 </td>
+
+<td>0.166 / 0.175 / 12.887 / 0.190 </td>
+
+<td>1857 / 1875 / 1896 / 5 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>28 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.015 </td>
+
+<td>0.160 / 0.196 / 1.478 / 0.071 </td>
+
+<td>2217 / 2235 / 2252 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>32 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.500 / 183.750 / 19.308 </td>
+
+<td>0.135 / 0.142 / 9.949 / 0.148 </td>
+
+<td>222 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>32 </td>
+
+<td>8 </td>
+
+<td>46.500 / 56.250 / 63.250 / 2.194 </td>
+
+<td>0.164 / 0.181 / 1.209 / 0.063 </td>
+
+<td>349 / 370 / 390 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>32 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.012 </td>
+
+<td>0.164 / 0.171 / 11.529 / 0.170 </td>
+
+<td>1837 / 1848 / 1854 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>32 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.016 </td>
+
+<td>0.164 / 0.167 / 10.276 / 0.154 </td>
+
+<td>1836 / 1850 / 1856 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>32 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.015 </td>
+
+<td>0.164 / 0.184 / 11.611 / 0.227 </td>
+
+<td>1839 / 1851 / 1858 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>32 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.012 </td>
+
+<td>0.165 / 0.171 / 9.313 / 0.143 </td>
+
+<td>1842 / 1853 / 1862 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>400 </td>
+
+<td>32 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.013 </td>
+
+<td>0.166 / 0.180 / 10.708 / 0.162 </td>
+
+<td>1860 / 1877 / 1900 / 5 </td>
+ </tr>
+
+<tr class="a">
+
+<td>400 </td>
+
+<td>32 </td>
+
+<td>32 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.014 </td>
+
+<td>0.160 / 0.166 / 1.205 / 0.057 </td>
+
+<td>2217 / 2235 / 2249 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>4 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.333 / 219.667 / 17.222 </td>
+
+<td>0.202 / 0.240 / 13.390 / 0.237 </td>
+
+<td>218 / 220 / 220 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>8 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.417 / 231.333 / 17.340 </td>
+
+<td>0.202 / 0.224 / 9.662 / 0.150 </td>
+
+<td>220 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>8 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.500 / 24.333 / 3.240 </td>
+
+<td>0.234 / 0.271 / 11.443 / 0.216 </td>
+
+<td>382 / 392 / 403 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>12 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.167 / 188.167 / 17.324 </td>
+
+<td>0.202 / 0.210 / 1.320 / 0.055 </td>
+
+<td>219 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>12 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.500 / 23.833 / 3.243 </td>
+
+<td>0.235 / 0.246 / 10.821 / 0.164 </td>
+
+<td>382 / 392 / 403 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>12 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.667 / 4.500 / 0.694 </td>
+
+<td>0.240 / 0.259 / 12.004 / 0.183 </td>
+
+<td>2015 / 2101 / 2178 / 21 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>16 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.500 / 170.000 / 17.882 </td>
+
+<td>0.202 / 0.218 / 9.208 / 0.140 </td>
+
+<td>221 / 222 / 222 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>16 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.667 / 22.667 / 3.276 </td>
+
+<td>0.234 / 0.251 / 12.301 / 0.237 </td>
+
+<td>383 / 393 / 408 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>16 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.167 / 1.000 / 0.170 </td>
+
+<td>0.250 / 0.284 / 12.000 / 0.180 </td>
+
+<td>2199 / 2243 / 2282 / 12 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>16 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.167 / 1.167 / 0.171 </td>
+
+<td>0.239 / 0.291 / 21.163 / 0.333 </td>
+
+<td>2811 / 2859 / 2901 / 13 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>20 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.667 / 176.667 / 18.162 </td>
+
+<td>0.202 / 0.223 / 16.405 / 0.275 </td>
+
+<td>221 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>20 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.500 / 20.833 / 3.180 </td>
+
+<td>0.234 / 0.243 / 5.621 / 0.104 </td>
+
+<td>385 / 394 / 405 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>20 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.066 </td>
+
+<td>0.246 / 0.281 / 19.279 / 0.390 </td>
+
+<td>2541 / 2604 / 2663 / 15 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>20 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.667 / 0.071 </td>
+
+<td>0.248 / 0.258 / 24.627 / 0.387 </td>
+
+<td>2569 / 2627 / 2682 / 15 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>20 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.069 </td>
+
+<td>0.239 / 0.246 / 13.606 / 0.248 </td>
+
+<td>2938 / 2954 / 2972 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>24 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.333 / 210.333 / 18.157 </td>
+
+<td>0.202 / 0.218 / 11.991 / 0.180 </td>
+
+<td>222 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>24 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.667 / 21.167 / 3.184 </td>
+
+<td>0.234 / 0.258 / 11.130 / 0.238 </td>
+
+<td>385 / 394 / 406 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>24 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.017 </td>
+
+<td>0.246 / 0.272 / 12.032 / 0.191 </td>
+
+<td>2912 / 2935 / 2954 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>24 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.019 </td>
+
+<td>0.246 / 0.268 / 19.358 / 0.282 </td>
+
+<td>2916 / 2940 / 2959 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>24 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.015 </td>
+
+<td>0.248 / 0.259 / 20.293 / 0.343 </td>
+
+<td>2948 / 2980 / 3009 / 9 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>24 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.017 </td>
+
+<td>0.239 / 0.252 / 11.721 / 0.230 </td>
+
+<td>3586 / 3618 / 3643 / 7 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>28 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.667 / 182.667 / 17.309 </td>
+
+<td>0.202 / 0.210 / 11.957 / 0.177 </td>
+
+<td>223 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>28 </td>
+
+<td>8 </td>
+
+<td>10.333 / 28.000 / 42.833 / 3.687 </td>
+
+<td>0.233 / 0.238 / 10.166 / 0.210 </td>
+
+<td>377 / 392 / 403 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>28 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.333 / 0.013 </td>
+
+<td>0.246 / 0.252 / 9.907 / 0.158 </td>
+
+<td>3005 / 3024 / 3041 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>28 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.012 </td>
+
+<td>0.246 / 0.256 / 18.673 / 0.273 </td>
+
+<td>3007 / 3026 / 3045 / 5 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>28 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.011 </td>
+
+<td>0.246 / 0.283 / 17.762 / 0.355 </td>
+
+<td>3005 / 3029 / 3047 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>28 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.011 </td>
+
+<td>0.248 / 0.254 / 8.821 / 0.138 </td>
+
+<td>3035 / 3065 / 3098 / 8 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>28 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.011 </td>
+
+<td>0.239 / 0.242 / 12.171 / 0.177 </td>
+
+<td>3589 / 3621 / 3646 / 7 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>32 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.333 / 200.000 / 18.294 </td>
+
+<td>0.202 / 0.205 / 9.036 / 0.138 </td>
+
+<td>223 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>32 </td>
+
+<td>8 </td>
+
+<td>25.167 / 37.167 / 46.833 / 3.185 </td>
+
+<td>0.232 / 0.236 / 9.061 / 0.144 </td>
+
+<td>377 / 389 / 403 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>32 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.013 </td>
+
+<td>0.246 / 0.256 / 11.119 / 0.169 </td>
+
+<td>3028 / 3043 / 3055 / 4 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>32 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.010 </td>
+
+<td>0.246 / 0.257 / 11.557 / 0.213 </td>
+
+<td>3030 / 3045 / 3057 / 4 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>32 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.011 </td>
+
+<td>0.246 / 0.266 / 20.353 / 0.295 </td>
+
+<td>3030 / 3047 / 3071 / 4 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>32 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.012 </td>
+
+<td>0.246 / 0.270 / 11.461 / 0.183 </td>
+
+<td>3035 / 3050 / 3073 / 4 </td>
+ </tr>
+
+<tr class="b">
+
+<td>600 </td>
+
+<td>32 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.013 </td>
+
+<td>0.248 / 0.260 / 11.755 / 0.242 </td>
+
+<td>3060 / 3085 / 3108 / 7 </td>
+ </tr>
+
+<tr class="a">
+
+<td>600 </td>
+
+<td>32 </td>
+
+<td>32 </td>
+
+<td>0.000 / 0.000 / 0.167 / 0.012 </td>
+
+<td>0.239 / 0.248 / 10.380 / 0.157 </td>
+
+<td>3593 / 3621 / 3645 / 7 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>4 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.250 / 188.250 / 17.248 </td>
+
+<td>0.269 / 0.280 / 10.818 / 0.212 </td>
+
+<td>219 / 220 / 220 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>8 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.875 / 184.125 / 17.721 </td>
+
+<td>0.270 / 0.298 / 16.706 / 0.252 </td>
+
+<td>220 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>8 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.625 / 22.000 / 3.420 </td>
+
+<td>0.303 / 0.315 / 12.489 / 0.243 </td>
+
+<td>386 / 395 / 406 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>12 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.750 / 196.500 / 16.870 </td>
+
+<td>0.269 / 0.289 / 13.533 / 0.235 </td>
+
+<td>220 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>12 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.750 / 25.875 / 3.359 </td>
+
+<td>0.303 / 0.330 / 18.306 / 0.365 </td>
+
+<td>386 / 395 / 408 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>12 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.750 / 4.375 / 0.677 </td>
+
+<td>0.322 / 0.357 / 20.131 / 0.350 </td>
+
+<td>2449 / 2603 / 2694 / 27 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>16 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.750 / 195.250 / 17.455 </td>
+
+<td>0.269 / 0.274 / 10.472 / 0.205 </td>
+
+<td>221 / 222 / 222 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>16 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.750 / 20.375 / 3.357 </td>
+
+<td>0.302 / 0.308 / 10.637 / 0.165 </td>
+
+<td>386 / 396 / 409 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>16 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.125 / 1.375 / 0.170 </td>
+
+<td>0.334 / 0.344 / 10.745 / 0.221 </td>
+
+<td>2712 / 2836 / 2882 / 15 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>16 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.250 / 1.375 / 0.169 </td>
+
+<td>0.321 / 0.334 / 13.979 / 0.252 </td>
+
+<td>3390 / 3625 / 3668 / 19 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>20 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.750 / 220.750 / 17.953 </td>
+
+<td>0.269 / 0.281 / 9.422 / 0.148 </td>
+
+<td>222 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>20 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.750 / 21.875 / 3.305 </td>
+
+<td>0.302 / 0.358 / 24.073 / 0.435 </td>
+
+<td>387 / 397 / 408 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>20 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.067 </td>
+
+<td>0.331 / 0.345 / 13.461 / 0.211 </td>
+
+<td>3249 / 3304 / 3360 / 17 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>20 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.069 </td>
+
+<td>0.333 / 0.349 / 8.271 / 0.146 </td>
+
+<td>3190 / 3331 / 3392 / 17 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>20 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.375 / 0.069 </td>
+
+<td>0.320 / 0.335 / 11.307 / 0.239 </td>
+
+<td>3539 / 3752 / 3771 / 5 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>24 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.125 / 184.500 / 18.184 </td>
+
+<td>0.269 / 0.283 / 1.541 / 0.079 </td>
+
+<td>222 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>24 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.625 / 23.125 / 3.301 </td>
+
+<td>0.302 / 0.317 / 10.845 / 0.169 </td>
+
+<td>388 / 397 / 409 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>24 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.017 </td>
+
+<td>0.330 / 0.352 / 10.221 / 0.244 </td>
+
+<td>3703 / 3729 / 3749 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>24 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.018 </td>
+
+<td>0.330 / 0.344 / 13.454 / 0.239 </td>
+
+<td>3711 / 3734 / 3753 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>24 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.018 </td>
+
+<td>0.333 / 0.348 / 16.987 / 0.374 </td>
+
+<td>3751 / 3786 / 3826 / 10 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>24 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.018 </td>
+
+<td>0.320 / 0.333 / 17.097 / 0.288 </td>
+
+<td>4567 / 4605 / 4630 / 8 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>28 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.375 / 326.125 / 18.121 </td>
+
+<td>0.269 / 0.274 / 9.600 / 0.149 </td>
+
+<td>223 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>28 </td>
+
+<td>8 </td>
+
+<td>0.375 / 19.875 / 34.500 / 4.445 </td>
+
+<td>0.302 / 0.313 / 10.025 / 0.181 </td>
+
+<td>387 / 397 / 409 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>28 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.125 / 0.011 </td>
+
+<td>0.330 / 0.343 / 11.347 / 0.216 </td>
+
+<td>3797 / 3821 / 3839 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>28 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.125 / 0.010 </td>
+
+<td>0.330 / 0.344 / 10.748 / 0.219 </td>
+
+<td>3801 / 3823 / 3843 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>28 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.125 / 0.011 </td>
+
+<td>0.330 / 0.355 / 15.268 / 0.314 </td>
+
+<td>3802 / 3827 / 3845 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>28 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.125 / 0.011 </td>
+
+<td>0.334 / 0.360 / 11.192 / 0.243 </td>
+
+<td>3841 / 3874 / 3905 / 9 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>28 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.011 </td>
+
+<td>0.321 / 0.334 / 1.757 / 0.095 </td>
+
+<td>4575 / 4609 / 4635 / 8 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>32 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.125 / 161.000 / 18.370 </td>
+
+<td>0.269 / 0.279 / 20.700 / 0.344 </td>
+
+<td>223 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>32 </td>
+
+<td>8 </td>
+
+<td>10.500 / 27.750 / 39.500 / 3.883 </td>
+
+<td>0.301 / 0.333 / 18.781 / 0.332 </td>
+
+<td>385 / 395 / 406 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>32 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.011 </td>
+
+<td>0.330 / 0.371 / 18.883 / 0.317 </td>
+
+<td>3823 / 3840 / 3853 / 4 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>32 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.125 / 0.011 </td>
+
+<td>0.330 / 0.353 / 9.319 / 0.195 </td>
+
+<td>3824 / 3842 / 3855 / 4 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>32 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.125 / 0.012 </td>
+
+<td>0.330 / 0.353 / 15.582 / 0.305 </td>
+
+<td>3826 / 3843 / 3859 / 4 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>32 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.125 / 0.013 </td>
+
+<td>0.330 / 0.349 / 11.346 / 0.234 </td>
+
+<td>3830 / 3847 / 3862 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>800 </td>
+
+<td>32 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.250 / 0.013 </td>
+
+<td>0.333 / 0.368 / 14.482 / 0.309 </td>
+
+<td>3866 / 3894 / 3925 / 8 </td>
+ </tr>
+
+<tr class="a">
+
+<td>800 </td>
+
+<td>32 </td>
+
+<td>32 </td>
+
+<td>0.000 / 0.000 / 0.125 / 0.012 </td>
+
+<td>0.320 / 0.335 / 17.771 / 0.295 </td>
+
+<td>4576 / 4608 / 4633 / 8 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>4 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.800 / 187.500 / 18.576 </td>
+
+<td>0.337 / 0.374 / 10.384 / 0.179 </td>
+
+<td>219 / 220 / 220 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>8 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.600 / 181.100 / 18.035 </td>
+
+<td>0.336 / 0.372 / 13.220 / 0.262 </td>
+
+<td>220 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>8 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.900 / 23.100 / 3.425 </td>
+
+<td>0.371 / 0.398 / 9.495 / 0.175 </td>
+
+<td>386 / 398 / 409 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>12 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.600 / 178.900 / 17.559 </td>
+
+<td>0.336 / 0.351 / 11.176 / 0.265 </td>
+
+<td>220 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>12 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.800 / 22.700 / 3.438 </td>
+
+<td>0.371 / 0.392 / 10.942 / 0.188 </td>
+
+<td>387 / 398 / 411 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>12 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.800 / 4.200 / 0.684 </td>
+
+<td>0.403 / 0.443 / 11.219 / 0.279 </td>
+
+<td>2971 / 3242 / 3364 / 39 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>16 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.600 / 184.300 / 17.148 </td>
+
+<td>0.336 / 0.368 / 21.656 / 0.350 </td>
+
+<td>221 / 222 / 222 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>16 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.800 / 21.900 / 3.350 </td>
+
+<td>0.369 / 0.385 / 15.840 / 0.277 </td>
+
+<td>388 / 399 / 412 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>16 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.200 / 1.200 / 0.174 </td>
+
+<td>0.422 / 0.438 / 16.035 / 0.335 </td>
+
+<td>3428 / 3530 / 3628 / 16 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>16 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.200 / 1.200 / 0.173 </td>
+
+<td>0.405 / 0.420 / 13.370 / 0.305 </td>
+
+<td>4314 / 4484 / 4573 / 20 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>20 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.600 / 223.000 / 16.968 </td>
+
+<td>0.336 / 0.372 / 17.746 / 0.392 </td>
+
+<td>222 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>20 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.800 / 21.400 / 3.384 </td>
+
+<td>0.369 / 0.419 / 16.486 / 0.257 </td>
+
+<td>389 / 400 / 412 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>20 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.400 / 0.068 </td>
+
+<td>0.417 / 0.446 / 12.313 / 0.265 </td>
+
+<td>3954 / 4099 / 4158 / 18 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>20 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.500 / 0.069 </td>
+
+<td>0.422 / 0.439 / 11.284 / 0.262 </td>
+
+<td>4016 / 4134 / 4211 / 19 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>20 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.600 / 0.069 </td>
+
+<td>0.404 / 0.429 / 16.987 / 0.335 </td>
+
+<td>4517 / 4641 / 4651 / 5 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>24 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.600 / 226.700 / 17.761 </td>
+
+<td>0.339 / 0.351 / 12.804 / 0.275 </td>
+
+<td>222 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>24 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.800 / 24.100 / 3.403 </td>
+
+<td>0.371 / 0.406 / 15.710 / 0.298 </td>
+
+<td>390 / 400 / 412 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>24 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.018 </td>
+
+<td>0.417 / 0.449 / 12.253 / 0.294 </td>
+
+<td>4494 / 4614 / 4636 / 7 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>24 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.018 </td>
+
+<td>0.417 / 0.434 / 11.046 / 0.254 </td>
+
+<td>4594 / 4620 / 4641 / 7 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>24 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.019 </td>
+
+<td>0.422 / 0.458 / 13.433 / 0.270 </td>
+
+<td>4645 / 4682 / 4722 / 11 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>24 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.016 </td>
+
+<td>0.405 / 0.435 / 11.470 / 0.189 </td>
+
+<td>5635 / 5672 / 5703 / 9 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>28 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.200 / 148.500 / 17.046 </td>
+
+<td>0.338 / 0.363 / 20.446 / 0.380 </td>
+
+<td>224 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>28 </td>
+
+<td>8 </td>
+
+<td>0.000 / 15.100 / 30.700 / 4.794 </td>
+
+<td>0.371 / 0.389 / 18.272 / 0.359 </td>
+
+<td>390 / 400 / 411 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>28 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.013 </td>
+
+<td>0.417 / 0.432 / 13.575 / 0.264 </td>
+
+<td>4677 / 4700 / 4719 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>28 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.013 </td>
+
+<td>0.417 / 0.435 / 18.060 / 0.381 </td>
+
+<td>4545 / 4702 / 4723 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>28 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.013 </td>
+
+<td>0.417 / 0.448 / 17.596 / 0.344 </td>
+
+<td>4679 / 4707 / 4728 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>28 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.012 </td>
+
+<td>0.421 / 0.454 / 13.807 / 0.297 </td>
+
+<td>4733 / 4764 / 4801 / 10 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>28 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.013 </td>
+
+<td>0.406 / 0.456 / 20.533 / 0.353 </td>
+
+<td>5646 / 5675 / 5704 / 9 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>32 </td>
+
+<td>4 </td>
+
+<td>0.000 / 18.150 / 177.600 / 19.537 </td>
+
+<td>0.338 / 0.352 / 12.293 / 0.252 </td>
+
+<td>223 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>32 </td>
+
+<td>8 </td>
+
+<td>1.500 / 22.100 / 35.100 / 4.399 </td>
+
+<td>0.371 / 0.407 / 17.782 / 0.386 </td>
+
+<td>388 / 398 / 412 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>32 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.011 </td>
+
+<td>0.417 / 0.435 / 11.830 / 0.247 </td>
+
+<td>4698 / 4717 / 4730 / 5 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>32 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.013 </td>
+
+<td>0.417 / 0.443 / 18.214 / 0.276 </td>
+
+<td>4700 / 4719 / 4733 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>32 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.012 </td>
+
+<td>0.417 / 0.437 / 11.208 / 0.232 </td>
+
+<td>4703 / 4720 / 4737 / 5 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>32 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.013 </td>
+
+<td>0.417 / 0.448 / 18.288 / 0.405 </td>
+
+<td>4706 / 4724 / 4740 / 5 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1000 </td>
+
+<td>32 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.200 / 0.011 </td>
+
+<td>0.420 / 0.437 / 13.389 / 0.245 </td>
+
+<td>4613 / 4782 / 4814 / 9 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1000 </td>
+
+<td>32 </td>
+
+<td>32 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.012 </td>
+
+<td>0.406 / 0.424 / 12.513 / 0.232 </td>
+
+<td>5641 / 5675 / 5704 / 9 </td>
+ </tr>
+ </tbody>
+</table></div></div></div>
+<div class="section">
+<h2><a name="Table_Cardinality_1500-5000_step_by_500"></a>Table Cardinality 1500-5000, step by 500</h2>
+<div class="section">
+<div class="section">
+<h4><a name="Options_Used"></a>Options Used</h4>
+
+<div class="source">
+<div class="source">
+<pre>num trials: 5000
+card min: 1500
+card max: 5000
+card step: 500
+card start: 1500
+sp min: 4
+sp max: 32
+sp step: 4
+error percentile: 50.0
+time percentile: 50.0
+size percentile: 50.0
+format err as %: true
+</pre></div></div>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>card </th>
+
+<th>sp </th>
+
+<th>p </th>
+
+<th>err l/m/h/std (% of actual) </th>
+
+<th>time l/m/h/std (ms) </th>
+
+<th>size l/m/h/std (b) </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>4 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.600 / 254.400 / 17.627 </td>
+
+<td>0.501 / 0.612 / 19.629 / 0.593 </td>
+
+<td>220 / 220 / 220 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>8 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.600 / 158.133 / 16.980 </td>
+
+<td>0.506 / 0.594 / 16.843 / 0.493 </td>
+
+<td>221 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>8 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.000 / 23.933 / 3.590 </td>
+
+<td>0.533 / 0.567 / 11.059 / 0.259 </td>
+
+<td>392 / 404 / 416 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>12 </td>
+
+<td>4 </td>
+
+<td>0.000 / 18.067 / 208.733 / 17.934 </td>
+
+<td>0.505 / 0.531 / 20.544 / 0.368 </td>
+
+<td>221 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>12 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.067 / 22.133 / 3.579 </td>
+
+<td>0.529 / 0.576 / 22.421 / 0.453 </td>
+
+<td>391 / 404 / 416 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>12 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.800 / 4.133 / 0.707 </td>
+
+<td>0.607 / 0.638 / 15.124 / 0.329 </td>
+
+<td>4338 / 4489 / 4656 / 43 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>16 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.200 / 167.333 / 17.918 </td>
+
+<td>0.508 / 0.546 / 17.715 / 0.381 </td>
+
+<td>221 / 222 / 222 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>16 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.933 / 23.667 / 3.605 </td>
+
+<td>0.530 / 0.564 / 18.635 / 0.407 </td>
+
+<td>393 / 405 / 417 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>16 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.200 / 1.067 / 0.166 </td>
+
+<td>0.621 / 0.689 / 17.660 / 0.364 </td>
+
+<td>5177 / 5271 / 5341 / 21 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>16 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.200 / 0.933 / 0.165 </td>
+
+<td>0.612 / 0.654 / 19.700 / 0.419 </td>
+
+<td>6688 / 6775 / 6855 / 23 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>20 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.600 / 231.333 / 17.757 </td>
+
+<td>0.509 / 0.544 / 20.194 / 0.369 </td>
+
+<td>223 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>20 </td>
+
+<td>8 </td>
+
+<td>0.000 / 3.933 / 25.600 / 3.728 </td>
+
+<td>0.529 / 0.570 / 15.702 / 0.372 </td>
+
+<td>395 / 406 / 417 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>20 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.067 / 0.333 / 0.047 </td>
+
+<td>0.612 / 0.659 / 12.620 / 0.343 </td>
+
+<td>6097 / 6191 / 6281 / 24 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>20 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.067 / 0.400 / 0.047 </td>
+
+<td>0.619 / 0.648 / 13.508 / 0.300 </td>
+
+<td>6163 / 6244 / 6344 / 24 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>20 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.067 / 0.333 / 0.047 </td>
+
+<td>0.610 / 0.650 / 12.478 / 0.323 </td>
+
+<td>7019 / 7048 / 7093 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>24 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.467 / 156.333 / 18.009 </td>
+
+<td>0.508 / 0.569 / 17.878 / 0.386 </td>
+
+<td>223 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>24 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.067 / 27.400 / 3.599 </td>
+
+<td>0.529 / 0.564 / 21.583 / 0.397 </td>
+
+<td>394 / 406 / 416 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>24 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.017 </td>
+
+<td>0.611 / 0.655 / 16.659 / 0.383 </td>
+
+<td>6974 / 7008 / 7039 / 8 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>24 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.017 </td>
+
+<td>0.612 / 0.644 / 10.934 / 0.313 </td>
+
+<td>6981 / 7016 / 7048 / 9 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>24 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.017 </td>
+
+<td>0.620 / 0.683 / 21.873 / 0.512 </td>
+
+<td>7064 / 7115 / 7167 / 14 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>24 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.017 </td>
+
+<td>0.609 / 0.634 / 13.249 / 0.315 </td>
+
+<td>8645 / 8684 / 8722 / 11 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>28 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.467 / 157.067 / 17.312 </td>
+
+<td>0.508 / 0.529 / 14.149 / 0.336 </td>
+
+<td>224 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>28 </td>
+
+<td>8 </td>
+
+<td>0.000 / 8.533 / 25.933 / 4.865 </td>
+
+<td>0.529 / 0.561 / 15.511 / 0.380 </td>
+
+<td>396 / 406 / 419 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>28 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.012 </td>
+
+<td>0.611 / 0.657 / 10.977 / 0.294 </td>
+
+<td>7178 / 7209 / 7239 / 8 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>28 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.013 </td>
+
+<td>0.611 / 0.636 / 12.958 / 0.319 </td>
+
+<td>7177 / 7211 / 7240 / 8 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>28 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.012 </td>
+
+<td>0.612 / 0.646 / 13.818 / 0.327 </td>
+
+<td>7185 / 7218 / 7248 / 9 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>28 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.012 </td>
+
+<td>0.619 / 0.662 / 11.359 / 0.279 </td>
+
+<td>7258 / 7305 / 7354 / 13 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>28 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.012 </td>
+
+<td>0.609 / 0.648 / 12.260 / 0.300 </td>
+
+<td>8643 / 8687 / 8732 / 12 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>32 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.267 / 175.933 / 18.415 </td>
+
+<td>0.508 / 0.529 / 11.408 / 0.321 </td>
+
+<td>224 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>32 </td>
+
+<td>8 </td>
+
+<td>0.000 / 14.667 / 29.200 / 4.897 </td>
+
+<td>0.529 / 0.563 / 11.784 / 0.278 </td>
+
+<td>393 / 404 / 416 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>32 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.012 </td>
+
+<td>0.611 / 0.668 / 13.335 / 0.342 </td>
+
+<td>7226 / 7252 / 7273 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>32 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.012 </td>
+
+<td>0.611 / 0.655 / 19.309 / 0.454 </td>
+
+<td>7230 / 7254 / 7276 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>32 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.012 </td>
+
+<td>0.612 / 0.643 / 12.975 / 0.314 </td>
+
+<td>7231 / 7256 / 7277 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>32 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.067 / 0.013 </td>
+
+<td>0.612 / 0.666 / 12.496 / 0.357 </td>
+
+<td>7236 / 7262 / 7286 / 7 </td>
+ </tr>
+
+<tr class="b">
+
+<td>1500 </td>
+
+<td>32 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.013 </td>
+
+<td>0.625 / 0.657 / 21.381 / 0.401 </td>
+
+<td>7305 / 7347 / 7383 / 11 </td>
+ </tr>
+
+<tr class="a">
+
+<td>1500 </td>
+
+<td>32 </td>
+
+<td>32 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.012 </td>
+
+<td>0.609 / 0.652 / 19.148 / 0.387 </td>
+
+<td>8643 / 8687 / 8724 / 12 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>4 </td>
+
+<td>4 </td>
+
+<td>0.050 / 17.950 / 200.300 / 17.701 </td>
+
+<td>0.678 / 0.712 / 13.274 / 0.370 </td>
+
+<td>220 / 220 / 220 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>8 </td>
+
+<td>4 </td>
+
+<td>0.050 / 17.200 / 227.450 / 17.112 </td>
+
+<td>0.678 / 0.706 / 12.428 / 0.357 </td>
+
+<td>221 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>8 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.150 / 28.100 / 3.687 </td>
+
+<td>0.705 / 0.746 / 11.845 / 0.297 </td>
+
+<td>397 / 408 / 418 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>12 </td>
+
+<td>4 </td>
+
+<td>0.050 / 17.075 / 166.300 / 17.289 </td>
+
+<td>0.679 / 0.746 / 19.810 / 0.436 </td>
+
+<td>221 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>12 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.050 / 26.000 / 3.697 </td>
+
+<td>0.700 / 0.772 / 19.166 / 0.385 </td>
+
+<td>396 / 408 / 419 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>12 </td>
+
+<td>12 </td>
+
+<td>0.050 / 0.850 / 4.150 / 0.733 </td>
+
+<td>0.818 / 0.904 / 20.237 / 0.460 </td>
+
+<td>5109 / 5354 / 5633 / 103 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>16 </td>
+
+<td>4 </td>
+
+<td>0.050 / 17.650 / 196.200 / 17.539 </td>
+
+<td>0.678 / 0.747 / 17.211 / 0.435 </td>
+
+<td>222 / 222 / 222 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>16 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.100 / 23.650 / 3.720 </td>
+
+<td>0.699 / 0.770 / 17.887 / 0.378 </td>
+
+<td>397 / 409 / 420 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>16 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.200 / 1.100 / 0.163 </td>
+
+<td>0.835 / 0.915 / 24.825 / 0.515 </td>
+
+<td>6456 / 6547 / 6647 / 23 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>16 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.200 / 1.100 / 0.167 </td>
+
+<td>0.820 / 0.906 / 15.149 / 0.427 </td>
+
+<td>8177 / 8278 / 8384 / 27 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>20 </td>
+
+<td>4 </td>
+
+<td>0.050 / 17.650 / 184.550 / 17.468 </td>
+
+<td>0.678 / 0.742 / 19.468 / 0.449 </td>
+
+<td>223 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>20 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.150 / 27.500 / 3.693 </td>
+
+<td>0.700 / 0.756 / 12.525 / 0.289 </td>
+
+<td>399 / 410 / 420 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>20 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.050 / 0.350 / 0.044 </td>
+
+<td>0.830 / 0.921 / 19.963 / 0.553 </td>
+
+<td>7551 / 7643 / 7742 / 23 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>20 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.050 / 0.300 / 0.044 </td>
+
+<td>0.840 / 0.916 / 16.698 / 0.448 </td>
+
+<td>7615 / 7702 / 7789 / 23 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>20 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.050 / 0.300 / 0.044 </td>
+
+<td>0.824 / 0.909 / 23.619 / 0.600 </td>
+
+<td>8602 / 8633 / 8652 / 7 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>24 </td>
+
+<td>4 </td>
+
+<td>0.050 / 17.600 / 249.350 / 17.740 </td>
+
+<td>0.675 / 0.754 / 18.418 / 0.431 </td>
+
+<td>223 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>24 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.150 / 24.700 / 3.663 </td>
+
+<td>0.697 / 0.753 / 24.521 / 0.466 </td>
+
+<td>398 / 410 / 421 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>24 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.150 / 0.017 </td>
+
+<td>0.825 / 0.907 / 16.320 / 0.373 </td>
+
+<td>8559 / 8591 / 8621 / 8 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>24 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.017 </td>
+
+<td>0.826 / 0.876 / 20.177 / 0.403 </td>
+
+<td>8567 / 8600 / 8630 / 9 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>24 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.017 </td>
+
+<td>0.836 / 0.891 / 18.865 / 0.429 </td>
+
+<td>8670 / 8715 / 8762 / 13 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>24 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.150 / 0.017 </td>
+
+<td>0.823 / 0.874 / 13.270 / 0.398 </td>
+
+<td>10491 / 10531 / 10569 / 11 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>28 </td>
+
+<td>4 </td>
+
+<td>0.050 / 17.600 / 206.800 / 17.750 </td>
+
+<td>0.675 / 0.726 / 18.063 / 0.424 </td>
+
+<td>224 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>28 </td>
+
+<td>8 </td>
+
+<td>0.000 / 6.050 / 23.850 / 4.359 </td>
+
+<td>0.698 / 0.752 / 24.616 / 0.531 </td>
+
+<td>400 / 411 / 421 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>28 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.013 </td>
+
+<td>0.826 / 0.879 / 16.497 / 0.458 </td>
+
+<td>8600 / 8623 / 8641 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>28 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.012 </td>
+
+<td>0.826 / 0.898 / 16.948 / 0.417 </td>
+
+<td>8602 / 8626 / 8645 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>28 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.150 / 0.013 </td>
+
+<td>0.826 / 0.876 / 21.015 / 0.435 </td>
+
+<td>8609 / 8634 / 8656 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>28 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.012 </td>
+
+<td>0.836 / 0.878 / 19.773 / 0.462 </td>
+
+<td>8704 / 8746 / 8791 / 12 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>28 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.012 </td>
+
+<td>0.823 / 0.877 / 14.429 / 0.366 </td>
+
+<td>10489 / 10535 / 10573 / 11 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>32 </td>
+
+<td>4 </td>
+
+<td>0.050 / 18.300 / 180.850 / 18.635 </td>
+
+<td>0.676 / 0.723 / 18.331 / 0.431 </td>
+
+<td>224 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>32 </td>
+
+<td>8 </td>
+
+<td>0.000 / 10.900 / 26.400 / 5.122 </td>
+
+<td>0.698 / 0.742 / 18.170 / 0.443 </td>
+
+<td>397 / 409 / 420 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>32 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.013 </td>
+
+<td>0.825 / 0.869 / 12.927 / 0.376 </td>
+
+<td>8603 / 8623 / 8642 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>32 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.012 </td>
+
+<td>0.826 / 0.876 / 12.052 / 0.376 </td>
+
+<td>8601 / 8625 / 8644 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>32 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.012 </td>
+
+<td>0.824 / 0.883 / 19.872 / 0.492 </td>
+
+<td>8602 / 8627 / 8644 / 6 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>32 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.012 </td>
+
+<td>0.826 / 0.879 / 21.842 / 0.509 </td>
+
+<td>8610 / 8635 / 8654 / 6 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2000 </td>
+
+<td>32 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.150 / 0.012 </td>
+
+<td>0.836 / 0.894 / 18.071 / 0.448 </td>
+
+<td>8707 / 8748 / 8791 / 12 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2000 </td>
+
+<td>32 </td>
+
+<td>32 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.012 </td>
+
+<td>0.823 / 0.870 / 13.917 / 0.398 </td>
+
+<td>10491 / 10535 / 10575 / 11 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>4 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.480 / 215.840 / 18.035 </td>
+
+<td>0.844 / 0.896 / 17.723 / 0.444 </td>
+
+<td>220 / 220 / 220 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>8 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.960 / 369.960 / 18.503 </td>
+
+<td>0.844 / 0.897 / 20.260 / 0.542 </td>
+
+<td>221 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>8 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.080 / 23.280 / 3.639 </td>
+
+<td>0.874 / 0.928 / 21.505 / 0.472 </td>
+
+<td>399 / 412 / 420 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>12 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.520 / 204.760 / 17.973 </td>
+
+<td>0.845 / 0.963 / 21.008 / 0.499 </td>
+
+<td>221 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>12 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.080 / 30.120 / 3.683 </td>
+
+<td>0.869 / 0.946 / 18.277 / 0.488 </td>
+
+<td>400 / 412 / 421 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>12 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.880 / 4.600 / 0.747 </td>
+
+<td>1.025 / 1.134 / 20.157 / 0.532 </td>
+
+<td>6140 / 6934 / 7154 / 78 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>16 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.240 / 173.400 / 17.729 </td>
+
+<td>0.845 / 0.902 / 15.768 / 0.421 </td>
+
+<td>222 / 222 / 222 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>16 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.160 / 25.520 / 3.846 </td>
+
+<td>0.868 / 0.929 / 11.497 / 0.362 </td>
+
+<td>401 / 413 / 422 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>16 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.200 / 1.080 / 0.164 </td>
+
+<td>1.048 / 1.156 / 20.371 / 0.514 </td>
+
+<td>8704 / 8924 / 9057 / 33 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>16 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.200 / 1.000 / 0.166 </td>
+
+<td>1.031 / 1.108 / 13.336 / 0.432 </td>
+
+<td>11159 / 11486 / 11633 / 42 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>20 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.600 / 154.600 / 17.373 </td>
+
+<td>0.845 / 0.951 / 14.707 / 0.424 </td>
+
+<td>223 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>20 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.200 / 23.720 / 3.781 </td>
+
+<td>0.868 / 0.932 / 20.062 / 0.477 </td>
+
+<td>397 / 414 / 424 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>20 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.040 / 0.320 / 0.044 </td>
+
+<td>1.032 / 1.119 / 17.855 / 0.560 </td>
+
+<td>10273 / 10533 / 10639 / 33 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>20 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.040 / 0.320 / 0.043 </td>
+
+<td>1.045 / 1.107 / 16.834 / 0.461 </td>
+
+<td>10370 / 10621 / 10730 / 33 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>20 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.040 / 0.400 / 0.044 </td>
+
+<td>1.028 / 1.087 / 22.330 / 0.536 </td>
+
+<td>11718 / 11995 / 12020 / 12 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>24 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.480 / 156.160 / 17.171 </td>
+
+<td>0.844 / 0.890 / 18.096 / 0.406 </td>
+
+<td>223 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>24 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.140 / 22.280 / 3.716 </td>
+
+<td>0.868 / 0.915 / 19.031 / 0.458 </td>
+
+<td>403 / 414 / 423 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>24 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.160 / 0.017 </td>
+
+<td>1.030 / 1.088 / 22.283 / 0.500 </td>
+
+<td>11894 / 11934 / 11971 / 11 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>24 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.120 / 0.017 </td>
+
+<td>1.032 / 1.090 / 18.220 / 0.547 </td>
+
+<td>11896 / 11947 / 11990 / 12 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>24 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.120 / 0.018 </td>
+
+<td>1.043 / 1.101 / 17.194 / 0.516 </td>
+
+<td>12056 / 12115 / 12187 / 18 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>24 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.120 / 0.018 </td>
+
+<td>1.027 / 1.098 / 21.530 / 0.547 </td>
+
+<td>14736 / 14798 / 14849 / 15 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>28 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.720 / 199.480 / 17.641 </td>
+
+<td>0.845 / 0.900 / 20.585 / 0.511 </td>
+
+<td>224 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>28 </td>
+
+<td>8 </td>
+
+<td>0.000 / 5.200 / 22.920 / 4.106 </td>
+
+<td>0.869 / 0.914 / 12.212 / 0.359 </td>
+
+<td>403 / 414 / 425 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>28 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.080 / 0.012 </td>
+
+<td>1.031 / 1.085 / 17.978 / 0.500 </td>
+
+<td>12267 / 12329 / 12367 / 11 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>28 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.120 / 0.013 </td>
+
+<td>1.030 / 1.094 / 18.134 / 0.519 </td>
+
+<td>11986 / 12331 / 12372 / 13 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>28 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.120 / 0.011 </td>
+
+<td>1.031 / 1.086 / 20.068 / 0.550 </td>
+
+<td>12296 / 12341 / 12382 / 12 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>28 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.080 / 0.012 </td>
+
+<td>1.043 / 1.121 / 21.095 / 0.587 </td>
+
+<td>12117 / 12486 / 12547 / 17 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>28 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.080 / 0.012 </td>
+
+<td>1.028 / 1.097 / 20.162 / 0.573 </td>
+
+<td>14747 / 14802 / 14864 / 15 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>32 </td>
+
+<td>4 </td>
+
+<td>0.000 / 18.200 / 187.680 / 18.797 </td>
+
+<td>0.844 / 0.889 / 20.143 / 0.460 </td>
+
+<td>224 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>32 </td>
+
+<td>8 </td>
+
+<td>0.000 / 8.680 / 25.600 / 4.951 </td>
+
+<td>0.868 / 0.923 / 20.998 / 0.525 </td>
+
+<td>402 / 413 / 422 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>32 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.080 / 0.012 </td>
+
+<td>1.030 / 1.130 / 24.624 / 0.666 </td>
+
+<td>12378 / 12414 / 12443 / 8 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>32 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.080 / 0.012 </td>
+
+<td>1.030 / 1.090 / 17.364 / 0.532 </td>
+
+<td>12381 / 12416 / 12443 / 8 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>32 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.120 / 0.012 </td>
+
+<td>1.029 / 1.152 / 27.049 / 0.647 </td>
+
+<td>12386 / 12418 / 12444 / 8 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>32 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.080 / 0.013 </td>
+
+<td>1.031 / 1.083 / 26.018 / 0.575 </td>
+
+<td>12390 / 12427 / 12460 / 9 </td>
+ </tr>
+
+<tr class="b">
+
+<td>2500 </td>
+
+<td>32 </td>
+
+<td>28 </td>
+
+<td>0.000 / 0.000 / 0.120 / 0.013 </td>
+
+<td>1.042 / 1.102 / 18.739 / 0.486 </td>
+
+<td>12513 / 12568 / 12621 / 14 </td>
+ </tr>
+
+<tr class="a">
+
+<td>2500 </td>
+
+<td>32 </td>
+
+<td>32 </td>
+
+<td>0.000 / 0.000 / 0.120 / 0.012 </td>
+
+<td>1.027 / 1.088 / 16.996 / 0.499 </td>
+
+<td>14742 / 14802 / 14851 / 15 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>4 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.833 / 186.867 / 17.498 </td>
+
+<td>1.013 / 1.080 / 18.433 / 0.461 </td>
+
+<td>220 / 220 / 220 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>8 </td>
+
+<td>4 </td>
+
+<td>0.033 / 17.600 / 201.567 / 17.855 </td>
+
+<td>1.013 / 1.074 / 19.563 / 0.528 </td>
+
+<td>221 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>8 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.300 / 26.467 / 3.786 </td>
+
+<td>1.042 / 1.101 / 13.911 / 0.435 </td>
+
+<td>403 / 414 / 422 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>12 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.467 / 212.600 / 18.720 </td>
+
+<td>1.014 / 1.070 / 21.257 / 0.509 </td>
+
+<td>221 / 221 / 221 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>12 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.167 / 30.600 / 3.794 </td>
+
+<td>1.039 / 1.104 / 20.074 / 0.553 </td>
+
+<td>405 / 414 / 422 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>12 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.833 / 4.833 / 0.745 </td>
+
+<td>1.233 / 1.322 / 20.558 / 0.638 </td>
+
+<td>7453 / 7706 / 7923 / 67 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>16 </td>
+
+<td>4 </td>
+
+<td>0.000 / 18.117 / 152.933 / 17.161 </td>
+
+<td>1.013 / 1.101 / 21.535 / 0.564 </td>
+
+<td>222 / 222 / 222 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>16 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.200 / 25.633 / 3.817 </td>
+
+<td>1.037 / 1.100 / 18.806 / 0.554 </td>
+
+<td>405 / 415 / 423 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>16 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.200 / 1.033 / 0.169 </td>
+
+<td>1.258 / 1.331 / 22.352 / 0.682 </td>
+
+<td>10292 / 10439 / 10570 / 37 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>16 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.200 / 0.967 / 0.166 </td>
+
+<td>1.238 / 1.317 / 18.279 / 0.589 </td>
+
+<td>13313 / 13458 / 13612 / 43 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>20 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.467 / 164.967 / 17.757 </td>
+
+<td>1.013 / 1.113 / 19.652 / 0.508 </td>
+
+<td>223 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>20 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.167 / 24.267 / 3.743 </td>
+
+<td>1.038 / 1.092 / 18.642 / 0.554 </td>
+
+<td>405 / 416 / 425 / 3 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>20 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.033 / 0.400 / 0.045 </td>
+
+<td>1.240 / 1.303 / 14.404 / 0.517 </td>
+
+<td>12293 / 12419 / 12571 / 35 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>20 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.033 / 0.300 / 0.045 </td>
+
+<td>1.256 / 1.327 / 17.597 / 0.537 </td>
+
+<td>12406 / 12522 / 12655 / 35 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>20 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.033 / 0.333 / 0.044 </td>
+
+<td>1.237 / 1.308 / 20.509 / 0.558 </td>
+
+<td>14100 / 14156 / 14250 / 11 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>24 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.417 / 215.733 / 18.149 </td>
+
+<td>1.013 / 1.091 / 21.116 / 0.517 </td>
+
+<td>223 / 223 / 223 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>24 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.233 / 21.767 / 3.773 </td>
+
+<td>1.036 / 1.092 / 11.990 / 0.434 </td>
+
+<td>406 / 416 / 426 / 3 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>24 </td>
+
+<td>12 </td>
+
+<td>0.000 / 0.000 / 0.100 / 0.017 </td>
+
+<td>1.239 / 1.343 / 19.689 / 0.541 </td>
+
+<td>14036 / 14088 / 14126 / 12 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>24 </td>
+
+<td>16 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.017 </td>
+
+<td>1.239 / 1.311 / 14.029 / 0.532 </td>
+
+<td>14050 / 14102 / 14146 / 13 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>24 </td>
+
+<td>20 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.017 </td>
+
+<td>1.255 / 1.327 / 25.879 / 0.611 </td>
+
+<td>14218 / 14302 / 14381 / 20 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>24 </td>
+
+<td>24 </td>
+
+<td>0.000 / 0.000 / 0.133 / 0.017 </td>
+
+<td>1.233 / 1.299 / 20.804 / 0.615 </td>
+
+<td>17434 / 17491 / 17548 / 17 </td>
+ </tr>
+
+<tr class="a">
+
+<td>3000 </td>
+
+<td>28 </td>
+
+<td>4 </td>
+
+<td>0.000 / 17.467 / 160.633 / 17.603 </td>
+
+<td>1.014 / 1.069 / 15.224 / 0.466 </td>
+
+<td>224 / 224 / 224 / 0 </td>
+ </tr>
+
+<tr class="b">
+
+<td>3000 </td>
+
+<td>28 </td>
+
+<td>8 </td>
+
+<td>0.000 / 4.733 / 22.600 / 3.937 </td>
+
+<td>1.037 / 1.099 / 17.982 / 0.528 </td>
+
+<td>407 / 417 / 426 / 3 </td>
+ </tr>
+
+<tr
<TRUNCATED>
[03/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-platform/metron-data-management/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-data-management/index.html b/site/current-book/metron-platform/metron-data-management/index.html
new file mode 100644
index 0000000..8029ef2
--- /dev/null
+++ b/site/current-book/metron-platform/metron-data-management/index.html
@@ -0,0 +1,1014 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Resource Data Management</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Resource Data Management</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-down"></i>
+ Platform</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-platform/metron-api/index.html" title="Api">
+ <i class="none"></i>
+ Api</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-common/index.html" title="Common">
+ <i class="none"></i>
+ Common</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Data-management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
+ <i class="none"></i>
+ Enrichment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
+ <i class="none"></i>
+ Indexing</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-management/index.html" title="Management">
+ <i class="none"></i>
+ Management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
+ <i class="none"></i>
+ Parsers</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
+ <i class="none"></i>
+ Pcap-backend</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Resource Data Management</h1>
+<p><a name="Resource_Data_Management"></a></p>
+<p>This project is a collection of classes to assist with loading of various enrichment and threat intelligence sources into Metron.</p>
+<div class="section">
+<h2><a name="Simple_HBase_EnrichmentsThreat_Intelligence"></a>Simple HBase Enrichments/Threat Intelligence</h2>
+<p>The vast majority of enrichments and threat intelligence processing tend toward the following pattern:</p>
+
+<ul>
+
+<li>Take a field</li>
+
+<li>Look up the field in a key/value store</li>
+
+<li>If the key exists, then either it’s a threat to be alerted or it should be enriched with the value associated with the key.</li>
+</ul>
+<p>As such, we have created this capability as a default threat intel and enrichment adapter. The basic primitive for simple enrichments and threat intelligence sources is a complex key containing the following:</p>
+
+<ul>
+
+<li>Type : The type of threat intel or enrichment (e.g. malicious_ip)</li>
+
+<li>Indicator : The indicator in question</li>
+
+<li>Value : The value to associate with the type, indicator pair. This is a JSON map.</li>
+</ul>
+<p>At present, all of the dataloads utilities function by converting raw data sources to this primitive key (type, indicator) and value to be placed in HBase.</p>
+<p>In the case of threat intel, a hit on the threat intel table will result in:</p>
+
+<ul>
+
+<li>The <tt>is_alert</tt> field being set to <tt>true</tt> in the index</li>
+
+<li>A field named <tt>threatintels.hbaseThreatIntel.$field.$threatintel_type</tt> is set to <tt>alert</tt>
+
+<ul>
+
+<li><tt>$field</tt> is the field in the original document that was a match (e.g. <tt>src_ip_addr</tt>)</li>
+
+<li><tt>$threatintel_type</tt> is the type of threat intel imported (defined in the Extractor configuration below).</li>
+ </ul></li>
+</ul>
+<p>In the case of simple hbase enrichment, a hit on the enrichments table will result in the following new field for each key in the value:<tt>enrichments.hbaseEnrichment.$field.$enrichment_type.$key</tt> </p>
+
+<ul>
+
+<li><tt>$field</tt> is the field in the original document that was a match (e.g. <tt>src_ip_addr</tt>)</li>
+
+<li><tt>$enrichment_type</tt> is the type of enrichment imported (defined in the Extractor configuration below).</li>
+
+<li><tt>$key</tt> is a key in the JSON map associated with the row in HBase.</li>
+</ul>
+<p>For instance, in the situation where we had the following very silly key/value in HBase in the enrichment table:</p>
+
+<ul>
+
+<li>indicator: <tt>127.0.0.1</tt></li>
+
+<li>type : <tt>important_addresses</tt></li>
+
+<li>value: <tt>{ "name" : "localhost", "location" : "home" }</tt></li>
+</ul>
+<p>If we had a document whose <tt>ip_src_addr</tt> came through with a value of <tt>127.0.0.1</tt>, we would have the following fields added to the indexed document:</p>
+
+<ul>
+
+<li><tt>enrichments.hbaseEnrichment.ip_src_addr.important_addresses.name</tt> : <tt>localhost</tt></li>
+
+<li><tt>enrichments.hbaseEnrichment.ip_src_addr.important_addresses.location</tt> : <tt>home</tt></li>
+</ul></div>
+<div class="section">
+<h2><a name="Extractor_Framework"></a>Extractor Framework</h2>
+<p>For the purpose of ingesting data of a variety of formats, we have created an Extractor framework which allows for common data formats to be interpreted as enrichment or threat intelligence sources. The formats supported at present are:</p>
+
+<ul>
+
+<li>CSV (both threat intel and enrichment)</li>
+
+<li>STIX (threat intel only)</li>
+
+<li>Custom (pass your own class)</li>
+</ul>
+<p>All of the current utilities take a JSON file to configure how to interpret input data. This JSON describes the type of data and the schema if necessary for the data if it is not fixed (as in STIX, e.g.).</p>
+<div class="section">
+<h3><a name="CSV_Extractor"></a>CSV Extractor</h3>
+<p>Consider the following example configuration file which describes how to process a CSV file.</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "config" : {
+ "columns" : {
+ "ip" : 0
+ ,"source" : 2
+ }
+ ,"indicator_column" : "ip"
+ ,"type" : "malicious_ip"
+ ,"separator" : ","
+ }
+ ,"extractor" : "CSV"
+}
+</pre></div></div>
+<p>In this example, we have instructed the extractor of the schema (i.e. the columns field), two columns at the first and third position. We have indicated that the <tt>ip</tt> column is the indicator type and that the enrichment type is named <tt>malicious_ip</tt>. We have also indicated that the extractor to use is the CSV Extractor. The other option is the STIX extractor or a fully qualified classname for your own extractor.</p>
+<p>The meta column values will show up in the value in HBase because it is called out as a non-indicator column. The key for the value will be ‘meta’. For instance, given an input string of <tt>123.45.123.12,something,the grapevine</tt>, the following key, value would be extracted:</p>
+
+<ul>
+
+<li>Indicator : <tt>123.45.123.12</tt></li>
+
+<li>Type : <tt>malicious_ip</tt></li>
+
+<li>Value : <tt>{ "ip" : "123.45.123.12", "source" : "the grapevine" }</tt></li>
+</ul></div>
+<div class="section">
+<h3><a name="STIX_Extractor"></a>STIX Extractor</h3>
+<p>Consider the following config for importing STIX documents. This is a threat intelligence interchange format, so it is particularly relevant and attractive data to import for our purposes. Because STIX is a standard format, there is no need to specify the schema or how to interpret the documents.</p>
+<p>We support a subset of STIX messages for importation:</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>STIX Type </th>
+
+<th>Specific Type </th>
+
+<th>Enrichment Type Name </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>Address </td>
+
+<td>IPV_4_ADDR </td>
+
+<td>address:IPV_4_ADDR </td>
+ </tr>
+
+<tr class="a">
+
+<td>Address </td>
+
+<td>IPV_6_ADDR </td>
+
+<td>address:IPV_6_ADDR </td>
+ </tr>
+
+<tr class="b">
+
+<td>Address </td>
+
+<td>E_MAIL </td>
+
+<td>address:E_MAIL </td>
+ </tr>
+
+<tr class="a">
+
+<td>Address </td>
+
+<td>MAC </td>
+
+<td>address:MAC </td>
+ </tr>
+
+<tr class="b">
+
+<td>Domain </td>
+
+<td>FQDN </td>
+
+<td>domain:FQDN </td>
+ </tr>
+
+<tr class="a">
+
+<td>Hostname </td>
+
+<td> </td>
+
+<td>hostname </td>
+ </tr>
+ </tbody>
+</table>
+<p>NOTE: The enrichment type will be used as the type above.</p>
+<p>Consider the following configuration for an Extractor</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "config" : {
+ "stix_address_categories" : "IPV_4_ADDR"
+ }
+ ,"extractor" : "STIX"
+}
+</pre></div></div>
+<p>In here, we’re configuring the STIX extractor to load from a series of STIX files, however we only want to bring in IPv4 addresses from the set of all possible addresses. Note that if no categories are specified for import, all are assumed. Also, only address and domain types allow filtering via <tt>stix_address_categories</tt> and <tt>stix_domain_categories</tt> config parameters.</p></div>
+<div class="section">
+<h3><a name="Common_Extractor_Properties"></a>Common Extractor Properties</h3>
+<p>Users also have the ability to transform and filter enrichment and threat intel data using Stellar as it is loaded into HBase. This feature is available to all extractor types.</p>
+<p>As an example, we will be providing a CSV list of top domains as an enrichment and filtering the value metadata, as well as the indicator column, with Stellar expressions.</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "config" : {
+ "zk_quorum" : "node1:2181",
+ "columns" : {
+ "rank" : 0,
+ "domain" : 1
+ },
+ "value_transform" : {
+ "domain" : "DOMAIN_REMOVE_TLD(domain)"
+ },
+ "value_filter" : "LENGTH(domain) > 0",
+ "indicator_column" : "domain",
+ "indicator_transform" : {
+ "indicator" : "DOMAIN_REMOVE_TLD(indicator)"
+ },
+ "indicator_filter" : "LENGTH(indicator) > 0",
+ "type" : "top_domains",
+ "separator" : ","
+ },
+ "extractor" : "CSV"
+}
+</pre></div></div>
+<p>There are 2 property maps that work with full Stellar expressions, and 2 properties that will work with Stellar predicates.</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Property </th>
+
+<th>Description </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>value_transform </td>
+
+<td>Transform fields defined in the “columns” mapping with Stellar transformations. New keys introduced in the transform will be added to the key metadata. </td>
+ </tr>
+
+<tr class="a">
+
+<td>value_filter </td>
+
+<td>Allows additional filtering with Stellar predicates based on results from the value transformations. In this example, records whose domain property is empty after removing the TLD will be omitted. </td>
+ </tr>
+
+<tr class="b">
+
+<td>indicator_transform </td>
+
+<td>Transform the indicator column independent of the value transformations. You can refer to the original indicator value by using “indicator” as the variable name, as shown in the example above. In addition, if you prefer to piggyback your transformations, you can refer to the variable “domain”, which will allow your indicator transforms to inherit transformations done to this value during the value transformations. </td>
+ </tr>
+
+<tr class="a">
+
+<td>indicator_filter </td>
+
+<td>Allows additional filtering with Stellar predicates based on results from the value transformations. In this example, records whose indicator value is empty after removing the TLD will be omitted. </td>
+ </tr>
+ </tbody>
+</table>
+<p>top-list.csv</p>
+
+<div class="source">
+<div class="source">
+<pre>1,google.com
+2,youtube.com
+...
+</pre></div></div>
+<p>Running a file import with the above data and extractor configuration would result in the following 2 extracted data records:</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Indicator </th>
+
+<th>Type </th>
+
+<th>Value </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>google </td>
+
+<td>top_domains </td>
+
+<td>{ “rank” : “1”, “domain” : “google” } </td>
+ </tr>
+
+<tr class="a">
+
+<td>yahoo </td>
+
+<td>top_domains </td>
+
+<td>{ “rank” : “2”, “domain” : “yahoo” } </td>
+ </tr>
+ </tbody>
+</table>
+<p>Similar to the parser framework, providing a Zookeeper quorum via the zk_quorum property will enable Stellar to access properties that reside in the global config. Expanding on our example above, if the global config looks as follows:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "global_property" : "metron-ftw"
+}
+</pre></div></div>
+<p>And we expand our value_tranform:</p>
+
+<div class="source">
+<div class="source">
+<pre>...
+ "value_transform" : {
+ "domain" : "DOMAIN_REMOVE_TLD(domain)",
+ "a-new-prop" : "global_property"
+ },
+...
+
+</pre></div></div>
+<p>The resulting value data would look like the following:</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Indicator </th>
+
+<th>Type </th>
+
+<th>Value </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>google </td>
+
+<td>top_domains </td>
+
+<td>{ “rank” : “1”, “domain” : “google”, “a-new-prop” : “metron-ftw” } </td>
+ </tr>
+
+<tr class="a">
+
+<td>yahoo </td>
+
+<td>top_domains </td>
+
+<td>{ “rank” : “2”, “domain” : “yahoo”, “a-new-prop” : “metron-ftw” } </td>
+ </tr>
+ </tbody>
+</table></div></div>
+<div class="section">
+<h2><a name="Enrichment_Config"></a>Enrichment Config</h2>
+<p>In order to automatically add new enrichment and threat intel types to existing, running enrichment topologies, you will need to add new fields and new types to the zookeeper configuration. A convenience parameter has been made to assist in this when doing an import. Namely, you can specify the enrichment configs and how they associate with the fields of the documents flowing through the enrichment topology.</p>
+<p>Consider the following Enrichment Configuration JSON. This one is for a threat intelligence type:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "zkQuorum" : "localhost:2181"
+ ,"sensorToFieldList" : {
+ "bro" : {
+ "type" : "THREAT_INTEL"
+ ,"fieldToEnrichmentTypes" : {
+ "ip_src_addr" : [ "malicious_ip" ]
+ ,"ip_dst_addr" : [ "malicious_ip" ]
+ }
+ }
+ }
+}
+</pre></div></div>
+<p>We have to specify the following:</p>
+
+<ul>
+
+<li>The zookeeper quorum which holds the cluster configuration</li>
+
+<li>The mapping between the fields in the enriched documents and the enrichment types.</li>
+</ul>
+<p>This configuration allows the ingestion tools to update zookeeper post-ingestion so that the enrichment topology can take advantage immediately of the new type.</p></div>
+<div class="section">
+<h2><a name="Loading_Utilities"></a>Loading Utilities</h2>
+<p>The two configurations above are used in the three separate ingestion tools:</p>
+
+<ul>
+
+<li>Taxii Loader</li>
+
+<li>Bulk load from HDFS via MapReduce</li>
+
+<li>Flat File ingestion</li>
+</ul>
+<div class="section">
+<h3><a name="Taxii_Loader"></a>Taxii Loader</h3>
+<p>The shell script <tt>$METRON_HOME/bin/threatintel_taxii_load.sh</tt> can be used to poll a Taxii server for STIX documents and ingest them into HBase.<br />It is quite common for this Taxii server to be an aggregation server such as Soltra Edge.</p>
+<p>In addition to the Enrichment and Extractor configs described above, this loader requires a configuration file describing the connection information to the Taxii server. An illustrative example of such a configuration file is:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "endpoint" : "http://localhost:8282/taxii-discovery-service"
+ ,"type" : "DISCOVER"
+ ,"collection" : "guest.Abuse_ch"
+ ,"table" : "threat_intel"
+ ,"columnFamily" : "cf"
+ ,"allowedIndicatorTypes" : [ "domainname:FQDN", "address:IPV_4_ADDR" ]
+}
+</pre></div></div>
+<p>As you can see, we are specifying the following information:</p>
+
+<ul>
+
+<li>endpoint : The URL of the endpoint</li>
+
+<li>type : <tt>POLL</tt> or <tt>DISCOVER</tt> depending on the endpoint.</li>
+
+<li>collection : The Taxii collection to ingest</li>
+
+<li>table : The HBase table to import into</li>
+
+<li>columnFamily : The column family to import into</li>
+
+<li>allowedIndicatorTypes : an array of acceptable threat intel types (see the “Enrichment Type Name” column of the Stix table above for the possibilities).</li>
+</ul>
+<p>The parameters for the utility are as follows:</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Short Code </th>
+
+<th>Long Code </th>
+
+<th>Is Required? </th>
+
+<th>Description </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>-h </td>
+
+<td> </td>
+
+<td>No </td>
+
+<td>Generate the help screen/set of options </td>
+ </tr>
+
+<tr class="a">
+
+<td>-e </td>
+
+<td>–extractor_config </td>
+
+<td>Yes </td>
+
+<td>JSON Document describing the extractor for this input data source </td>
+ </tr>
+
+<tr class="b">
+
+<td>-c </td>
+
+<td>–taxii_connection_config </td>
+
+<td>Yes </td>
+
+<td>The JSON config file to configure the connection </td>
+ </tr>
+
+<tr class="a">
+
+<td>-p </td>
+
+<td>–time_between_polls </td>
+
+<td>No </td>
+
+<td>The time between polling the Taxii server in milliseconds. (default: 1 hour) </td>
+ </tr>
+
+<tr class="b">
+
+<td>-b </td>
+
+<td>–begin_time </td>
+
+<td>No </td>
+
+<td>Start time to poll the Taxii server (all data from that point will be gathered in the first pull). The format for the date is yyyy-MM-dd HH:mm:ss </td>
+ </tr>
+
+<tr class="a">
+
+<td>-l </td>
+
+<td>–log4j </td>
+
+<td>No </td>
+
+<td>The Log4j Properties to load </td>
+ </tr>
+
+<tr class="b">
+
+<td>-n </td>
+
+<td>–enrichment_config </td>
+
+<td>No </td>
+
+<td>The JSON document describing the enrichments to configure. Unlike other loaders, this is run first if specified. </td>
+ </tr>
+ </tbody>
+</table></div>
+<div class="section">
+<h3><a name="Flatfile_Loader"></a>Flatfile Loader</h3>
+<p>The shell script <tt>$METRON_HOME/bin/flatfile_loader.sh</tt> will read data from local disk, HDFS or URLs and load the enrichment or threat intel data into an HBase table.<br />Note: This utility works for enrichment as well as threat intel due to the underlying infrastructure being the same.</p>
+<p>One special thing to note here is that there is a special configuration parameter to the Extractor config that is only considered during this loader:</p>
+
+<ul>
+
+<li>inputFormat : This specifies how to consider the data. The two implementations are <tt>BY_LINE</tt> and <tt>WHOLE_FILE</tt>.</li>
+</ul>
+<p>The default is <tt>BY_LINE</tt>, which makes sense for a list of CSVs where each line indicates a unit of information which can be imported. However, if you are importing a set of STIX documents, then you want each document to be considered as input to the Extractor.</p>
+<p>The parameters for the utility are as follows:</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Short Code </th>
+
+<th>Long Code </th>
+
+<th>Is Required? </th>
+
+<th>Description </th>
+
+<th> </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>-h </td>
+
+<td> </td>
+
+<td>No </td>
+
+<td>Generate the help screen/set of options </td>
+
+<td> </td>
+ </tr>
+
+<tr class="a">
+
+<td>-q </td>
+
+<td>–quiet </td>
+
+<td>No </td>
+
+<td>Do not update progress</td>
+ </tr>
+
+<tr class="b">
+
+<td>-e </td>
+
+<td>–extractor_config </td>
+
+<td>Yes </td>
+
+<td>JSON Document describing the extractor for this input data source </td>
+
+<td> </td>
+ </tr>
+
+<tr class="a">
+
+<td>-m </td>
+
+<td>–import_mode </td>
+
+<td>No </td>
+
+<td>The Import mode to use: LOCAL, MR. Default: LOCAL </td>
+
+<td> </td>
+ </tr>
+
+<tr class="b">
+
+<td>-t </td>
+
+<td>–hbase_table </td>
+
+<td>Yes </td>
+
+<td>The HBase table to import into </td>
+
+<td> </td>
+ </tr>
+
+<tr class="a">
+
+<td>-c </td>
+
+<td>–hbase_cf </td>
+
+<td>Yes </td>
+
+<td>The HBase table column family to import into </td>
+
+<td> </td>
+ </tr>
+
+<tr class="b">
+
+<td>-i </td>
+
+<td>–input </td>
+
+<td>Yes </td>
+
+<td>The input data location on local disk. If this is a file, then that file will be loaded. If this is a directory, then the files will be loaded recursively under that directory. </td>
+
+<td> </td>
+ </tr>
+
+<tr class="a">
+
+<td>-l </td>
+
+<td>–log4j </td>
+
+<td>No </td>
+
+<td>The log4j properties file to load </td>
+
+<td> </td>
+ </tr>
+
+<tr class="b">
+
+<td>-n </td>
+
+<td>–enrichment_config </td>
+
+<td>No </td>
+
+<td>The JSON document describing the enrichments to configure. Unlike other loaders, this is run first if specified. </td>
+
+<td> </td>
+ </tr>
+
+<tr class="a">
+
+<td>-p </td>
+
+<td>–threads </td>
+
+<td>No </td>
+
+<td>The number of threads to use when extracting data. The default is the number of cores. </td>
+
+<td> </td>
+ </tr>
+
+<tr class="b">
+
+<td>-b </td>
+
+<td>–batchSize </td>
+
+<td>No </td>
+
+<td>The batch size to use for HBase puts </td>
+
+<td> </td>
+ </tr>
+ </tbody>
+</table></div>
+<div class="section">
+<h3><a name="GeoLite2_Loader"></a>GeoLite2 Loader</h3>
+<p>The shell script <tt>$METRON_HOME/bin/geo_enrichment_load.sh</tt> will retrieve MaxMind GeoLite2 data and load data into HDFS, and update the configuration.</p>
+<p>THIS SCRIPT WILL NOT UPDATE AMBARI’S GLOBAL.JSON, JUST THE ZK CONFIGS. CHANGES WILL GO INTO EFFECT, BUT WILL NOT PERSIST PAST AN AMBARI RESTART UNTIL UPDATED THERE.</p>
+<p>The parameters for the utility are as follows:</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Short Code </th>
+
+<th>Long Code </th>
+
+<th>Is Required? </th>
+
+<th>Description </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td>-h </td>
+
+<td> </td>
+
+<td>No </td>
+
+<td>Generate the help screen/set of options </td>
+ </tr>
+
+<tr class="a">
+
+<td>-g </td>
+
+<td>–geo_url </td>
+
+<td>No </td>
+
+<td>GeoIP URL - defaults to <a class="externalLink" href="http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz">http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz</a></td>
+ </tr>
+
+<tr class="b">
+
+<td>-r </td>
+
+<td>–remote_dir </td>
+
+<td>No </td>
+
+<td>HDFS directory to land formatted GeoIP file - defaults to /apps/metron/geo/<epoch millis>/</td>
+ </tr>
+
+<tr class="a">
+
+<td>-t </td>
+
+<td>–tmp_dir </td>
+
+<td>No </td>
+
+<td>Directory for landing the temporary GeoIP data - defaults to /tmp</td>
+ </tr>
+
+<tr class="b">
+
+<td>-z </td>
+
+<td>–zk_quorum </td>
+
+<td>Yes </td>
+
+<td>Zookeeper Quorum URL (zk1:port,zk2:port,…)</td>
+ </tr>
+ </tbody>
+</table></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-platform/metron-enrichment/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-enrichment/index.html b/site/current-book/metron-platform/metron-enrichment/index.html
new file mode 100644
index 0000000..33615dd
--- /dev/null
+++ b/site/current-book/metron-platform/metron-enrichment/index.html
@@ -0,0 +1,670 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Enrichment</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Enrichment</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-down"></i>
+ Platform</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-platform/metron-api/index.html" title="Api">
+ <i class="none"></i>
+ Api</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-common/index.html" title="Common">
+ <i class="none"></i>
+ Common</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
+ <i class="none"></i>
+ Data-management</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Enrichment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
+ <i class="none"></i>
+ Indexing</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-management/index.html" title="Management">
+ <i class="none"></i>
+ Management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
+ <i class="none"></i>
+ Parsers</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
+ <i class="none"></i>
+ Pcap-backend</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Enrichment</h1>
+<p><a name="Enrichment"></a></p>
+<div class="section">
+<h2><a name="Introduction"></a>Introduction</h2>
+<p>The <tt>enrichment</tt> topology is a topology dedicated to taking the data from the parsing topologies that have been normalized into the Metron data format (e.g. a JSON Map structure with <tt>original_message</tt> and <tt>timestamp</tt>) and </p>
+
+<ul>
+
+<li>Enriching messages with external data from data stores (e.g. hbase) by adding new fields based on existing fields in the messages.</li>
+
+<li>Marking messages as threats based on data in external data stores</li>
+
+<li>Marking threat alerts with a numeric triage level based on a set of Stellar rules.</li>
+</ul></div>
+<div class="section">
+<h2><a name="Enrichment_Architecture"></a>Enrichment Architecture</h2>
+<p><img src="../../images/enrichment_arch.png" alt="Architecture" /></p></div>
+<div class="section">
+<h2><a name="Enrichment_Configuration"></a>Enrichment Configuration</h2>
+<p>The configuration for the <tt>enrichment</tt> topology, the topology primarily responsible for enrichment and threat intelligence enrichment, is defined by JSON documents stored in zookeeper.</p>
+<p>There are two types of configurations at the moment, <tt>global</tt> and <tt>sensor</tt> specific. </p></div>
+<div class="section">
+<h2><a name="Global_Configuration"></a>Global Configuration</h2>
+<p>See the “<a href="../metron-common/index.html">Global Configuration</a>” section.</p></div>
+<div class="section">
+<h2><a name="Sensor_Enrichment_Configuration"></a>Sensor Enrichment Configuration</h2>
+<p>The sensor specific configuration is intended to configure the individual enrichments and threat intelligence enrichments for a given sensor type (e.g. <tt>snort</tt>).</p>
+<p>Just like the global config, the format is a JSON stored in zookeeper. The configuration is a complex JSON object with the following top level fields:</p>
+
+<ul>
+
+<li><tt>enrichment</tt> : A complex JSON object representing the configuration of the enrichments</li>
+
+<li><tt>threatIntel</tt> : A complex JSON object representing the configuration of the threat intelligence enrichments</li>
+</ul>
+<div class="section">
+<h3><a name="The_enrichment_Configuration"></a>The <tt>enrichment</tt> Configuration</h3>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Field </th>
+
+<th>Description </th>
+
+<th>Example </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td><tt>fieldToTypeMap</tt> </td>
+
+<td>In the case of a simple HBase enrichment (i.e. a key/value lookup), the mapping between fields and the enrichment types associated with those fields must be known. This enrichment type is used as part of the HBase key. Note: applies to hbaseEnrichment only. </td>
+
+<td><tt>"fieldToTypeMap" : { "ip_src_addr" : [ "asset_enrichment" ] }</tt> </td>
+ </tr>
+
+<tr class="a">
+
+<td><tt>fieldMap</tt> </td>
+
+<td>The map of enrichment bolts names to configuration handlers which know how to split the message up. The simplest of which is just a list of fields. More complex examples would be the stellar enrichment which provides stellar statements. Each field listed in the array arg is sent to the enrichment referenced in the key. Cardinality of fields to enrichments is many-to-many. </td>
+
+<td><tt>"fieldMap": {"hbaseEnrichment": ["ip_src_addr","ip_dst_addr"]}</tt> </td>
+ </tr>
+
+<tr class="b">
+
+<td><tt>config</tt> </td>
+
+<td>The general configuration for the enrichment </td>
+
+<td><tt>"config": {"typeToColumnFamily": { "asset_enrichment" : "cf" } }</tt> </td>
+ </tr>
+ </tbody>
+</table>
+<p>The <tt>config</tt> map is intended to house enrichment specific configuration. For instance, for the <tt>hbaseEnrichment</tt>, the mappings between the enrichment types to the column families is specified.</p>
+<p>The <tt>fieldMap</tt>contents are of interest because they contain the routing and configuration information for the enrichments. When we say ‘routing’, we mean how the messages get split up and sent to the enrichment adapter bolts. The simplest, by far, is just providing a simple list as in</p>
+
+<div class="source">
+<div class="source">
+<pre> "fieldMap": {
+ "geo": [
+ "ip_src_addr",
+ "ip_dst_addr"
+ ],
+ "host": [
+ "ip_src_addr",
+ "ip_dst_addr"
+ ],
+ "hbaseEnrichment": [
+ "ip_src_addr",
+ "ip_dst_addr"
+ ]
+ }
+</pre></div></div>
+<p>Based on this sample config, both ip_src_addr and ip_dst_addr will go to the <tt>geo</tt>, <tt>host</tt>, and <tt>hbaseEnrichment</tt> adapter bolts. For the <tt>geo</tt>, <tt>host</tt> and <tt>hbaseEnrichment</tt>, this is sufficient. However, more complex enrichments may contain their own configuration. Currently, the <tt>stellar</tt> enrichment requires a more complex configuration, such as:</p>
+
+<div class="source">
+<div class="source">
+<pre> "fieldMap": {
+ ...
+ "stellar" : {
+ "config" : {
+ "numeric" : {
+ "foo": "1 + 1"
+ }
+ ,"ALL_CAPS" : "TO_UPPER(source.type)"
+ }
+ }
+ }
+</pre></div></div>
+<p>Whereas the simpler enrichments just need a set of fields explicitly stated so they can be separated from the message and sent to the enrichment adapter bolt for enrichment and ultimately joined back in the join bolt, the stellar enrichment has its set of required fields implicitly stated through usage. For instance, if your stellar statement references a field, it should be included and if not, then it should not be included. We did not want to require users to make explicit the implicit.</p>
+<p>The other way in which the stellar enrichment is somewhat more complex is in how the statements are executed. In the general purpose case for a list of fields, those fields are used to create a message to send to the enrichment adapter bolt and that bolt’s worker will handle the fields one by one in serial for a given message. For stellar enrichment, we wanted to have a more complex design so that users could specify the groups of stellar statements sent to the same worker in the same message (and thus executed sequentially). Consider the following configuration:</p>
+
+<div class="source">
+<div class="source">
+<pre> "fieldMap": {
+ "stellar" : {
+ "config" : {
+ "numeric" : {
+ "foo": "1 + 1"
+ "bar" : TO_LOWER(source.type)"
+ }
+ ,"text" : {
+ "ALL_CAPS" : "TO_UPPER(source.type)"
+ }
+ }
+ }
+ }
+</pre></div></div>
+<p>We have a group called <tt>numeric</tt> whose stellar statements will be executed sequentially. In parallel to that, we have the group of stellar statements under the group <tt>text</tt> executing. The intent here is to allow you to not force higher latency operations to be done sequentially. You can use any name for your groupings you like. Be aware that the configuration is a map and duplicate configuration keys’ values are not combined, so the duplicate configuration value will be overwritten.</p></div>
+<div class="section">
+<h3><a name="The_threatIntel_Configuration"></a>The <tt>threatIntel</tt> Configuration</h3>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Field </th>
+
+<th>Description </th>
+
+<th>Example </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td><tt>fieldToTypeMap</tt> </td>
+
+<td>In the case of a simple HBase threat intel enrichment (i.e. a key/value lookup), the mapping between fields and the enrichment types associated with those fields must be known. This enrichment type is used as part of the HBase key. Note: applies to hbaseThreatIntel only. </td>
+
+<td><tt>"fieldToTypeMap" : { "ip_src_addr" : [ "malicious_ips" ] }</tt> </td>
+ </tr>
+
+<tr class="a">
+
+<td><tt>fieldMap</tt> </td>
+
+<td>The map of threat intel enrichment bolts names to fields in the JSON messages. Each field is sent to the threat intel enrichment bolt referenced in the key. Each field listed in the array arg is sent to the enrichment referenced in the key. Cardinality of fields to enrichments is many-to-many. </td>
+
+<td><tt>"fieldMap": {"hbaseThreatIntel": ["ip_src_addr","ip_dst_addr"]}</tt> </td>
+ </tr>
+
+<tr class="b">
+
+<td><tt>triageConfig</tt> </td>
+
+<td>The configuration of the threat triage scorer. In the situation where a threat is detected, a score is assigned to the message and embedded in the indexed message. </td>
+
+<td><tt>"riskLevelRules" : { "IN_SUBNET(ip_dst_addr, '192.168.0.0/24')" : 10 }</tt> </td>
+ </tr>
+
+<tr class="a">
+
+<td><tt>config</tt> </td>
+
+<td>The general configuration for the Threat Intel </td>
+
+<td><tt>"config": {"typeToColumnFamily": { "malicious_ips","cf" } }</tt> </td>
+ </tr>
+ </tbody>
+</table>
+<p>The <tt>config</tt> map is intended to house threat intel specific configuration. For instance, for the <tt>hbaseThreatIntel</tt> threat intel adapter, the mappings between the enrichment types to the column families is specified.</p>
+<p>The <tt>triageConfig</tt> field is also a complex field and it bears some description:</p>
+
+<table border="0" class="table table-striped">
+ <thead>
+
+<tr class="a">
+
+<th>Field </th>
+
+<th>Description </th>
+
+<th>Example </th>
+ </tr>
+ </thead>
+ <tbody>
+
+<tr class="b">
+
+<td><tt>riskLevelRules</tt> </td>
+
+<td>This is a list of rules (represented as Stellar expressions) associated with scores with optional names and comments </td>
+
+<td>see below</td>
+ </tr>
+
+<tr class="a">
+
+<td><tt>aggregator</tt> </td>
+
+<td>An aggregation function that takes all non-zero scores representing the matching queries from <tt>riskLevelRules</tt> and aggregates them into a single score. </td>
+
+<td><tt>"MAX"</tt> </td>
+ </tr>
+ </tbody>
+</table>
+<p>A risk level rule is of the following format:</p>
+
+<ul>
+
+<li><tt>name</tt> : The name of the threat triage rule</li>
+
+<li><tt>comment</tt> : A comment describing the rule</li>
+
+<li><tt>rule</tt> : The rule, represented as a Stellar statement</li>
+
+<li><tt>score</tt> : Associated threat triage score for the rule</li>
+</ul>
+<p>An example of a rule is as follows:</p>
+
+<div class="source">
+<div class="source">
+<pre> "riskLevelRules" : [
+ {
+ "name" : "is internal"
+ , "comment" : "determines if the destination is internal."
+ , rule" : "IN_SUBNET(ip_dst_addr, '192.168.0.0/24')"
+ , "score" : 10
+ }
+ ]
+</pre></div></div>
+<p>The supported aggregation functions are:</p>
+
+<ul>
+
+<li><tt>MAX</tt> : The max of all of the associated values for matching queries</li>
+
+<li><tt>MIN</tt> : The min of all of the associated values for matching queries</li>
+
+<li><tt>MEAN</tt> : The mean of all of the associated values for matching queries</li>
+
+<li><tt>POSITIVE_MEAN</tt> : The mean of the positive associated values for the matching queries.</li>
+</ul></div>
+<div class="section">
+<h3><a name="Example_Configuration"></a>Example Configuration</h3>
+<p>An example configuration for the YAF sensor is as follows:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "enrichment": {
+ "fieldMap": {
+ "geo": [
+ "ip_src_addr",
+ "ip_dst_addr"
+ ],
+ "host": [
+ "ip_src_addr",
+ "ip_dst_addr"
+ ],
+ "hbaseEnrichment": [
+ "ip_src_addr",
+ "ip_dst_addr"
+ ]
+ }
+ ,"fieldToTypeMap": {
+ "ip_src_addr": [
+ "playful_classification"
+ ],
+ "ip_dst_addr": [
+ "playful_classification"
+ ]
+ }
+ },
+ "threatIntel": {
+ "fieldMap": {
+ "hbaseThreatIntel": [
+ "ip_src_addr",
+ "ip_dst_addr"
+ ]
+ },
+ "fieldToTypeMap": {
+ "ip_src_addr": [
+ "malicious_ip"
+ ],
+ "ip_dst_addr": [
+ "malicious_ip"
+ ]
+ },
+ "triageConfig" : {
+ "riskLevelRules" : [
+ {
+ "rule" : "ip_src_addr == '10.0.2.3' or ip_dst_addr == '10.0.2.3'",
+ "score" : 10
+ }
+ ],
+ "aggregator" : "MAX"
+ }
+ }
+}
+</pre></div></div>
+<p>ThreatIntel alert levels are emitted as a new field “threat.triage.level.” So for the example above, an incoming message that trips the <tt>ip_src_addr</tt> rule will have a new field threat.triage.level=10.</p>
+<p><a name="Example_Enrichment_via_Stellar"></a></p>
+<h1>Example Enrichment via Stellar</h1>
+<p>Let’s walk through doing a simple enrichment using Stellar on your cluster using the Squid topology.</p></div></div>
+<div class="section">
+<h2><a name="Install_Prerequisites"></a>Install Prerequisites</h2>
+<p>Now let’s install some prerequisites:</p>
+
+<ul>
+
+<li>Squid client via <tt>yum install squid</tt></li>
+
+<li>ES Head plugin via <tt>/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head</tt></li>
+</ul>
+<p>Start Squid via <tt>service squid start</tt></p></div>
+<div class="section">
+<h2><a name="Adjust_Enrichment_Configurations_for_Squid_to_Call_Stellar"></a>Adjust Enrichment Configurations for Squid to Call Stellar</h2>
+<p>Let’s adjust the configurations for the Squid topology to annotate the messages using some Stellar functions.</p>
+
+<ul>
+
+<li>
+<p>Edit the squid enrichment configuration at <tt>$METRON_HOME/config/zookeeper/enrichments/squid.json</tt> (this file will not exist, so create a new one) to add some new fields based on stellar queries:</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "enrichment" : {
+"fieldMap": {
+ "stellar" : {
+ "config" : {
+ "numeric" : {
+ "foo": "1 + 1"
+ }
+ ,"ALL_CAPS" : "TO_UPPER(source.type)"
+ }
+ }
+ }
+ },
+ "threatIntel" : {
+"fieldMap":{
+ "stellar" : {
+ "config" : {
+ "bar" : "TO_UPPER(source.type)"
+ }
+ }
+},
+"triageConfig" : {
+}
+ }
+}
+</pre></div></div>
+<p>We have added the following fields as part of the enrichment phase of the enrichment topology:</p></li>
+
+<li>
+<p><tt>foo</tt> == 2</p></li>
+
+<li><tt>ALL_CAPS</tt> == SQUID</li>
+</ul>
+<p>We have added the following as part of the threat intel:</p>
+
+<ul>
+
+<li><tt>bar</tt> == SQUID</li>
+</ul>
+<p>Please note that foo and ALL_CAPS will be applied in separate workers due to them being in separate groups.</p>
+
+<ul>
+
+<li>Upload new configs via <tt>$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -i $METRON_HOME/config/zookeeper -z node1:2181</tt></li>
+
+<li>Make the Squid topic in kafka via <tt>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper node1:2181 --create --topic squid --partitions 1 --replication-factor 1</tt></li>
+</ul></div>
+<div class="section">
+<h2><a name="Start_Topologies_and_Send_Data"></a>Start Topologies and Send Data</h2>
+<p>Now we need to start the topologies and send some data:</p>
+
+<ul>
+
+<li>Start the squid topology via <tt>$METRON_HOME/bin/start_parser_topology.sh -k node1:6667 -z node1:2181 -s squid</tt></li>
+
+<li>Generate some data via the squid client:
+
+<ul>
+
+<li><tt>squidclient http://yahoo.com</tt></li>
+
+<li><tt>squidclient http://cnn.com</tt></li>
+ </ul></li>
+
+<li>Send the data to kafka via <tt>cat /var/log/squid/access.log | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic squid</tt></li>
+
+<li>Browse the data in elasticsearch via the ES Head plugin @ <a class="externalLink" href="http://node1:9200/_plugin/head/">http://node1:9200/_plugin/head/</a> and verify that in the squid index you have two documents</li>
+
+<li>Ensure that the documents have new fields <tt>foo</tt>, <tt>bar</tt> and <tt>ALL_CAPS</tt> with values as described above.</li>
+</ul>
+<p>Note that we could have used any Stellar statements here, including calling out to HBase via <tt>ENRICHMENT_GET</tt> and <tt>ENRICHMENT_EXISTS</tt> or even calling a machine learning model via <a href="../../metron-analytics/metron-maas-service/index.html">Model as a Service</a>.</p>
+<p><a name="Notes_on_Performance_Tuning"></a></p>
+<h1>Notes on Performance Tuning</h1>
+<p>Default installed Metron is untuned for production deployment. There are a few knobs to tune to get the most out of your system.</p></div>
+<div class="section">
+<h2><a name="Kafka_Queue"></a>Kafka Queue</h2>
+<p>The <tt>enrichments</tt> kafka queue is a collection point from all of the parser topologies. As such, make sure that the number of partitions in the kafka topic is sufficient to handle the throughput that you expect from your parser topologies.</p></div>
+<div class="section">
+<h2><a name="Enrichment_Topology"></a>Enrichment Topology</h2>
+<p>The enrichment topology as started by the <tt>$METRON_HOME/bin/start_enrichment_topology.sh</tt> script uses a default of one executor per bolt. In a real production system, this should be customized by modifying the flux file in <tt>$METRON_HOME/flux/enrichment/remote.yaml</tt>. </p>
+
+<ul>
+
+<li>Add a <tt>parallelism</tt> field to the bolts to give Storm a parallelism hint for the various components. Give bolts which appear to be bottlenecks (e.g. stellar enrichment bolt, hbase enrichment and threat intel bolts) a larger hint.</li>
+
+<li>Add a <tt>parallelism</tt> field to the kafka spout which matches the number of partitions for the enrichment kafka queue.</li>
+
+<li>Adjust the number of workers for the topology by adjusting the <tt>topology.workers</tt> field for the topology.</li>
+</ul>
+<p>Finally, if workers and executors are new to you or you don’t know where to modify the flux file, the following might be of use to you:</p>
+
+<ul>
+
+<li><a class="externalLink" href="http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/">Understanding the Parallelism of a Storm Topology</a></li>
+
+<li><a class="externalLink" href="http://storm.apache.org/releases/current/flux.html">Flux Docs</a></li>
+</ul></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-platform/metron-indexing/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-indexing/index.html b/site/current-book/metron-platform/metron-indexing/index.html
new file mode 100644
index 0000000..1f5d0cf
--- /dev/null
+++ b/site/current-book/metron-platform/metron-indexing/index.html
@@ -0,0 +1,428 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Indexing</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Indexing</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-down"></i>
+ Platform</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-platform/metron-api/index.html" title="Api">
+ <i class="none"></i>
+ Api</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-common/index.html" title="Common">
+ <i class="none"></i>
+ Common</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
+ <i class="none"></i>
+ Data-management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
+ <i class="none"></i>
+ Enrichment</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Indexing</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-management/index.html" title="Management">
+ <i class="none"></i>
+ Management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
+ <i class="none"></i>
+ Parsers</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
+ <i class="none"></i>
+ Pcap-backend</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Indexing</h1>
+<p><a name="Indexing"></a></p>
+<div class="section">
+<h2><a name="Introduction"></a>Introduction</h2>
+<p>The <tt>indexing</tt> topology is a topology dedicated to taking the data from the enrichment topology that have been enriched and storing the data in one or more supported indices</p>
+
+<ul>
+
+<li>HDFS as rolled text files, one JSON blob per line</li>
+
+<li>Elasticsearch</li>
+
+<li>Solr</li>
+</ul>
+<p>By default, this topology writes out to both HDFS and one of Elasticsearch and Solr.</p>
+<p>Indices are written in batch and the batch size is specified in the <a href="#Sensor_Indexing_Configuration">Sensor Indexing Configuration</a> via the <tt>batchSize</tt> parameter. This config is variable by sensor type.</p></div>
+<div class="section">
+<h2><a name="Indexing_Architecture"></a>Indexing Architecture</h2>
+<p><img src="../../images/indexing_arch.png" alt="Architecture" /></p>
+<p>The indexing topology is extremely simple. Data is ingested into kafka and sent to </p>
+
+<ul>
+
+<li>An indexing bolt configured to write to either elasticsearch or Solr</li>
+
+<li>An indexing bolt configured to write to HDFS under <tt>/apps/metron/enrichment/indexed</tt></li>
+</ul>
+<p>Errors during indexing are sent to a kafka queue called <tt>index_errors</tt></p></div>
+<div class="section">
+<h2><a name="Sensor_Indexing_Configuration"></a>Sensor Indexing Configuration</h2>
+<p>The sensor specific configuration is intended to configure the indexing used for a given sensor type (e.g. <tt>snort</tt>). </p>
+<p>Just like the global config, the format is a JSON stored in zookeeper and on disk at <tt>$METRON_HOME/config/zookeeper/indexing</tt>. Within the sensor-specific configuration, you can configure the individual writers. The writers currently supported are:</p>
+
+<ul>
+
+<li><tt>elasticsearch</tt></li>
+
+<li><tt>hdfs</tt></li>
+
+<li><tt>solr</tt></li>
+</ul>
+<p>Depending on how you start the indexing topology, it will have either elasticsearch or solr and hdfs writers running.</p>
+<p>The configuration for an individual writer-specific configuration is a JSON map with the following fields:</p>
+
+<ul>
+
+<li><tt>index</tt> : The name of the index to write to (defaulted to the name of the sensor).</li>
+
+<li><tt>batchSize</tt> : The size of the batch that is written to the indices at once (defaulted to <tt>1</tt>).</li>
+
+<li><tt>enabled</tt> : Whether the writer is enabled (default <tt>true</tt>).</li>
+</ul>
+<div class="section">
+<h3><a name="Indexing_Configuration_Examples"></a>Indexing Configuration Examples</h3>
+<p>For a given sensor, the following scenarios would be indicated by the following cases:</p>
+<div class="section">
+<h4><a name="Base_Case"></a>Base Case</h4>
+
+<div class="source">
+<div class="source">
+<pre>{
+}
+</pre></div></div>
+<p>or no file at all.</p>
+
+<ul>
+
+<li>elasticsearch writer
+
+<ul>
+
+<li>enabled</li>
+
+<li>batch size of 1</li>
+
+<li>index name the same as the sensor</li>
+ </ul></li>
+
+<li>hdfs writer
+
+<ul>
+
+<li>enabled</li>
+
+<li>batch size of 1</li>
+
+<li>index name the same as the sensor</li>
+ </ul></li>
+</ul>
+<p>If a writer config is unspecified, then a warning is indicated in the Storm console. e.g.: <tt>WARNING: Default and (likely) unoptimized writer config used for hdfs writer and sensor squid</tt></p></div>
+<div class="section">
+<h4><a name="Fully_specified"></a>Fully specified</h4>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "elasticsearch": {
+ "index": "foo",
+ "batchSize" : 100,
+ "enabled" : true
+ },
+ "hdfs": {
+ "index": "foo",
+ "batchSize": 1,
+ "enabled" : true
+ }
+}
+</pre></div></div>
+
+<ul>
+
+<li>elasticsearch writer
+
+<ul>
+
+<li>enabled</li>
+
+<li>batch size of 100</li>
+
+<li>index name of “foo”</li>
+ </ul></li>
+
+<li>hdfs writer
+
+<ul>
+
+<li>enabled</li>
+
+<li>batch size of 1</li>
+
+<li>index name of “foo”</li>
+ </ul></li>
+</ul></div>
+<div class="section">
+<h4><a name="HDFS_Writer_turned_off"></a>HDFS Writer turned off</h4>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "elasticsearch": {
+ "index": "foo",
+ "enabled" : true
+ },
+ "hdfs": {
+ "index": "foo",
+ "batchSize": 100,
+ "enabled" : false
+ }
+}
+</pre></div></div>
+
+<ul>
+
+<li>elasticsearch writer
+
+<ul>
+
+<li>enabled</li>
+
+<li>batch size of 1</li>
+
+<li>index name of “foo”</li>
+ </ul></li>
+
+<li>hdfs writer
+
+<ul>
+
+<li>disabled</li>
+ </ul></li>
+</ul>
+<p><a name="Notes_on_Performance_Tuning"></a></p>
+<h1>Notes on Performance Tuning</h1>
+<p>Default installed Metron is untuned for production deployment. By far and wide, the most likely piece to require TLC from a performance perspective is the indexing layer. An index that does not keep up will back up and you will see errors in the kafka bolt. There are a few knobs to tune to get the most out of your system.</p></div></div></div>
+<div class="section">
+<h2><a name="Kafka_Queue"></a>Kafka Queue</h2>
+<p>The <tt>indexing</tt> kafka queue is a collection point from the enrichment topology. As such, make sure that the number of partitions in the kafka topic is sufficient to handle the throughput that you expect.</p></div>
+<div class="section">
+<h2><a name="Indexing_Topology"></a>Indexing Topology</h2>
+<p>The enrichment topology as started by the <tt>$METRON_HOME/bin/start_elasticsearch_topology.sh</tt> or <tt>$METRON_HOME/bin/start_solr_topology.sh</tt> script uses a default of one executor per bolt. In a real production system, this should be customized by modifying the flux file in <tt>$METRON_HOME/flux/indexing/remote.yaml</tt>. </p>
+
+<ul>
+
+<li>Add a <tt>parallelism</tt> field to the bolts to give Storm a parallelism hint for the various components. Give bolts which appear to be bottlenecks (e.g. the indexing bolt) a larger hint.</li>
+
+<li>Add a <tt>parallelism</tt> field to the kafka spout which matches the number of partitions for the enrichment kafka queue.</li>
+
+<li>Adjust the number of workers for the topology by adjusting the <tt>topology.workers</tt> field for the topology.</li>
+</ul>
+<p>Finally, if workers and executors are new to you or you don’t know where to modify the flux file, the following might be of use to you:</p>
+
+<ul>
+
+<li><a class="externalLink" href="http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/">Understanding the Parallelism of a Storm Topology</a></li>
+
+<li><a class="externalLink" href="http://storm.apache.org/releases/current/flux.html">Flux Docs</a></li>
+</ul></div>
+<div class="section">
+<h2><a name="Zeppelin_Notebooks"></a>Zeppelin Notebooks</h2>
+<p>Zeppelin notebooks can be added to <tt>/src/main/config/zeppelin/</tt> (and subdirectories can be created for organization). The placed files must be .json files and be named appropriately. These files must be added to the metron.spec file and the RPMs rebuilt to be available to be loaded into Ambari.</p>
+<p>The notebook files will be found on the server in <tt>$METRON_HOME/config/zeppelin</tt></p>
+<p>The Ambari Management Pack has a custom action to load these templates, ZEPPELIN_DASHBOARD_INSTALL, that will import them into Zeppelin.</p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
[06/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/roles/opentaxii/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/opentaxii/index.html b/site/current-book/metron-deployment/roles/opentaxii/index.html
new file mode 100644
index 0000000..5965ce9
--- /dev/null
+++ b/site/current-book/metron-deployment/roles/opentaxii/index.html
@@ -0,0 +1,419 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – OpenTAXII</title>
+ <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../css/site.css" />
+ <link rel="stylesheet" href="../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">OpenTAXII</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-down"></i>
+ Roles</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/kibana/index.html" title="Kibana">
+ <i class="none"></i>
+ Kibana</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/monit/index.html" title="Monit">
+ <i class="none"></i>
+ Monit</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Opentaxii</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
+ <i class="none"></i>
+ Pcap_replay</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
+ <i class="none"></i>
+ Sensor-stubs</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
+ <i class="none"></i>
+ Sensor-test-mode</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>OpenTAXII</h1>
+<p><a name="OpenTAXII"></a></p>
+<p>Installs <a class="externalLink" href="https://github.com/EclecticIQ/OpenTAXII">OpenTAXII</a> as a deamon that can be launched via a SysV service script. The complementary client implementation, <a class="externalLink" href="https://github.com/EclecticIQ/cabby">Cabby</a> is also installed.</p>
+<p>OpenTAXII is a robust Python implementation of TAXII Services that delivers a rich feature set and friendly pythonic API. <a class="externalLink" href="https://stixproject.github.io/">TAXII</a> (Trusted Automated eXchange of Indicator Information) is a collection of specifications defining a set of services and message exchanges used for sharing cyber threat intelligence information between parties.</p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>After deployment completes the OpenTAXII service is installed and running. A set of <a class="externalLink" href="http://hailataxii.com/">Hail a TAXII</a> threat intel collections have been defined and configured. Use the <tt>status</tt> option to view the collections that have been defined.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ service opentaxii status
+Checking opentaxii... Running
+guest.phishtank_com 0
+guest.Abuse_ch 0
+guest.CyberCrime_Tracker 0
+guest.EmergingThreats_rules 0
+guest.Lehigh_edu 0
+guest.MalwareDomainList_Hostlist 0
+guest.blutmagie_de_torExits 0
+guest.dataForLast_7daysOnly 0
+guest.dshield_BlockList 0
+</pre></div></div>
+<p>Notice that each collections contain zero records. None of the data is automatically synced during deployment. To sync the data manually use the <tt>sync</tt> option as defined below. The following example does not provide a begin and end time so the data will be fetched for the current day only.</p>
+
+<div class="source">
+<div class="source">
+<pre># service opentaxii sync guest.blutmagie_de_torExits
+2016-04-21 20:34:42,511 INFO: Starting new HTTP connection (1): localhost
+2016-04-21 20:34:42,540 INFO: Response received for Inbox_Message from http://localhost:9000/services/inbox
+2016-04-21 20:34:42,542 INFO: Sending Inbox_Message to http://localhost:9000/services/inbox
+...
+2016-04-21 20:34:42,719 INFO: Response received for Poll_Request from http://localhost:9000/services/poll
+2016-04-21 20:34:42,719 INFO: Content blocks count: 1618, is partial: False
+</pre></div></div>
+<p>The OpenTAXII service now contains 1,618 threat intel records indicating Tor Exit nodes.</p>
+
+<div class="source">
+<div class="source">
+<pre>[root@source ~]# service opentaxii status
+Checking opentaxii... Running
+guest.phishtank_com 0
+guest.Abuse_ch 0
+guest.CyberCrime_Tracker 0
+guest.EmergingThreats_rules 0
+guest.Lehigh_edu 0
+guest.MalwareDomainList_Hostlist 0
+guest.blutmagie_de_torExits 1618
+guest.dataForLast_7daysOnly 0
+guest.dshield_BlockList 0
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Usage"></a>Usage</h2>
+<p>A standard SysV script has been installed to manage OpenTAXII. The following functions are available.</p>
+<p><tt>start</tt> <tt>stop</tt> <tt>restart</tt> the OpenTAXII service</p>
+<p><tt>status</tt> of the OpenTAXII service. The command displays the collections that have been defined and the number of records in each.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ service opentaxii status
+Checking opentaxii... Running
+guest.phishtank_com 984
+guest.Abuse_ch 45
+guest.CyberCrime_Tracker 482
+guest.EmergingThreats_rules 0
+guest.Lehigh_edu 1030
+guest.MalwareDomainList_Hostlist 84
+guest.blutmagie_de_torExits 3236
+guest.dataForLast_7daysOnly 3377
+guest.dshield_BlockList 0
+</pre></div></div>
+<p><tt>setup</tt> Initializes the services and collections required to operate the OpenTAXII service. This will destroy all existing data. The user is prompted to continue before any data is destroyed.</p>
+
+<div class="source">
+<div class="source">
+<pre># service opentaxii setup
+WARNING: force reset and destroy all opentaxii data? [Ny]: y
+Stopping opentaxii ..Ok
+2016-04-21T19:56:01.886157Z [opentaxii.server] info: api.persistence.loaded {timestamp=2016-04-21T19:56:01.886157Z, logger=opentaxii.server, api_class=SQLDatabaseAPI, event=api.persistence.loaded, level=info}
+2016-04-21T19:56:01.896503Z [opentaxii.server] info: api.auth.loaded {timestamp=2016-04-21T19:56:01.896503Z, logger=opentaxii.server, api_class=SQLDatabaseAPI, event=api.auth.loaded, level=info}
+2016-04-21T19:56:01.896655Z [opentaxii.server] info: taxiiserver.configured {timestamp=2016-04-21T19:56:01.896655Z, logger=opentaxii.server, event=taxiiserver.configured, level=info}
+...
+Ok
+</pre></div></div>
+<p><tt>sync [collection] [begin-at] [end-at]</tt> Syncs the threat intel data available at <a class="externalLink" href="http://hailataxii.com/">Hail a TAXII</a>. If no begin and end date is provided then data is synced over the current day only.</p>
+
+<ul>
+
+<li><tt>collection</tt> Name of the collection to sync.</li>
+
+<li><tt>begin-at</tt> Exclusive begin of time window; ISO8601</li>
+
+<li><tt>end-at</tt> Inclusive end of time window; ISO8601</li>
+</ul>
+
+<div class="source">
+<div class="source">
+<pre>$ service opentaxii sync guest.phishtank_com
++ /usr/local/opentaxii/opentaxii-venv/bin/taxii-proxy --poll-path http://hailataxii.com/taxii-data --poll-collection guest.phishtank_com --inbox-path http://localhost:9000/services/guest.phishtank_com-inbox --inbox-collection guest.phishtank_com --binding urn:stix.mitre.org:xml:1.1.1 --begin 2016-04-21 --end 2016-04-22
+2016-04-21 17:36:23,778 INFO: Sending Poll_Request to http://hailataxii.com/taxii-data
+2016-04-21 17:36:23,784 INFO: Starting new HTTP connection (1): hailataxii.com
+2016-04-21 17:36:24,175 INFO: Response received for Poll_Request from http://hailataxii.com/taxii-data
+2016-04-21 17:36:24,274 INFO: Sending Inbox_Message to http://localhost:9000/services/guest.phishtank_com-inbox
+...
+2016-04-21 17:36:34,867 INFO: Response received for Poll_Request from http://localhost:9000/services/guest.phishtank_com-poll
+2016-04-21 17:36:34,868 INFO: Content blocks count: 6993, is partial: False
+</pre></div></div>
+<div class="section">
+<h3><a name="Troubleshooting"></a>Troubleshooting</h3>
+<p>Should you need to explore the installation, here are instructions on doing so.</p>
+<p>OpenTAXII is installed in a virtual environment. Before exploring the environment run the following commands to perform the necessary setup. The specific paths may change depending on your Ansible settings.</p>
+
+<div class="source">
+<div class="source">
+<pre>export LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64
+export OPENTAXII_CONFIG=/usr/local/opentaxii/etc/opentaxii-conf.yml
+cd /usr/local/opentaxii
+. opentaxii-venv/bin/activate
+</pre></div></div>
+<p>Discover available services.</p>
+
+<div class="source">
+<div class="source">
+<pre>taxii-discovery --discovery http://localhost:9000/services/discovery
+taxii-discovery --discovery http://hailataxii.com/taxii-data
+</pre></div></div>
+<p>Explore available collections.</p>
+
+<div class="source">
+<div class="source">
+<pre>taxii-collections --discovery http://localhost:9000/services/discovery
+taxii-collections --discovery http://hailataxii.com/taxii-data
+</pre></div></div>
+<p>Read data from a collection.</p>
+
+<div class="source">
+<div class="source">
+<pre>taxii-poll --discovery http://localhost:9000/services/discovery -c guest.phishtank_com
+taxii-poll --discovery http://hailataxii.com/taxii-data -c guest.phishtank_com --begin 2016-04-20
+</pre></div></div>
+<p>Manually load data into a collection.</p>
+
+<div class="source">
+<div class="source">
+<pre>taxii-push \
+ --discovery http://localhost:9000/services/discovery \
+ --dest phishtank \
+ --content-file data.xml \
+ --username guest \
+ --password guest
+</pre></div></div>
+<p>Fetch data from a remote service and mirror it locally.</p>
+
+<div class="source">
+<div class="source">
+<pre>taxii-proxy --poll-path http://hailataxii.com/taxii-data \
+ --poll-collection guest.phishtank_com \
+ --inbox-path http://localhost:9000/services/guest.phishtank_com-inbox \
+ --inbox-collection guest.phishtank_com \
+ --binding urn:stix.mitre.org:xml:1.1.1 \
+ --inbox-username guest \
+ --inbox-password guest \
+ --begin 2016-04-20
+</pre></div></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/roles/pcap_replay/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/pcap_replay/index.html b/site/current-book/metron-deployment/roles/pcap_replay/index.html
new file mode 100644
index 0000000..cd95448
--- /dev/null
+++ b/site/current-book/metron-deployment/roles/pcap_replay/index.html
@@ -0,0 +1,294 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Pcap Replay</title>
+ <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../css/site.css" />
+ <link rel="stylesheet" href="../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Pcap Replay</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-down"></i>
+ Roles</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/kibana/index.html" title="Kibana">
+ <i class="none"></i>
+ Kibana</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/monit/index.html" title="Monit">
+ <i class="none"></i>
+ Monit</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
+ <i class="none"></i>
+ Opentaxii</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Pcap_replay</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
+ <i class="none"></i>
+ Sensor-stubs</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
+ <i class="none"></i>
+ Sensor-test-mode</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Pcap Replay</h1>
+<p>This project enables packet capture data to be replayed through a network interface to simulate live network traffic. This can be used to support functional, performance, and load testing of Apache Metron.</p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>To replay packet capture data, simply start the <tt>pcap-replay</tt> SysV service. To do this run the following command.</p>
+
+<div class="source">
+<div class="source">
+<pre>service pcap-replay start
+</pre></div></div>
+<p>All additional options accepted by <tt>tcpreplay</tt> can be passed to the service script to modify how the network data is replayed. For example, this makes it simple to control the amount and rate of data replayed during functional, performance and load testing.</p>
+<p>Example: Replay data at a rate of 10 mbps.</p>
+
+<div class="source">
+<div class="source">
+<pre>service pcap-replay start --mbps 10
+</pre></div></div>
+<p>Example: Replay data at a rate of 10 packets per second.</p>
+
+<div class="source">
+<div class="source">
+<pre>service pcap-replay start --pps 10
+</pre></div></div>
+<p>All nodes on the same subnet with their network interface set to promiscuous mode will then be able to capture the network traffic being replayed. To validate, simply run something like the following.</p>
+
+<div class="source">
+<div class="source">
+<pre>tcpdump -i eth1
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Data"></a>Data</h2>
+<p>An example packet capture file has been installed at <tt>/opt/pcap-replay/example.pcap</tt>. By default, the network traffic contained within this file is continually replayed. </p>
+<p>To replay your own packet capture data, simply add any number of files containing <tt>libpcap</tt> formatted packet capture data to <tt>/opt/pcap-replay</tt>. The files must end with the <tt>.pcap</tt> extension. To pick up newly installed files, simply restart the service.</p>
+
+<div class="source">
+<div class="source">
+<pre>service pcap-replay restart
+</pre></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/roles/sensor-stubs/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/sensor-stubs/index.html b/site/current-book/metron-deployment/roles/sensor-stubs/index.html
new file mode 100644
index 0000000..5f16354
--- /dev/null
+++ b/site/current-book/metron-deployment/roles/sensor-stubs/index.html
@@ -0,0 +1,340 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Sensor Stubs</title>
+ <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../css/site.css" />
+ <link rel="stylesheet" href="../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Sensor Stubs</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-down"></i>
+ Roles</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/kibana/index.html" title="Kibana">
+ <i class="none"></i>
+ Kibana</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/monit/index.html" title="Monit">
+ <i class="none"></i>
+ Monit</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
+ <i class="none"></i>
+ Opentaxii</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
+ <i class="none"></i>
+ Pcap_replay</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Sensor-stubs</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
+ <i class="none"></i>
+ Sensor-test-mode</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Sensor Stubs</h1>
+<p><a name="Sensor_Stubs"></a></p>
+<p>A service has been created to simulate the behavior of a sensor by sending canned telemetry data to a Kafka topic. These “Sensor Stubs” consume fewer resources than the actual sensor that they replace.</p>
+<div class="section">
+<div class="section">
+<h3><a name="aQ_How_do_the_sensor_stubs_work"></a>(Q) How do the sensor stubs work?</h3>
+<p>The stubs are installed with a set of canned data for each sensor type; Bro, Snort and YAF. A subset of this canned data is randomly selected and sent to the Kafka topic in batches. The timestamp of each message is updated to match current system time. </p></div>
+<div class="section">
+<h3><a name="aQ_How_do_I_configure_the_message_rate"></a>(Q) How do I configure the message rate?</h3>
+<p>The number of telemetry messages sent in each batch, along with the time delay between batches is configurable. Before installation, these values can be configured by redefining <tt>sensor_stubs_delay</tt> and <tt>sensor_stubs_count</tt>. The values can also be configured by altering the deployed system service script at <tt>/etc/init.d/sensor-stubs</tt>.</p></div>
+<div class="section">
+<h3><a name="aQ_How_do_I_install_the_sensor_stubs"></a>(Q) How do I install the sensor stubs?</h3>
+<p>Using the default playbooks, this role can be installed by using the Ansible tag <tt>sensor-stubs</tt>. This service is installed on the same hosts where the sensors would be; defined by the <tt>sensors</tt> host group.</p>
+<p>The defaults for the “Quick Dev” and “Full Dev” environments have been changed so that the Sensor Stubs are installed by default, rather than the sensors themselves. The Amazon EC2 environment continues to install the original sensors by default.</p></div>
+<div class="section">
+<h3><a name="aQ_How_do_I_use_the_sensor_stubs"></a>(Q) How do I use the sensor stubs?</h3>
+<p>Start all sensor stubs. The output includes the PID for each running sensor stub.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ service sensor-stubs start
+Starting sensor-stubs...
+ bro: Ok [26505]
+ yaf: Ok [26507]
+ snort: Ok [26509]
+</pre></div></div>
+<p>Check the status of each sensor stub.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ service sensor-stubs status
+Checking sensor-stubs...
+ bro: Running [26505]
+ yaf: Running [26507]
+ snort: Running [26509]
+</pre></div></div>
+<p>Stop all sensor stubs.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ service sensor-stubs stop
+Stopping sensor-stubs...
+.. bro: Ok [26505]
+.. yaf: Ok [26507]
+.. snort: Ok [26509]
+</pre></div></div>
+<p>Check the status. All sensor stubs should be stopped.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ service sensor-stubs status
+Checking sensor-stubs...
+ bro: Not running
+ yaf: Not running
+ snort: Not running
+</pre></div></div>
+<p>Start only the Bro sensor stub.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ service sensor-stubs start bro
+Starting sensor-stubs...
+ bro: OK [11616]
+</pre></div></div>
+<p>Stop the Bro sensor stub.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ service sensor-stubs stop bro
+Stopping sensor-stubs...
+.. bro: Ok [11616]
+</pre></div></div></div>
+<div class="section">
+<h3><a name="aQ_How_do_I_install_the_original_sensors"></a>(Q) How do I install the original sensors?</h3>
+<p>The default behavior can be changed by skipping the <tt>sensor-stubs</tt> flag and including the <tt>sensors</tt> flag. For example, to deploy “Quick Dev” with the original sensors run the following command.</p>
+
+<div class="source">
+<div class="source">
+<pre>cd metron-deployment/vagrant/quick-dev-platform
+vagrant --ansible-skip-tags="sensor-stubs,solr" up
+</pre></div></div></div>
+<div class="section">
+<h3><a name="aQ_Where_does_the_mock_data_come_from"></a>(Q) Where does the mock data come from?</h3>
+<p>The data produced by the sensor stubs was generated by running the sensors against the example pcap file that is distributed with Metron. This ensures that the data produced by the sensor stubs is similar to the data produced when using the actual sensors.</p></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/roles/sensor-test-mode/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/sensor-test-mode/index.html b/site/current-book/metron-deployment/roles/sensor-test-mode/index.html
new file mode 100644
index 0000000..a55f54f
--- /dev/null
+++ b/site/current-book/metron-deployment/roles/sensor-test-mode/index.html
@@ -0,0 +1,284 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Sensor Test Mode</title>
+ <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../css/site.css" />
+ <link rel="stylesheet" href="../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Sensor Test Mode</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-down"></i>
+ Roles</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/kibana/index.html" title="Kibana">
+ <i class="none"></i>
+ Kibana</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/monit/index.html" title="Monit">
+ <i class="none"></i>
+ Monit</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
+ <i class="none"></i>
+ Opentaxii</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
+ <i class="none"></i>
+ Pcap_replay</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
+ <i class="none"></i>
+ Sensor-stubs</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Sensor-test-mode</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-right"></i>
+ Vagrant</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Sensor Test Mode</h1>
+<p>A role that configures each of the sensors to produce the maximum amount of telemetry data. This role is useful only for testing. It can be useful to support functional, performance, and load testing of Apache Metron.</p>
+<p>The role does the following to maximize the amount of telemetry data produced by each Metron sensor.</p>
+
+<ul>
+
+<li>Plays a packet capture file through a network interface to simulate live network traffic.</li>
+
+<li>Configures <a class="externalLink" href="https://tools.netsa.cert.org/yaf/yaf.html">YAF</a> with <tt>idle-timeout=0</tt>. This causes a flow record to be produced for every network packet received.</li>
+
+<li>Configures <a class="externalLink" href="https://www.snort.org/">Snort</a> to produce an alert for every network packet received.</li>
+</ul>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>To enable the <tt>sensor-test-mode</tt> role apply the role to the <tt>sensors</tt> host group in your Ansible playbook.</p>
+
+<div class="source">
+<div class="source">
+<pre>- hosts: sensors
+ roles:
+ - role: sensor-test-mode
+</pre></div></div>
+<p>The role has also been added to the default <tt>metron_install.yml</tt> playbook so that it can be turned on/off with a property in both the local Virtualbox and the remote EC2 deployments.</p>
+
+<div class="source">
+<div class="source">
+<pre>sensor_test_mode: True
+</pre></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/vagrant/codelab-platform/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/vagrant/codelab-platform/index.html b/site/current-book/metron-deployment/vagrant/codelab-platform/index.html
new file mode 100644
index 0000000..39ba07b
--- /dev/null
+++ b/site/current-book/metron-deployment/vagrant/codelab-platform/index.html
@@ -0,0 +1,315 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Developer Image for Apache Metron on Virtualbox</title>
+ <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../css/site.css" />
+ <link rel="stylesheet" href="../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Developer Image for Apache Metron on Virtualbox</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-right"></i>
+ Roles</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-down"></i>
+ Vagrant</a>
+ <ul class="nav nav-list">
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Codelab-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/fastcapa-test-platform/index.html" title="Fastcapa-test-platform">
+ <i class="none"></i>
+ Fastcapa-test-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/full-dev-platform/index.html" title="Full-dev-platform">
+ <i class="none"></i>
+ Full-dev-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/quick-dev-platform/index.html" title="Quick-dev-platform">
+ <i class="none"></i>
+ Quick-dev-platform</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Developer Image for Apache Metron on Virtualbox</h1>
+<p>This image is a fully functional Metron installation that has been pre-loaded with Ambari, HDP and Metron.</p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<p>As with the Singlenode Full Image, the computer used to deploy Apache Metron will need to have the following components installed.</p>
+
+<ul>
+
+<li><a class="externalLink" href="https://github.com/ansible/ansible">Ansible</a> 2.0.0.2</li>
+
+<li><a class="externalLink" href="https://www.vagrantup.com">Vagrant</a> 1.8.1</li>
+
+<li><a class="externalLink" href="https://virtualbox.org">Virtualbox</a> 5.0.16</li>
+
+<li>Python 2.7.11</li>
+
+<li>Maven 3.3.9</li>
+</ul></div>
+<div class="section">
+<h3><a name="Ensure_vagrant_hostmanager_is_installed"></a>Ensure vagrant hostmanager is installed</h3>
+<p>To check and make sure you have the plugin installed execute the following:</p>
+
+<div class="source">
+<div class="source">
+<pre> vagrant plugin list
+</pre></div></div>
+<p>If you have it installed you should see the following listed in the output:</p>
+
+<div class="source">
+<div class="source">
+<pre> vagrant-hostmanager (1.8.1)
+</pre></div></div>
+<p>If it is not installed, you can install it with the following command:</p>
+
+<div class="source">
+<div class="source">
+<pre> vagrant plugin install vagrant-hostmanager
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Launch_the_Metron_Development_Image"></a>Launch the Metron Development Image</h3>
+<p>Start the image with the following commands:</p>
+
+<div class="source">
+<div class="source">
+<pre> cd metron-deployment/vagrant/codelab-platform
+ ./run.sh
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Work_with_Metron"></a>Work with Metron</h3>
+<p>As you build out new capabilities for Metron, you will need to re-deploy the Storm topologies. To do so, first HALT the running Storm topologies and then run:</p>
+
+<div class="source">
+<div class="source">
+<pre>./run_enrichment_role.sh
+</pre></div></div>
+<p>Remember Navigate to the following resources to explore your newly minted Apache Metron environment.</p>
+
+<ul>
+
+<li><a class="externalLink" href="http://node1:5000">Metron</a></li>
+
+<li><a class="externalLink" href="http://node1:8080">Ambari</a></li>
+
+<li><a class="externalLink" href="http://node1:2812">Services</a></li>
+</ul>
+<p>Connecting to the host through SSH is as simple as running the following command.</p>
+
+<div class="source">
+<div class="source">
+<pre> vagrant ssh
+</pre></div></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/vagrant/fastcapa-test-platform/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/vagrant/fastcapa-test-platform/index.html b/site/current-book/metron-deployment/vagrant/fastcapa-test-platform/index.html
new file mode 100644
index 0000000..47d7aed
--- /dev/null
+++ b/site/current-book/metron-deployment/vagrant/fastcapa-test-platform/index.html
@@ -0,0 +1,313 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Fastcapa Test Environment</title>
+ <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../css/site.css" />
+ <link rel="stylesheet" href="../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Fastcapa Test Environment</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-right"></i>
+ Roles</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-down"></i>
+ Vagrant</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/codelab-platform/index.html" title="Codelab-platform">
+ <i class="none"></i>
+ Codelab-platform</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Fastcapa-test-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/full-dev-platform/index.html" title="Full-dev-platform">
+ <i class="none"></i>
+ Full-dev-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/quick-dev-platform/index.html" title="Quick-dev-platform">
+ <i class="none"></i>
+ Quick-dev-platform</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Fastcapa Test Environment</h1>
+<p>Provides a test environment for the development and testing of Fastcapa. The environment is automatically validated after it is created to ensure that Fastcapa is behaving correctly.</p>
+<p>Two virtualized nodes are launched with Vagrant that can communicate with one another over a private network. </p>
+
+<ul>
+
+<li>The <tt>source</tt> node uses Metron’s <tt>pcap_replay</tt> functionality to transmit raw network packet data over a private network.</li>
+
+<li>The <tt>sink</tt> node is running <tt>fastcapa</tt> and is capturing these network packets.</li>
+
+<li>Fastcapa then transforms and bundles the packets into a message.</li>
+
+<li>The message is sent to a Kafka broker running on the <tt>source</tt> node.</li>
+</ul>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>Simply run <tt>vagrant up</tt> to launch the environment. Automated tests are executed after provisioning completes to ensure that Fastcapa and the rest of the environment is functioning properly.</p>
+
+<div class="source">
+<div class="source">
+<pre>$ vagrant up
+==> source: Running provisioner: ansible...
+ source: Running ansible-playbook...
+...
+TASK [debug] *******************************************************************
+ok: [source] => {
+ "msg": "Successfully received packets sent from pcap-replay!"
+}
+...
+TASK [debug] *******************************************************************
+ok: [source] => {
+ "msg": "Successfully received a Kafka message from fastcapa!"
+}
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Going_Deeper"></a>Going Deeper</h2>
+<p>This section will outline in more detail the environment and how to interact with it.</p>
+<div class="section">
+<h3><a name="source"></a><tt>source</tt></h3>
+<p>To validate that the <tt>source</tt> node is functioning properly, run the following commands.</p>
+<p>First, ensure that the <tt>pcap-replay</tt> service is running.</p>
+
+<div class="source">
+<div class="source">
+<pre>vagrant ssh source
+sudo service pcap-replay status
+</pre></div></div>
+<p>Use <tt>tcpdump</tt> to ensure that the raw packet data is being sent over the private network. Enter ‘CTRL-C’ to kill the <tt>tcpdump</tt> process once you are able to see that packets are being sent.</p>
+
+<div class="source">
+<div class="source">
+<pre>sudo yum -y install tcpdump
+sudo tcpdump -i enp0s8
+</pre></div></div></div>
+<div class="section">
+<h3><a name="sink"></a><tt>sink</tt></h3>
+<p>Next validate that the <tt>sink</tt> is functioning properly. Run the following commands starting from the host operating system. </p>
+<p>First, ensure that the <tt>fastcapa</tt> service is running.</p>
+
+<div class="source">
+<div class="source">
+<pre>vagrant ssh sink
+service fastcapa status
+</pre></div></div>
+<p>Ensure that the raw network packet data is being received by Kafka.</p>
+
+<div class="source">
+<div class="source">
+<pre>/usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic pcap
+</pre></div></div>
+<p>Enter ‘CTRL-C’ to kill the <tt>kafka-console-consumer</tt> process once you are able to see that packets are being sent. These packets will appear to be gibberish in the console. This is the raw binary network packet data after all.</p></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
[05/14] incubator-metron git commit: METRON-766: Release 0.3.1 closes
apache/incubator-metron#477
Posted by ce...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/vagrant/full-dev-platform/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/vagrant/full-dev-platform/index.html b/site/current-book/metron-deployment/vagrant/full-dev-platform/index.html
new file mode 100644
index 0000000..26a1336
--- /dev/null
+++ b/site/current-book/metron-deployment/vagrant/full-dev-platform/index.html
@@ -0,0 +1,374 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Full Development Platform</title>
+ <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../css/site.css" />
+ <link rel="stylesheet" href="../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Full Development Platform</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-right"></i>
+ Roles</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-down"></i>
+ Vagrant</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/codelab-platform/index.html" title="Codelab-platform">
+ <i class="none"></i>
+ Codelab-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/fastcapa-test-platform/index.html" title="Fastcapa-test-platform">
+ <i class="none"></i>
+ Fastcapa-test-platform</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Full-dev-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/quick-dev-platform/index.html" title="Quick-dev-platform">
+ <i class="none"></i>
+ Quick-dev-platform</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Full Development Platform</h1>
+<p>This project fully automates the provisioning and deployment of Apache Metron and all necessary prerequisites on a single, virtualized host running on Virtualbox. </p>
+<p>Metron is composed of many components and installing all of these on a single host, especially a virtualized one, will greatly stress the resources of the host. The host will require at least 8 GB of RAM and a fair amount of patience. It is highly recommended that you shut down all unnecessary services. To that end the vagrant file configuration defaults to disabling solr and yaf.</p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<p>The computer used to deploy Apache Metron will need to have the following components installed.</p>
+
+<ul>
+
+<li><a class="externalLink" href="https://github.com/ansible/ansible">Ansible</a> 2.0.0.2</li>
+
+<li><a class="externalLink" href="https://www.vagrantup.com">Vagrant</a> 1.8.1</li>
+
+<li><a class="externalLink" href="https://virtualbox.org">Virtualbox</a> 5.0.16</li>
+
+<li>Python 2.7.11</li>
+
+<li>Maven 3.3.9</li>
+</ul>
+<div class="section">
+<h4><a name="macOS"></a>macOS</h4>
+<p>Any platform that supports these tools is suitable, but the following instructions cover installation on macOS. The easiest means of installing these tools on a Mac is to use the excellent <a class="externalLink" href="http://brew.sh/">Homebrew</a> project.</p>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Install Homebrew by following the instructions at <a class="externalLink" href="http://brew.sh/">Homebrew</a>.</p></li>
+
+<li>
+<p>Run the following command in a terminal to install all of the required tools.</p>
+
+<div class="source">
+<div class="source">
+<pre> brew cask install vagrant virtualbox java
+ brew install maven git
+ brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/ee1273bf919a5e4e50838513a9e55ea423e1d7ce/Formula/ansible.rb
+ brew switch ansible 2.0.0.2
+</pre></div></div></li>
+</ol></div></div>
+<div class="section">
+<h3><a name="Deploy_Metron"></a>Deploy Metron</h3>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Build Metron</p>
+
+<div class="source">
+<div class="source">
+<pre> cd incubator-metron
+ mvn clean package -DskipTests
+</pre></div></div></li>
+
+<li>
+<p>Install Vagrant Hostmanager.</p>
+
+<div class="source">
+<div class="source">
+<pre> vagrant plugin install vagrant-hostmanager
+</pre></div></div></li>
+
+<li>
+<p>Deploy Metron</p>
+
+<div class="source">
+<div class="source">
+<pre> cd metron-deployment/vagrant/full-dev-platform
+ vagrant up
+</pre></div></div>
+<p>Should the process fail before completing the deployment, the following command will continue the deployment process without re-instantiating the host.</p>
+
+<div class="source">
+<div class="source">
+<pre> vagrant provision
+</pre></div></div></li>
+</ol></div>
+<div class="section">
+<h3><a name="Explore_Metron"></a>Explore Metron</h3>
+<p>Navigate to the following resources to explore your newly minted Apache Metron environment.</p>
+
+<ul>
+
+<li><a class="externalLink" href="http://node1:8080">Ambari</a></li>
+
+<li><a class="externalLink" href="http://node1:5000">Metron</a></li>
+
+<li><a class="externalLink" href="http://node1:2812">Services</a></li>
+</ul>
+<p>Connecting to the host through SSH is as simple as running the following command.</p>
+
+<div class="source">
+<div class="source">
+<pre>vagrant ssh
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Working_with_Metron"></a>Working with Metron</h3>
+<p>In addition to re-running the entire provisioning play book, you may now re-run an individual Ansible tag or a collection of tags in the following ways. The following commands will re-run the <tt>web</tt> role on the Vagrant image. This will install components (if necessary) and start the UI.</p>
+
+<div class="source">
+<div class="source">
+<pre>./run_ansible_role.sh web
+</pre></div></div>
+<p>or</p>
+
+<div class="source">
+<div class="source">
+<pre>vagrant --ansible-tags="web" provision
+</pre></div></div>
+<div class="section">
+<h4><a name="Using_Tags"></a>Using Tags</h4>
+<p>A collection of tags is specified as a comma separated list.</p>
+
+<div class="source">
+<div class="source">
+<pre>./run_ansible_role.sh "sensors,enrichment"
+
+</pre></div></div>
+<p>Tags are listed in the playbooks, some frequently used tags:</p>
+
+<ul>
+
+<li><tt>hdp-install</tt> - Install HDP</li>
+
+<li><tt>hdp-deploy</tt> - Deploy and Start HDP Services (will start all Hadoop Services)</li>
+
+<li><tt>sensors</tt> - Deploy and Start Sensors.</li>
+
+<li><tt>enrichment</tt> - Deploy and Start Enrichment Topology.</li>
+</ul>
+<p>Note also that there is a convenience script <tt>./run_enrichment_role.sh</tt> which executes Vagrant with the <tt>enrichment</tt> tag.</p></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/vagrant/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/vagrant/index.html b/site/current-book/metron-deployment/vagrant/index.html
new file mode 100644
index 0000000..373405e
--- /dev/null
+++ b/site/current-book/metron-deployment/vagrant/index.html
@@ -0,0 +1,255 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Vagrant Deployment</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Vagrant Deployment</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-right"></i>
+ Roles</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="icon-chevron-down"></i>Vagrant</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../metron-deployment/vagrant/codelab-platform/index.html" title="Codelab-platform">
+ <i class="none"></i>
+ Codelab-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/vagrant/fastcapa-test-platform/index.html" title="Fastcapa-test-platform">
+ <i class="none"></i>
+ Fastcapa-test-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/vagrant/full-dev-platform/index.html" title="Full-dev-platform">
+ <i class="none"></i>
+ Full-dev-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/vagrant/quick-dev-platform/index.html" title="Quick-dev-platform">
+ <i class="none"></i>
+ Quick-dev-platform</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Vagrant Deployment</h1>
+<p><a name="Vagrant_Deployment"></a></p>
+
+<ul>
+
+<li>Codelab Platform</li>
+
+<li>Fast CAPA Test Platform</li>
+
+<li>Full Dev Platform</li>
+
+<li>Quick Dev Platform</li>
+</ul>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html b/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html
new file mode 100644
index 0000000..123efc6
--- /dev/null
+++ b/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html
@@ -0,0 +1,327 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Quick Development Platform</title>
+ <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../../css/site.css" />
+ <link rel="stylesheet" href="../../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Quick Development Platform</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-down"></i>
+ Deployment</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
+ <i class="none"></i>
+ Amazon-ec2</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
+ <i class="none"></i>
+ Ansible-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
+ <i class="none"></i>
+ Rpm-docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/packer-build/index.html" title="Packer-build">
+ <i class="none"></i>
+ Packer-build</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/roles/index.html" title="Roles">
+ <i class="icon-chevron-right"></i>
+ Roles</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
+ <i class="icon-chevron-down"></i>
+ Vagrant</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/codelab-platform/index.html" title="Codelab-platform">
+ <i class="none"></i>
+ Codelab-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/fastcapa-test-platform/index.html" title="Fastcapa-test-platform">
+ <i class="none"></i>
+ Fastcapa-test-platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-deployment/vagrant/full-dev-platform/index.html" title="Full-dev-platform">
+ <i class="none"></i>
+ Full-dev-platform</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Quick-dev-platform</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Quick Development Platform</h1>
+<p>This project fully automates the provisioning and deployment of Apache Metron and all necessary prerequisites on a single, virtualized host running on Virtualbox. </p>
+<p>This image is designed for quick deployment of a single node Metron cluster running on Virtualbox. This platform is ideal for use by Metron developers. It uses a base image that has been pre-loaded with Ambari and HDP.</p>
+<p>Metron is composed of many components and installing all of these on a single host, especially a virtualized one, will greatly stress the resources of the host. The host will require at least 8 GB of RAM and a fair amount of patience. It is highly recommended that you shut down all unnecessary services. To that end the vagrant file configuration defaults to disabling solr and yaf.</p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<p>As with the Full Development Platform (<tt>metron-deployment/vagrant/full-dev-platform</tt>), the computer used to deploy Apache Metron will need the following components installed.</p>
+
+<ul>
+
+<li><a class="externalLink" href="https://github.com/ansible/ansible">Ansible</a> 2.0.0.2</li>
+
+<li><a class="externalLink" href="https://www.vagrantup.com">Vagrant</a> 1.8.1</li>
+
+<li><a class="externalLink" href="https://virtualbox.org">Virtualbox</a> 5.0.16</li>
+
+<li>Python 2.7.11</li>
+
+<li>Maven 3.3.9</li>
+</ul></div>
+<div class="section">
+<h3><a name="Deploy_Metron"></a>Deploy Metron</h3>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Build Metron</p>
+
+<div class="source">
+<div class="source">
+<pre> cd incubator-metron
+ mvn clean package -DskipTests
+</pre></div></div></li>
+
+<li>
+<p>Install Vagrant Hostmanager.</p>
+
+<div class="source">
+<div class="source">
+<pre> vagrant plugin install vagrant-hostmanager
+</pre></div></div></li>
+
+<li>
+<p>Deploy Metron</p>
+
+<div class="source">
+<div class="source">
+<pre> cd metron-deployment/vagrant/quick-dev-platform
+ vagrant up
+</pre></div></div>
+<p>Should the process fail before completing the deployment, the following command will continue the deployment process without re-instantiating the host.</p>
+
+<div class="source">
+<div class="source">
+<pre> vagrant provision
+</pre></div></div></li>
+</ol></div>
+<div class="section">
+<h3><a name="Explore_Metron"></a>Explore Metron</h3>
+<p>Navigate to the following resources to explore your newly minted Apache Metron environment.</p>
+
+<ul>
+
+<li><a class="externalLink" href="http://node1:8080">Ambari</a></li>
+
+<li><a class="externalLink" href="http://node1:5000">Metron</a></li>
+
+<li><a class="externalLink" href="http://node1:2812">Services</a></li>
+</ul></div>
+<div class="section">
+<h3><a name="Working_with_Metron"></a>Working with Metron</h3>
+<p>As you build out new capabilities for Metron, you will need to re-deploy the Storm topologies. To do so, first HALT the running Storm topologies and then run the following command.</p>
+
+<div class="source">
+<div class="source">
+<pre>./run_enrichment_role.sh
+</pre></div></div>
+<p>Connecting to the host through SSH is as simple as running the following command.</p>
+
+<div class="source">
+<div class="source">
+<pre>vagrant ssh
+</pre></div></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-docker/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-docker/index.html b/site/current-book/metron-docker/index.html
new file mode 100644
index 0000000..a87945b
--- /dev/null
+++ b/site/current-book/metron-docker/index.html
@@ -0,0 +1,470 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Metron Docker</title>
+ <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../css/site.css" />
+ <link rel="stylesheet" href="../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Metron Docker</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Metron Docker</h1>
+<p><a name="Metron_Docker"></a></p>
+<p>Metron Docker is a <a class="externalLink" href="https://docs.docker.com/compose/overview/">Docker Compose</a> application that is intended for development and integration testing of Metron. Use this instead of Vagrant when:</p>
+
+<ul>
+
+<li>You want an environment that can be built and spun up quickly</li>
+
+<li>You need to frequently rebuild and restart services</li>
+
+<li>You only need to test, troubleshoot or develop against a subset of services</li>
+</ul>
+<p>Metron Docker includes these images that have been customized for Metron:</p>
+
+<ul>
+
+<li>Kafka (with Zookeeper)</li>
+
+<li>HBase</li>
+
+<li>Storm (with all topologies deployed)</li>
+
+<li>Elasticsearch</li>
+
+<li>Kibana</li>
+</ul>
+<div class="section">
+<h2><a name="Setup"></a>Setup</h2>
+<p>Install <a class="externalLink" href="https://docs.docker.com/docker-for-mac/">Docker for Mac</a> or <a class="externalLink" href="https://docs.docker.com/docker-for-windows/">Docker for Windows</a>. The following versions have been tested:</p>
+
+<ul>
+
+<li>Docker version 1.12.0</li>
+
+<li>docker-machine version 0.8.0</li>
+
+<li>docker-compose version 1.8.0</li>
+</ul>
+<p>Build Metron from the top level directory with:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_HOME
+$ mvn clean install -DskipTests
+</pre></div></div>
+<p>You are welcome to use an existing Docker host but we prefer one with more resources. You can create one of those with this script:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ export METRON_DOCKER_HOME=$METRON_HOME/metron-docker
+$ cd $METRON_DOCKER_HOME && ./scripts/create-docker-machine.sh
+</pre></div></div>
+<p>This will create a host called “metron-machine”. Anytime you want to run Docker commands against this host, make sure you run this first to set the Docker environment variables:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ eval "$(docker-machine env metron-machine)"
+</pre></div></div>
+<p>If you wish to use a local docker-engine install, please set an environment variable BROKER_IP_ADDR to the IP address of your host machine. This cannot be the loopback address.</p></div>
+<div class="section">
+<h2><a name="Usage"></a>Usage</h2>
+<p>Navigate to the compose application root:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_DOCKER_HOME/compose/
+</pre></div></div>
+<p>The Metron Docker environment lifecycle is controlled by the <a class="externalLink" href="https://docs.docker.com/compose/reference/overview/">docker-compose</a> command. The service names can be found in the docker-compose.yml file. For example, to build and start the environment run this command:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ eval "$(docker-machine env metron-machine)"
+$ docker-compose up -d
+</pre></div></div>
+<p>After all services have started list the containers and ensure their status is ‘Up’:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker ps --format 'table {{.Names}}\t{{.Status}}'
+NAMES STATUS
+metron_storm_1 Up 5 minutes
+metron_hbase_1 Up 5 minutes
+metron_kibana_1 Up 5 minutes
+metron_kafkazk_1 Up 5 minutes
+metron_elasticsearch_1 Up 5 minutes
+</pre></div></div>
+<p>Various services are exposed through http on the Docker host. Get the host ip from the URL property:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-machine ls
+NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
+metron-machine * virtualbox Running tcp://192.168.99.100:2376 v1.12.5
+</pre></div></div>
+<p>Then, assuming a host ip of <tt>192.168.99.100</tt>, the UIs and APIs are available at:</p>
+
+<ul>
+
+<li>Storm - <a class="externalLink" href="http://192.168.99.100:8080/">http://192.168.99.100:8080/</a></li>
+
+<li>HBase - <a class="externalLink" href="http://192.168.99.100:16010/">http://192.168.99.100:16010/</a></li>
+
+<li>Elasticsearch - <a class="externalLink" href="http://192.168.99.100:9200/">http://192.168.99.100:9200/</a></li>
+
+<li>Kibana - <a class="externalLink" href="http://192.168.99.100:5601/">http://192.168.99.100:5601/</a></li>
+</ul>
+<p>The Storm logs can be useful when troubleshooting topologies. They can be found on the Storm container in <tt>/usr/share/apache-storm/logs</tt>.</p>
+<p>When done using the machine, shut it down with:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose down
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Examples"></a>Examples</h2>
+
+<ul>
+
+<li><a href="#Deploy_a_new_parser_class">Deploy a new parser class</a></li>
+
+<li><a href="#Connect_to_a_container">Connect to a container</a></li>
+
+<li><a href="create-a-sensor-from-sample-data/index.html">Create a sensor from sample data</a></li>
+
+<li><a href="upload-configs-to-zookeeper/index.html">Upload configs to Zookeeper</a></li>
+
+<li><a href="manage-a-topology/index.html">Manage a topology</a></li>
+
+<li><a href="run-sensor-data-end-to-end/index.html">Run sensor data end to end</a></li>
+</ul>
+<div class="section">
+<h3><a name="Deploy_a_new_parser_class"></a>Deploy a new parser class</h3>
+<p>After adding a new parser to metron-parsers, build Metron from the top level directory:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_HOME
+$ mvn clean install -DskipTests
+</pre></div></div>
+<p>Then run these commands to redeploy the parsers to the Storm image:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_DOCKER_HOME/compose
+$ docker-compose down
+$ docker-compose build storm
+$ docker-compose up -d
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Connect_to_a_container"></a>Connect to a container</h3>
+<p>Suppose there is a problem with Kafka and the logs are needed for further investigation. Run this command to connect and explore the running Kafka container:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_DOCKER_HOME/compose
+$ docker-compose exec kafkazk bash
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Create_a_sensor_from_sample_data"></a>Create a sensor from sample data</h3>
+<p>A tool for producing test data in Kafka is included with the Kafka/Zookeeper image. It loops through lines in a test data file and outputs them to Kafka at the desired frequency. Create a test data file in <tt>./kafkazk/data/</tt> and rebuild the Kafka/Zookeeper image:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_DOCKER_HOME/compose
+$ printf 'first test data\nsecond test data\nthird test data\n' > ./kafkazk/data/TestData.txt
+$ docker-compose down
+$ docker-compose build kafkazk
+$ docker-compose up -d
+</pre></div></div>
+<p>This will deploy the test data file to the Kafka/Zookeeper container. Now that data can be streamed to a Kafka topic:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose exec kafkazk ./bin/produce-data.sh
+Usage: produce-data.sh data_path topic [message_delay_in_seconds]
+
+# Stream data in TestData.txt to the 'test' Kafka topic at a frequency of 5 seconds (default is 1 second)
+$ docker-compose exec kafkazk ./bin/produce-data.sh /data/TestData.txt test 5
+</pre></div></div>
+<p>The Kafka/Zookeeper image comes with sample Bro and Squid data:</p>
+
+<div class="source">
+<div class="source">
+<pre># Stream Bro test data every 1 second
+$ docker-compose exec kafkazk ./bin/produce-data.sh /data/BroExampleOutput.txt bro
+
+# Stream Squid test data every 0.1 seconds
+$ docker-compose exec kafkazk ./bin/produce-data.sh /data/SquidExampleOutput.txt squid 0.1
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Upload_configs_to_Zookeeper"></a>Upload configs to Zookeeper</h3>
+<p>Parser configs and a global config configured for this Docker environment are included with the Kafka/Zookeeper image. Load them with:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose exec kafkazk bash
+# $METRON_HOME/bin/zk_load_configs.sh -z localhost:2181 -m PUSH -i $METRON_HOME/config/zookeeper
+# exit
+</pre></div></div>
+<p>Dump out the configs with:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose exec kafkazk bash
+# $METRON_HOME/bin/zk_load_configs.sh -z localhost:2181 -m DUMP
+# exit
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Manage_a_topology"></a>Manage a topology</h3>
+<p>The Storm image comes with a script to easily start parser topologies:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm ./bin/start_docker_parser_topology.sh sensor_name
+</pre></div></div>
+<p>The enrichment topology can be started with:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm ./bin/start_enrichment_topology.sh
+</pre></div></div>
+<p>The indexing topology can be started with:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm ./bin/start_elasticsearch_topology.sh
+</pre></div></div>
+<p>Topologies can be stopped using the Storm CLI. For example, stop the enrichment topology with:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm storm kill enrichments -w 0
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Run_sensor_data_end_to_end"></a>Run sensor data end to end</h3>
+<p>First ensure configs were uploaded as described in the previous example. Then start a sensor and leave it running:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ cd $METRON_DOCKER_HOME/compose
+$ docker-compose exec kafkazk ./bin/produce-data.sh /data/BroExampleOutput.txt bro
+</pre></div></div>
+<p>Open a separate console session and verify the sensor is running by consuming a message from Kafka:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ export METRON_DOCKER_HOME=$METRON_HOME/metron-docker
+$ cd $METRON_DOCKER_HOME/compose
+$ docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic bro
+</pre></div></div>
+<p>A new message should be printed every second. Now kill the consumer and start the Bro parser topology:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose exec storm ./bin/start_docker_parser_topology.sh bro
+</pre></div></div>
+<p>Bro data should be flowing through the bro parser topology and into the Kafka enrichments topic. The enrichments topic should be created automatically:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-compose exec kafkazk ./bin/kafka-topics.sh --zookeeper localhost:2181 --list
+bro
+enrichments
+indexing
+</pre></div></div>
+<p>Verify parsed Bro data is in the Kafka enrichments topic:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic enrichments
+</pre></div></div>
+<p>Now start the enrichment topology:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm ./bin/start_enrichment_topology.sh
+</pre></div></div>
+<p>Parsed Bro data should be flowing through the enrichment topology and into the Kafka indexing topic. Verify enriched Bro data is in the Kafka indexing topic:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic indexing
+</pre></div></div>
+<p>Now start the indexing topology:</p>
+
+<div class="source">
+<div class="source">
+<pre>docker-compose exec storm ./bin/start_elasticsearch_topology.sh
+</pre></div></div>
+<p>Enriched Bro data should now be present in the Elasticsearch container:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ docker-machine ls
+NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
+metron-machine * virtualbox Running tcp://192.168.99.100:2376 v1.12.5
+
+$ curl -XGET http://192.168.99.100:9200/_cat/indices?v
+health status index pri rep docs.count docs.deleted store.size pri.store.size
+yellow open .kibana 1 1 1 0 3.1kb 3.1kb
+yellow open bro_index_2016.12.19.18 5 1 180 0 475kb 475kb
+</pre></div></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-platform/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/index.html b/site/current-book/metron-platform/index.html
new file mode 100644
index 0000000..0546ef1
--- /dev/null
+++ b/site/current-book/metron-platform/index.html
@@ -0,0 +1,249 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Current Build</title>
+ <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../css/site.css" />
+ <link rel="stylesheet" href="../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Current Build</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><i class="icon-chevron-down"></i>Platform</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../metron-platform/metron-api/index.html" title="Api">
+ <i class="none"></i>
+ Api</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/metron-common/index.html" title="Common">
+ <i class="none"></i>
+ Common</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/metron-data-management/index.html" title="Data-management">
+ <i class="none"></i>
+ Data-management</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/metron-enrichment/index.html" title="Enrichment">
+ <i class="none"></i>
+ Enrichment</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/metron-indexing/index.html" title="Indexing">
+ <i class="none"></i>
+ Indexing</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/metron-management/index.html" title="Management">
+ <i class="none"></i>
+ Management</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/metron-parsers/index.html" title="Parsers">
+ <i class="none"></i>
+ Parsers</a>
+ </li>
+
+ <li>
+
+ <a href="../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
+ <i class="none"></i>
+ Pcap-backend</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <!-- Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License. --><h1>Current Build</h1>
+<p><a name="Current_Build"></a></p>
+<p>The latest build of metron-platform is 0.3.1.</p>
+<p>We are still in the process of merging/porting additional features from our production code base into this open source release. This release will be followed by a number of additional beta releases until the port is complete. We will also work on getting additional documentation and user/developer guides to the community as soon as we can. At this time we offer no support for the beta software, but will try to respond to requests as promptly as we can.</p>
+<p><a name="metron-platform"></a></p>
+<h1>metron-platform</h1>
+<p>Extensible set of Storm topologies and topology attributes for streaming, enriching, indexing, and storing telemetry in Hadoop. General information on Metron is available at <a class="externalLink" href="https://metron.incubator.apache.org/">https://metron.incubator.apache.org/</a></p>
+<p><a name="Documentation"></a></p>
+<h1>Documentation</h1>
+<p>Please see documentation within each individual module for description and usage instructions. Sample topologies are provided under Metron_Topologies to get you started with the framework. We pre-assume knowledge of Hadoop, Storm, Kafka, and HBase.</p>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/39059e5b/site/current-book/metron-platform/metron-api/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-api/index.html b/site/current-book/metron-platform/metron-api/index.html
new file mode 100644
index 0000000..fe2b9a6
--- /dev/null
+++ b/site/current-book/metron-platform/metron-api/index.html
@@ -0,0 +1,285 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-02-23
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170223" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Metron PCAP Service</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="http://metron.incubator.apache.org/" id="bannerLeft">
+ <img src="../../images/metron-logo.png" alt="Apache Metron - Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a href="http://incubator.apache.org/" id="bannerRight">
+ <img src="../../images/ApacheIncubating_Logo.png" alt="Apache Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org" class="externalLink" title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/" class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="../../index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Metron PCAP Service</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published: 2017-02-23</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.3.1</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="../../index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../../Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-analytics/index.html" title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-deployment/index.html" title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/index.html" title="Platform">
+ <i class="icon-chevron-down"></i>
+ Platform</a>
+ <ul class="nav nav-list">
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Api</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-common/index.html" title="Common">
+ <i class="none"></i>
+ Common</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
+ <i class="none"></i>
+ Data-management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
+ <i class="none"></i>
+ Enrichment</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
+ <i class="none"></i>
+ Indexing</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-management/index.html" title="Management">
+ <i class="none"></i>
+ Management</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
+ <i class="none"></i>
+ Parsers</a>
+ </li>
+
+ <li>
+
+ <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
+ <i class="none"></i>
+ Pcap-backend</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../../metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Metron PCAP Service</h1>
+<p><a name="Metron_PCAP_Service"></a></p>
+<p>The purpose of the Metron PCAP service is to provide a middle tier to negotiate retrieving packet capture data which flows into Metron. This packet data is of a form which <tt>libpcap</tt> based tools can read.</p>
+<div class="section">
+<h2><a name="Starting_the_Service"></a>Starting the Service</h2>
+<p>You can start the service either via the init.d script installed, <tt>/etc/init.d/pcapservice</tt> or directly via the <tt>yarn jar</tt> command: <tt>yarn jar $METRON_HOME/lib/metron-api-$METRON_VERSION.jar org.apache.metron.pcapservice.rest.PcapService -port $SERVICE_PORT -query_hdfs_path $QUERY_PATH -pcap_hdfs_path $PCAP_PATH</tt></p>
+<p>where</p>
+
+<ul>
+
+<li><tt>METRON_HOME</tt> is the location of the metron installation</li>
+
+<li><tt>METRON_VERSION</tt> is the version of the metron installation</li>
+
+<li><tt>SERVICE_PORT</tt> is the port to bind the REST service to.</li>
+
+<li><tt>QUERY_PATH</tt> is the temporary location to store query results. They are deleted after the service reads them.</li>
+
+<li><tt>PCAP_PATH</tt> is the path to the packet data on HDFS</li>
+</ul></div>
+<div class="section">
+<h2><a name="The_pcapGettergetPcapsByIdentifiers_endpoint"></a>The <tt>/pcapGetter/getPcapsByIdentifiers</tt> endpoint</h2>
+<p>This endpoint takes the following query parameters and returns the subset of packets matching this query:</p>
+
+<ul>
+
+<li><tt>srcIp</tt> : The source IP to match on</li>
+
+<li><tt>srcPort</tt> : The source port to match on</li>
+
+<li><tt>dstIp</tt> : The destination IP to match on</li>
+
+<li><tt>dstPort</tt> : The destination port to match on</li>
+
+<li><tt>startTime</tt> : The start time in milliseconds</li>
+
+<li><tt>endTime</tt> : The end time in milliseconds</li>
+
+<li><tt>numReducers</tt> : Specify the number of reducers to use when executing the mapreduce job</li>
+
+<li><tt>includeReverseTraffic</tt> : Indicates if filter should check swapped src/dest addresses and IPs</li>
+</ul></div>
+<div class="section">
+<h2><a name="The_pcapGettergetPcapsByQuery_endpoint"></a>The <tt>/pcapGetter/getPcapsByQuery</tt> endpoint</h2>
+<p>This endpoint takes the following query parameters and returns the subset of packets matching this query. This endpoint exposes Stellar querying capabilities:</p>
+
+<ul>
+
+<li><tt>query</tt> : The Stellar query to execute</li>
+
+<li><tt>startTime</tt> : The start time in milliseconds</li>
+
+<li><tt>endTime</tt> : The end time in milliseconds</li>
+
+<li><tt>numReducers</tt> : Specify the number of reducers to use when executing the mapreduce job</li>
+</ul>
+<p>Example: <tt>curl -XGET "http://node1:8081/pcapGetter/getPcapsByQuery?query=ip_src_addr+==+'192.168.66.121'+and+ip_src_port+==+'60500'&startTime=1476936000000"</tt></p>
+<p>All of these parameters are optional. In the case of a missing parameter, it is treated as a wildcard.</p>
+<p>Unlike the CLI tool, there is no paging mechanism. The REST API will stream back data as a single file.</p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>