You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@metron.apache.org by mm...@apache.org on 2018/06/11 15:45:07 UTC

[01/50] [abbrv] metron git commit: METRON-1592 Unable to use third party parser with Storm versions >= 1.1.0 (nickwallen) closes apache/metron#1042

Repository: metron
Updated Branches:
  refs/heads/feature/METRON-1554-pcap-query-panel b9453aabd -> 77570464b


METRON-1592 Unable to use third party parser with Storm versions >= 1.1.0 (nickwallen) closes apache/metron#1042


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/dbcc52cd
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/dbcc52cd
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/dbcc52cd

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: dbcc52cd658a07a923b656c07c16a5c15f697ec6
Parents: 939eb73
Author: nickwallen <ni...@nickallen.org>
Authored: Tue Jun 5 11:04:35 2018 -0400
Committer: nickallen <ni...@apache.org>
Committed: Tue Jun 5 11:04:35 2018 -0400

----------------------------------------------------------------------
 .../metron-parsers/3rdPartyParser.md            | 439 ++++++++++---------
 .../src/main/scripts/start_parser_topology.sh   |  27 +-
 2 files changed, 246 insertions(+), 220 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/dbcc52cd/metron-platform/metron-parsers/3rdPartyParser.md
----------------------------------------------------------------------
diff --git a/metron-platform/metron-parsers/3rdPartyParser.md b/metron-platform/metron-parsers/3rdPartyParser.md
index 61095ea..739328f 100644
--- a/metron-platform/metron-parsers/3rdPartyParser.md
+++ b/metron-platform/metron-parsers/3rdPartyParser.md
@@ -52,182 +52,185 @@ fields:
 For this demonstration, let's create a maven project to compile our
 project.  We'll call it `extra_parsers`, so in your workspace, let's set
 up the maven project:
+
 * Create the maven infrastructure for `extra_parsers` via
-```
-mkdir -p extra_parsers/src/{main,test}/java
-```
+  ```
+  mkdir -p extra_parsers/src/{main,test}/java
+  ```
+
 * Create a pom file indicating how we should build our parsers by
   editing `extra_parsers/pom.xml` with the following content:
-```
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
-  <modelVersion>4.0.0</modelVersion>
-  <groupId>com.3rdparty</groupId>
-  <artifactId>extra-parsers</artifactId>
-  <packaging>jar</packaging>
-  <version>1.0-SNAPSHOT</version>
-  <name>extra-parsers</name>
-  <url>http://thirdpartysoftware.org</url>
-  <properties>
-    <!-- The java version to conform to.  Metron works all the way to 1.8 -->
-    <java_version>1.8</java_version>
-    <!-- The version of Metron that we'll be targetting. -->
-    <metron_version>0.4.1</metron_version>
-    <!-- To complete the simulation, we'll depend on a common dependency -->
-    <guava_version>19.0</guava_version>
-    <!-- We will shade our dependencies to create a single jar at the end -->
-    <shade_version>2.4.3</shade_version>
-  </properties>
-  <dependencies>
-    <!--
-    We want to depend on Metron, but ensure that the scope is "provided"
-    as we do not want to include it in our bundle.
-    -->
-    <dependency>
-      <groupId>org.apache.metron</groupId>
-      <artifactId>metron-parsers</artifactId>
-      <version>${metron_version}</version>
-      <scope>provided</scope>
-    </dependency>
-    <dependency>
-      <groupId>com.google.guava</groupId>
-      <artifactId>guava</artifactId>
-      <version>${guava_version}</version>
-    </dependency>
-    <dependency>
-      <groupId>junit</groupId>
-      <artifactId>junit</artifactId>
-      <version>3.8.1</version>
-      <scope>test</scope>
-    </dependency>
-  </dependencies>
-  <build>
-    <plugins>
-     <!-- We will set up the shade plugin to create a single jar at the
-           end of the build lifecycle.  We will exclude some things and
-           relocate others to simulate a real situation.
-           
-           One thing to note is that it's a good practice to shade and
-           relocate common libraries that may be dependencies in Metron.
-           Your jar will be merged with the parsers jar, so the metron
-           version will be included for all overlapping classes.
-           So, shade and relocate to ensure that YOUR version of the library is used.
+  ```
+  <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>com.3rdparty</groupId>
+    <artifactId>extra-parsers</artifactId>
+    <packaging>jar</packaging>
+    <version>1.0-SNAPSHOT</version>
+    <name>extra-parsers</name>
+    <url>http://thirdpartysoftware.org</url>
+    <properties>
+      <!-- The java version to conform to.  Metron works all the way to 1.8 -->
+      <java_version>1.8</java_version>
+      <!-- The version of Metron that we'll be targetting. -->
+      <metron_version>0.4.1</metron_version>
+      <!-- To complete the simulation, we'll depend on a common dependency -->
+      <guava_version>19.0</guava_version>
+      <!-- We will shade our dependencies to create a single jar at the end -->
+      <shade_version>2.4.3</shade_version>
+    </properties>
+    <dependencies>
+      <!--
+      We want to depend on Metron, but ensure that the scope is "provided"
+      as we do not want to include it in our bundle.
       -->
-
-      <plugin>
-        <groupId>org.apache.maven.plugins</groupId>
-        <artifactId>maven-shade-plugin</artifactId>
-        <version>${shade_version}</version>
-        <configuration>
-          <createDependencyReducedPom>true</createDependencyReducedPom>
-          <artifactSet>
-            <excludes>
-              <!-- Exclude slf4j for no reason other than to illustrate how to exclude dependencies.
-                   The metron team has nothing against slf4j. :-)
-               -->
-              <exclude>*slf4j*</exclude>
-            </excludes>
-          </artifactSet>
-        </configuration>
-        <executions>
-          <execution>
-            <phase>package</phase>
-            <goals>
-              <goal>shade</goal>
-            </goals>
-            <configuration>
-              <shadedArtifactAttached>true</shadedArtifactAttached>
-              <shadedClassifierName>uber</shadedClassifierName>
-              <filters>
-                <filter>
-                  <!-- Sometimes these get added and confuse the uber jar out of shade -->
-                  <artifact>*:*</artifact>
+      <dependency>
+        <groupId>org.apache.metron</groupId>
+        <artifactId>metron-parsers</artifactId>
+        <version>${metron_version}</version>
+        <scope>provided</scope>
+      </dependency>
+      <dependency>
+        <groupId>com.google.guava</groupId>
+        <artifactId>guava</artifactId>
+        <version>${guava_version}</version>
+      </dependency>
+      <dependency>
+        <groupId>junit</groupId>
+        <artifactId>junit</artifactId>
+        <version>3.8.1</version>
+        <scope>test</scope>
+      </dependency>
+    </dependencies>
+    <build>
+      <plugins>
+       <!-- We will set up the shade plugin to create a single jar at the
+             end of the build lifecycle.  We will exclude some things and
+             relocate others to simulate a real situation.
+
+             One thing to note is that it's a good practice to shade and
+             relocate common libraries that may be dependencies in Metron.
+             Your jar will be merged with the parsers jar, so the metron
+             version will be included for all overlapping classes.
+             So, shade and relocate to ensure that YOUR version of the library is used.
+        -->
+
+        <plugin>
+          <groupId>org.apache.maven.plugins</groupId>
+          <artifactId>maven-shade-plugin</artifactId>
+          <version>${shade_version}</version>
+          <configuration>
+            <createDependencyReducedPom>true</createDependencyReducedPom>
+            <artifactSet>
+              <excludes>
+                <!-- Exclude slf4j for no reason other than to illustrate how to exclude dependencies.
+                     The metron team has nothing against slf4j. :-)
+                 -->
+                <exclude>*slf4j*</exclude>
+              </excludes>
+            </artifactSet>
+          </configuration>
+          <executions>
+            <execution>
+              <phase>package</phase>
+              <goals>
+                <goal>shade</goal>
+              </goals>
+              <configuration>
+                <shadedArtifactAttached>true</shadedArtifactAttached>
+                <shadedClassifierName>uber</shadedClassifierName>
+                <filters>
+                  <filter>
+                    <!-- Sometimes these get added and confuse the uber jar out of shade -->
+                    <artifact>*:*</artifact>
+                    <excludes>
+                      <exclude>META-INF/*.SF</exclude>
+                      <exclude>META-INF/*.DSA</exclude>
+                      <exclude>META-INF/*.RSA</exclude>
+                    </excludes>
+                  </filter>
+                </filters>
+                <relocations>
+                  <!-- Relocate guava as it's used in Metron and I really want 0.19 -->
+                  <relocation>
+                    <pattern>com.google</pattern>
+                    <shadedPattern>com.thirdparty.guava</shadedPattern>
+                  </relocation>
+                </relocations>
+                <artifactSet>
                   <excludes>
-                    <exclude>META-INF/*.SF</exclude>
-                    <exclude>META-INF/*.DSA</exclude>
-                    <exclude>META-INF/*.RSA</exclude>
+                    <!-- We can also exclude by artifactId and groupId -->
+                    <exclude>storm:storm-core:*</exclude>
+                    <exclude>storm:storm-lib:*</exclude>
+                    <exclude>org.slf4j.impl*</exclude>
+                    <exclude>org.slf4j:slf4j-log4j*</exclude>
                   </excludes>
-                </filter>
-              </filters>
-              <relocations>
-                <!-- Relocate guava as it's used in Metron and I really want 0.19 -->
-                <relocation>
-                  <pattern>com.google</pattern>
-                  <shadedPattern>com.thirdparty.guava</shadedPattern>
-                </relocation>
-              </relocations>
-              <artifactSet>
-                <excludes>
-                  <!-- We can also exclude by artifactId and groupId -->
-                  <exclude>storm:storm-core:*</exclude>
-                  <exclude>storm:storm-lib:*</exclude>
-                  <exclude>org.slf4j.impl*</exclude>
-                  <exclude>org.slf4j:slf4j-log4j*</exclude>
-                </excludes>
-              </artifactSet>
-            </configuration>
-          </execution>
-        </executions>
-      </plugin>
-      <!--
-      We want to make sure we compile using java 1.8.
-      -->
-      <plugin>
-        <groupId>org.apache.maven.plugins</groupId>
-        <artifactId>maven-compiler-plugin</artifactId>
-        <version>3.5.1</version>
-        <configuration>
-          <forceJavacCompilerUse>true</forceJavacCompilerUse>
-          <source>${java_version}</source>
-          <compilerArgument>-Xlint:unchecked</compilerArgument>
-          <target>${java_version}</target>
-          <showWarnings>true</showWarnings>
-        </configuration>
-      </plugin>
-    </plugins>
-  </build>
-</project>
-```
-* Now let's create our parser  `com.thirdparty.SimpleParser` by creating the file `extra-parsers/src/main/java/com/thirdparty/SimpleParser.java` with the following content:
-```
-package com.thirdparty;
-
-import com.google.common.base.Splitter;
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.Iterables;
-import org.apache.metron.parsers.BasicParser;
-import org.json.simple.JSONObject;
-
-import java.util.List;
-import java.util.Map;
-
-public class SimpleParser extends BasicParser {
-  @Override
-  public void init() {
-
-  }
-
-  @Override
-  public List<JSONObject> parse(byte[] bytes) {
-    String input = new String(bytes);
-    Iterable<String> it = Splitter.on(",").split(input);
-    JSONObject ret = new JSONObject();
-    ret.put("original_string", input);
-    ret.put("timestamp", System.currentTimeMillis());
-    ret.put("first", Iterables.getFirst(it, "missing"));
-    ret.put("last", Iterables.getLast(it, "missing"));
-    return ImmutableList.of(ret);
-  }
-
-  @Override
-  public void configure(Map<String, Object> map) {
+                </artifactSet>
+              </configuration>
+            </execution>
+          </executions>
+        </plugin>
+        <!--
+        We want to make sure we compile using java 1.8.
+        -->
+        <plugin>
+          <groupId>org.apache.maven.plugins</groupId>
+          <artifactId>maven-compiler-plugin</artifactId>
+          <version>3.5.1</version>
+          <configuration>
+            <forceJavacCompilerUse>true</forceJavacCompilerUse>
+            <source>${java_version}</source>
+            <compilerArgument>-Xlint:unchecked</compilerArgument>
+            <target>${java_version}</target>
+            <showWarnings>true</showWarnings>
+          </configuration>
+        </plugin>
+      </plugins>
+    </build>
+  </project>
+  ```
 
+* Now let's create our parser  `com.thirdparty.SimpleParser` by creating the file `extra-parsers/src/main/java/com/thirdparty/SimpleParser.java` with the following content:
+  ```
+  package com.thirdparty;
+
+  import com.google.common.base.Splitter;
+  import com.google.common.collect.ImmutableList;
+  import com.google.common.collect.Iterables;
+  import org.apache.metron.parsers.BasicParser;
+  import org.json.simple.JSONObject;
+
+  import java.util.List;
+  import java.util.Map;
+
+  public class SimpleParser extends BasicParser {
+    @Override
+    public void init() {
+
+    }
+
+    @Override
+    public List<JSONObject> parse(byte[] bytes) {
+      String input = new String(bytes);
+      Iterable<String> it = Splitter.on(",").split(input);
+      JSONObject ret = new JSONObject();
+      ret.put("original_string", input);
+      ret.put("timestamp", System.currentTimeMillis());
+      ret.put("first", Iterables.getFirst(it, "missing"));
+      ret.put("last", Iterables.getLast(it, "missing"));
+      return ImmutableList.of(ret);
+    }
+
+    @Override
+    public void configure(Map<String, Object> map) {
+
+    }
   }
-}
-```
+  ```
 * Compile the parser via `mvn clean package` in `extra_parsers`
 
-This will create a jar containing your parser and its dependencies (sans Metron dependencies) in `extra-parsers/target/extra-parsers-1.0-SNAPSHOT-uber.jar`
+* This will create a jar containing your parser and its dependencies (sans Metron dependencies) in `extra-parsers/target/extra-parsers-1.0-SNAPSHOT-uber.jar`
 
 # Deploying Your Custom Parser
 
@@ -255,16 +258,13 @@ Copy the jar file located in `extra-parsers/target/extra-parsers-1.0-SNAPSHOT-ub
 
 In order for new parsers to be picked up, the REST service must be restarted.  You can do that from within Ambari by restarting the `Metron REST` service.
 
-### Push the Zookeeper Configs
-
-Now push the config to Zookeeper with the following command:
-`$METRON_HOME/bin/zk_load_configs.sh -m PUSH -i $METRON_HOME/config/zookeeper/ -z $ZOOKEEPER`
-
-
 ### Create a Kafka Topic
 
-Create a kafka topic, let's call it `test` via:
-`/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic test --partitions 1 --replication-factor 1`
+Create a kafka topic, let's call it `test`.
+```
+KAFKA_HOME=/usr/hdp/current/kafka-broker
+$KAFKA_HOME/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic test --partitions 1 --replication-factor 1
+```
 
 Note, in a real deployment, that topic would be named something more descriptive and would have replication factor and partitions set to something less trivial.
 
@@ -278,52 +278,63 @@ Create the a file called `$METRON_HOME/config/zookeeper/parsers/test.json` with
 }
 ```
 
+### Push the Zookeeper Configs
+
+Now push the config to Zookeeper with the following command.
+```
+$METRON_HOME/bin/zk_load_configs.sh -m PUSH -i $METRON_HOME/config/zookeeper/ -z $ZOOKEEPER
+```
+
 ### Start Parser
 Now we can start the parser and send some data through:
+
 * Start the parser
-```
-$METRON_HOME/bin/start_parser_topology.sh -k $BROKERLIST -z $ZOOKEEPER -s test
-```
+  ```
+  $METRON_HOME/bin/start_parser_topology.sh -k $BROKERLIST -z $ZOOKEEPER -s test
+  ```
+
 * Send example data through:
-```
-echo "apache,metron" | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list $BROKERLIST --topic test
-```
+  ```
+  echo "apache,metron" | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list $BROKERLIST --topic test
+  ```
+
 * Validate data was written in ES:
-```
-curl -XPOST "http://$ES_HOST/test*/_search?pretty" -d '
-{
-  "_source" : [ "original_string", "timestamp", "first", "last"]
-}
-'
-```
-This should yield something like:
-```
-{
-  "took" : 23,
-  "timed_out" : false,
-  "_shards" : {
-    "total" : 1,
-    "successful" : 1,
-    "failed" : 0
-  },
-  "hits" : {
-    "total" : 1,
-    "max_score" : 1.0,
-    "hits" : [ {
-      "_index" : "test_index_2017.10.04.17",
-      "_type" : "test_doc",
-      "_id" : "3ae4dd4d-8c09-4f2a-93c0-26ec5508baaa",
-      "_score" : 1.0,
-      "_source" : {
-        "original_string" : "apache,metron",
-        "last" : "metron",
-        "first" : "apache",
-        "timestamp" : 1507138373223
-      }
-    } ]
+  ```
+  curl -XPOST "http://$ES_HOST/test*/_search?pretty" -d '
+  {
+    "_source" : [ "original_string", "timestamp", "first", "last"]
   }
-}
-```
+  '
+  ```
+
+* This should yield something like:
+  ```
+  {
+    "took" : 23,
+    "timed_out" : false,
+    "_shards" : {
+      "total" : 1,
+      "successful" : 1,
+      "failed" : 0
+    },
+    "hits" : {
+      "total" : 1,
+      "max_score" : 1.0,
+      "hits" : [ {
+        "_index" : "test_index_2017.10.04.17",
+        "_type" : "test_doc",
+        "_id" : "3ae4dd4d-8c09-4f2a-93c0-26ec5508baaa",
+        "_score" : 1.0,
+        "_source" : {
+          "original_string" : "apache,metron",
+          "last" : "metron",
+          "first" : "apache",
+          "timestamp" : 1507138373223
+        }
+      } ]
+    }
+  }
+  ```
 
 ### Via the Management UI
 

http://git-wip-us.apache.org/repos/asf/metron/blob/dbcc52cd/metron-platform/metron-parsers/src/main/scripts/start_parser_topology.sh
----------------------------------------------------------------------
diff --git a/metron-platform/metron-parsers/src/main/scripts/start_parser_topology.sh b/metron-platform/metron-parsers/src/main/scripts/start_parser_topology.sh
index 00ac809..bbbeaff 100755
--- a/metron-platform/metron-parsers/src/main/scripts/start_parser_topology.sh
+++ b/metron-platform/metron-parsers/src/main/scripts/start_parser_topology.sh
@@ -1,5 +1,5 @@
 #!/bin/bash
-# 
+#
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -7,9 +7,9 @@
 # to you under the Apache License, Version 2.0 (the
 # "License"); you may not use this file except in compliance
 # with the License.  You may obtain a copy of the License at
-# 
+#
 #     http://www.apache.org/licenses/LICENSE-2.0
-# 
+#
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -20,10 +20,25 @@ METRON_VERSION=${project.version}
 METRON_HOME=/usr/metron/$METRON_VERSION
 TOPOLOGY_JAR=metron-parsers-$METRON_VERSION-uber.jar
 PARSER_CONTRIB=${PARSER_CONTRIB:-$METRON_HOME/parser_contrib}
+EXTRA_ARGS="$@"
+
 if [ -d "$PARSER_CONTRIB" ]; then
+
   export STORM_EXT_CLASSPATH=$METRON_HOME/lib/$TOPOLOGY_JAR
   export EXTRA_JARS=$(ls -m $PARSER_CONTRIB/*.jar | tr -d ' ' | tr -d '\n' | sed 's/\/\//\//g')
-  storm jar $METRON_HOME/lib/$TOPOLOGY_JAR org.apache.metron.parsers.topology.ParserTopologyCLI "$@" -c client.jartransformer.class=org.apache.metron.parsers.topology.MergeAndShadeTransformer
-else
-  storm jar $METRON_HOME/lib/$TOPOLOGY_JAR org.apache.metron.parsers.topology.ParserTopologyCLI "$@"
+
+  STORM_VERSION=`storm version | grep Storm | awk '{print $2}' | cut -d. -f1,2,3`
+  STORM_COMPAT_CUTOFF="1.1.0"
+  STORM_COMPAT_VERSION=`printf "$STORM_COMPAT_CUTOFF\n$STORM_VERSION" | sort -V | head -n1`
+
+  if [ "$STORM_COMPAT_VERSION" = "$STORM_COMPAT_CUTOFF" ]; then
+    echo "Third party parser support for Storm >= 1.1.0; found Storm $STORM_VERSION"
+    EXTRA_ARGS+=" --jars $EXTRA_JARS"
+  else
+    echo "Legacy third party parser support for Storm < 1.1.0; found Storm $STORM_VERSION"
+    EXTRA_ARGS+=" -c client.jartransformer.class=org.apache.metron.parsers.topology.MergeAndShadeTransformer"
+  fi
 fi
+
+echo "Submitting parser topology; args='$EXTRA_ARGS'"
+storm jar $METRON_HOME/lib/$TOPOLOGY_JAR org.apache.metron.parsers.topology.ParserTopologyCLI $EXTRA_ARGS


[39/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/js/apache-maven-fluido-1.7.min.js
----------------------------------------------------------------------
diff --git a/site/current-book/js/apache-maven-fluido-1.7.min.js b/site/current-book/js/apache-maven-fluido-1.7.min.js
new file mode 100644
index 0000000..1324fb4
--- /dev/null
+++ b/site/current-book/js/apache-maven-fluido-1.7.min.js
@@ -0,0 +1,25 @@
+/*!
+ * jQuery JavaScript Library v1.11.2
+ * http://jquery.com/
+ *
+ * Includes Sizzle.js
+ * http://sizzlejs.com/
+ *
+ * Copyright 2005, 2014 jQuery Foundation, Inc. and other contributors
+ * Released under the MIT license
+ * http://jquery.org/license
+ *
+ * Date: 2014-12-17T15:27Z
+ */
+(function(b,a){if(typeof module==="object"&&typeof module.exports==="object"){module.exports=b.document?a(b,true):function(c){if(!c.document){throw new Error("jQuery requires a window with a document")}return a(c)}}else{a(b)}}(typeof window!=="undefined"?window:this,function(a5,av){var aP=[];var P=aP.slice;var az=aP.concat;var x=aP.push;var bU=aP.indexOf;var ac={};var y=ac.toString;var K=ac.hasOwnProperty;var D={};var ai="1.11.2",bI=function(e,i){return new bI.fn.init(e,i)},E=/^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g,bS=/^-ms-/,aW=/-([\da-z])/gi,O=function(e,i){return i.toUpperCase()};bI.fn=bI.prototype={jquery:ai,constructor:bI,selector:"",length:0,toArray:function(){return P.call(this)},get:function(e){return e!=null?(e<0?this[e+this.length]:this[e]):P.call(this)},pushStack:function(e){var i=bI.merge(this.constructor(),e);i.prevObject=this;i.context=this.context;return i},each:function(i,e){return bI.each(this,i,e)},map:function(e){return this.pushStack(bI.map(this,function(b7,b6){retu
 rn e.call(b7,b6,b7)}))},slice:function(){return this.pushStack(P.apply(this,arguments))},first:function(){return this.eq(0)},last:function(){return this.eq(-1)},eq:function(b7){var e=this.length,b6=+b7+(b7<0?e:0);return this.pushStack(b6>=0&&b6<e?[this[b6]]:[])},end:function(){return this.prevObject||this.constructor(null)},push:x,sort:aP.sort,splice:aP.splice};bI.extend=bI.fn.extend=function(){var e,cb,b6,b7,ce,cc,ca=arguments[0]||{},b9=1,b8=arguments.length,cd=false;if(typeof ca==="boolean"){cd=ca;ca=arguments[b9]||{};b9++}if(typeof ca!=="object"&&!bI.isFunction(ca)){ca={}}if(b9===b8){ca=this;b9--}for(;b9<b8;b9++){if((ce=arguments[b9])!=null){for(b7 in ce){e=ca[b7];b6=ce[b7];if(ca===b6){continue}if(cd&&b6&&(bI.isPlainObject(b6)||(cb=bI.isArray(b6)))){if(cb){cb=false;cc=e&&bI.isArray(e)?e:[]}else{cc=e&&bI.isPlainObject(e)?e:{}}ca[b7]=bI.extend(cd,cc,b6)}else{if(b6!==undefined){ca[b7]=b6}}}}}return ca};bI.extend({expando:"jQuery"+(ai+Math.random()).replace(/\D/g,""),isReady:true,err
 or:function(e){throw new Error(e)},noop:function(){},isFunction:function(e){return bI.type(e)==="function"},isArray:Array.isArray||function(e){return bI.type(e)==="array"},isWindow:function(e){return e!=null&&e==e.window},isNumeric:function(e){return !bI.isArray(e)&&(e-parseFloat(e)+1)>=0},isEmptyObject:function(i){var e;for(e in i){return false}return true},isPlainObject:function(b7){var i;if(!b7||bI.type(b7)!=="object"||b7.nodeType||bI.isWindow(b7)){return false}try{if(b7.constructor&&!K.call(b7,"constructor")&&!K.call(b7.constructor.prototype,"isPrototypeOf")){return false}}catch(b6){return false}if(D.ownLast){for(i in b7){return K.call(b7,i)}}for(i in b7){}return i===undefined||K.call(b7,i)},type:function(e){if(e==null){return e+""}return typeof e==="object"||typeof e==="function"?ac[y.call(e)]||"object":typeof e},globalEval:function(e){if(e&&bI.trim(e)){(a5.execScript||function(i){a5["eval"].call(a5,i)})(e)}},camelCase:function(e){return e.replace(bS,"ms-").replace(aW,O)},nodeN
 ame:function(i,e){return i.nodeName&&i.nodeName.toLowerCase()===e.toLowerCase()},each:function(ca,cb,b6){var b9,b7=0,b8=ca.length,e=ad(ca);if(b6){if(e){for(;b7<b8;b7++){b9=cb.apply(ca[b7],b6);if(b9===false){break}}}else{for(b7 in ca){b9=cb.apply(ca[b7],b6);if(b9===false){break}}}}else{if(e){for(;b7<b8;b7++){b9=cb.call(ca[b7],b7,ca[b7]);if(b9===false){break}}}else{for(b7 in ca){b9=cb.call(ca[b7],b7,ca[b7]);if(b9===false){break}}}}return ca},trim:function(e){return e==null?"":(e+"").replace(E,"")},makeArray:function(e,b6){var i=b6||[];if(e!=null){if(ad(Object(e))){bI.merge(i,typeof e==="string"?[e]:e)}else{x.call(i,e)}}return i},inArray:function(b8,b6,b7){var e;if(b6){if(bU){return bU.call(b6,b8,b7)}e=b6.length;b7=b7?b7<0?Math.max(0,e+b7):b7:0;for(;b7<e;b7++){if(b7 in b6&&b6[b7]===b8){return b7}}}return -1},merge:function(b9,b7){var e=+b7.length,b6=0,b8=b9.length;while(b6<e){b9[b8++]=b7[b6++]}if(e!==e){while(b7[b6]!==undefined){b9[b8++]=b7[b6++]}}b9.length=b8;return b9},grep:function(
 e,cc,b9){var cb,b8=[],b6=0,b7=e.length,ca=!b9;for(;b6<b7;b6++){cb=!cc(e[b6],b6);if(cb!==ca){b8.push(e[b6])}}return b8},map:function(b7,cc,e){var cb,b9=0,ca=b7.length,b6=ad(b7),b8=[];if(b6){for(;b9<ca;b9++){cb=cc(b7[b9],b9,e);if(cb!=null){b8.push(cb)}}}else{for(b9 in b7){cb=cc(b7[b9],b9,e);if(cb!=null){b8.push(cb)}}}return az.apply([],b8)},guid:1,proxy:function(b8,b7){var e,b6,i;if(typeof b7==="string"){i=b8[b7];b7=b8;b8=i}if(!bI.isFunction(b8)){return undefined}e=P.call(arguments,2);b6=function(){return b8.apply(b7||this,e.concat(P.call(arguments)))};b6.guid=b8.guid=b8.guid||bI.guid++;return b6},now:function(){return +(new Date())},support:D});bI.each("Boolean Number String Function Array Date RegExp Object Error".split(" "),function(b6,e){ac["[object "+e+"]"]=e.toLowerCase()});function ad(b6){var i=b6.length,e=bI.type(b6);if(e==="function"||bI.isWindow(b6)){return false}if(b6.nodeType===1&&i){return true}return e==="array"||i===0||typeof i==="number"&&i>0&&(i-1) in b6}var m=
+/*!
+ * Sizzle CSS Selector Engine v2.2.0-pre
+ * http://sizzlejs.com/
+ *
+ * Copyright 2008, 2014 jQuery Foundation, Inc. and other contributors
+ * Released under the MIT license
+ * http://jquery.org/license
+ *
+ * Date: 2014-12-16
+ */
+(function(de){var cy,dh,cn,cH,cK,ci,cW,dg,dm,cI,cX,cZ,cC,co,c8,c3,df,ce,cF,da="sizzle"+1*new Date(),cJ=de.document,di=0,c4=0,b9=cA(),c9=cA(),cG=cA(),cE=function(i,e){if(i===e){cX=true}return 0},cQ=1<<31,cO=({}).hasOwnProperty,dc=[],dd=dc.pop,cM=dc.push,b7=dc.push,cm=dc.slice,cd=function(dq,dp){var dn=0,e=dq.length;for(;dn<e;dn++){if(dq[dn]===dp){return dn}}return -1},b8="checked|selected|async|autofocus|autoplay|controls|defer|disabled|hidden|ismap|loop|multiple|open|readonly|required|scoped",cp="[\\x20\\t\\r\\n\\f]",b6="(?:\\\\.|[\\w-]|[^\\x00-\\xa0])+",cL=b6.replace("w","w#"),c6="\\["+cp+"*("+b6+")(?:"+cp+"*([*^$|!~]?=)"+cp+"*(?:'((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\"|("+cL+"))|)"+cp+"*\\]",ck=":("+b6+")(?:\\((('((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\")|((?:\\\\.|[^\\\\()[\\]]|"+c6+")*)|.*)\\)|)",cu=new RegExp(cp+"+","g"),cr=new RegExp("^"+cp+"+|((?:^|[^\\\\])(?:\\\\.)*)"+cp+"+$","g"),cv=new RegExp("^"+cp+"*,"+cp+"*"),cB=new RegExp("^"+cp+"*([>+~]|"+cp+")"+cp+"
 *"),ct=new RegExp("="+cp+"*([^\\]'\"]*?)"+cp+"*\\]","g"),cS=new RegExp(ck),cU=new RegExp("^"+cL+"$"),c2={ID:new RegExp("^#("+b6+")"),CLASS:new RegExp("^\\.("+b6+")"),TAG:new RegExp("^("+b6.replace("w","w*")+")"),ATTR:new RegExp("^"+c6),PSEUDO:new RegExp("^"+ck),CHILD:new RegExp("^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\("+cp+"*(even|odd|(([+-]|)(\\d*)n|)"+cp+"*(?:([+-]|)"+cp+"*(\\d+)|))"+cp+"*\\)|)","i"),bool:new RegExp("^(?:"+b8+")$","i"),needsContext:new RegExp("^"+cp+"*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\("+cp+"*((?:-\\d)?\\d*)"+cp+"*\\)|)(?=[^-]|$)","i")},cc=/^(?:input|select|textarea|button)$/i,cl=/^h\d$/i,cP=/^[^{]+\{\s*\[native \w/,cR=/^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/,c1=/[+~]/,cN=/'|\\/g,cs=new RegExp("\\\\([\\da-f]{1,6}"+cp+"?|("+cp+")|.)","ig"),c5=function(e,dp,i){var dn="0x"+dp-65536;return dn!==dn||i?dp:dn<0?String.fromCharCode(dn+65536):String.fromCharCode(dn>>10|55296,dn&1023|56320)},dl=function(){cZ()};try{b7.apply((dc=cm.call(cJ.childNodes)),cJ
 .childNodes);dc[cJ.childNodes.length].nodeType}catch(cD){b7={apply:dc.length?function(i,e){cM.apply(i,cm.call(e))}:function(dq,dp){var e=dq.length,dn=0;while((dq[e++]=dp[dn++])){}dq.length=e-1}}}function cw(dv,dn,dz,dB){var dA,ds,dt,dx,dy,dr,dq,e,dp,dw;if((dn?dn.ownerDocument||dn:cJ)!==cC){cZ(dn)}dn=dn||cC;dz=dz||[];dx=dn.nodeType;if(typeof dv!=="string"||!dv||dx!==1&&dx!==9&&dx!==11){return dz}if(!dB&&c8){if(dx!==11&&(dA=cR.exec(dv))){if((dt=dA[1])){if(dx===9){ds=dn.getElementById(dt);if(ds&&ds.parentNode){if(ds.id===dt){dz.push(ds);return dz}}else{return dz}}else{if(dn.ownerDocument&&(ds=dn.ownerDocument.getElementById(dt))&&cF(dn,ds)&&ds.id===dt){dz.push(ds);return dz}}}else{if(dA[2]){b7.apply(dz,dn.getElementsByTagName(dv));return dz}else{if((dt=dA[3])&&dh.getElementsByClassName){b7.apply(dz,dn.getElementsByClassName(dt));return dz}}}}if(dh.qsa&&(!c3||!c3.test(dv))){e=dq=da;dp=dn;dw=dx!==1&&dv;if(dx===1&&dn.nodeName.toLowerCase()!=="object"){dr=ci(dv);if((dq=dn.getAttribute("id"
 ))){e=dq.replace(cN,"\\$&")}else{dn.setAttribute("id",e)}e="[id='"+e+"'] ";dy=dr.length;while(dy--){dr[dy]=e+ch(dr[dy])}dp=c1.test(dv)&&cT(dn.parentNode)||dn;dw=dr.join(",")}if(dw){try{b7.apply(dz,dp.querySelectorAll(dw));return dz}catch(du){}finally{if(!dq){dn.removeAttribute("id")}}}}}return dg(dv.replace(cr,"$1"),dn,dz,dB)}function cA(){var i=[];function e(dn,dp){if(i.push(dn+" ")>cn.cacheLength){delete e[i.shift()]}return(e[dn+" "]=dp)}return e}function cj(e){e[da]=true;return e}function cf(i){var dp=cC.createElement("div");try{return !!i(dp)}catch(dn){return false}finally{if(dp.parentNode){dp.parentNode.removeChild(dp)}dp=null}}function dj(dn,dq){var e=dn.split("|"),dp=dn.length;while(dp--){cn.attrHandle[e[dp]]=dq}}function ca(i,e){var dp=e&&i,dn=dp&&i.nodeType===1&&e.nodeType===1&&(~e.sourceIndex||cQ)-(~i.sourceIndex||cQ);if(dn){return dn}if(dp){while((dp=dp.nextSibling)){if(dp===e){return -1}}}return i?1:-1}function cx(e){return function(dn){var i=dn.nodeName.toLowerCase();re
 turn i==="input"&&dn.type===e}}function cb(e){return function(dn){var i=dn.nodeName.toLowerCase();return(i==="input"||i==="button")&&dn.type===e}}function c7(e){return cj(function(i){i=+i;return cj(function(dn,ds){var dq,dp=e([],dn.length,i),dr=dp.length;while(dr--){if(dn[(dq=dp[dr])]){dn[dq]=!(ds[dq]=dn[dq])}}})})}function cT(e){return e&&typeof e.getElementsByTagName!=="undefined"&&e}dh=cw.support={};cK=cw.isXML=function(e){var i=e&&(e.ownerDocument||e).documentElement;return i?i.nodeName!=="HTML":false};cZ=cw.setDocument=function(dn){var e,i,dp=dn?dn.ownerDocument||dn:cJ;if(dp===cC||dp.nodeType!==9||!dp.documentElement){return cC}cC=dp;co=dp.documentElement;i=dp.defaultView;if(i&&i!==i.top){if(i.addEventListener){i.addEventListener("unload",dl,false)}else{if(i.attachEvent){i.attachEvent("onunload",dl)}}}c8=!cK(dp);dh.attributes=cf(function(dq){dq.className="i";return !dq.getAttribute("className")});dh.getElementsByTagName=cf(function(dq){dq.appendChild(dp.createComment(""));retur
 n !dq.getElementsByTagName("*").length});dh.getElementsByClassName=cP.test(dp.getElementsByClassName);dh.getById=cf(function(dq){co.appendChild(dq).id=da;return !dp.getElementsByName||!dp.getElementsByName(da).length});if(dh.getById){cn.find.ID=function(ds,dr){if(typeof dr.getElementById!=="undefined"&&c8){var dq=dr.getElementById(ds);return dq&&dq.parentNode?[dq]:[]}};cn.filter.ID=function(dr){var dq=dr.replace(cs,c5);return function(ds){return ds.getAttribute("id")===dq}}}else{delete cn.find.ID;cn.filter.ID=function(dr){var dq=dr.replace(cs,c5);return function(dt){var ds=typeof dt.getAttributeNode!=="undefined"&&dt.getAttributeNode("id");return ds&&ds.value===dq}}}cn.find.TAG=dh.getElementsByTagName?function(dq,dr){if(typeof dr.getElementsByTagName!=="undefined"){return dr.getElementsByTagName(dq)}else{if(dh.qsa){return dr.querySelectorAll(dq)}}}:function(dq,du){var dv,dt=[],ds=0,dr=du.getElementsByTagName(dq);if(dq==="*"){while((dv=dr[ds++])){if(dv.nodeType===1){dt.push(dv)}}retu
 rn dt}return dr};cn.find.CLASS=dh.getElementsByClassName&&function(dr,dq){if(c8){return dq.getElementsByClassName(dr)}};df=[];c3=[];if((dh.qsa=cP.test(dp.querySelectorAll))){cf(function(dq){co.appendChild(dq).innerHTML="<a id='"+da+"'></a><select id='"+da+"-\f]' msallowcapture=''><option selected=''></option></select>";if(dq.querySelectorAll("[msallowcapture^='']").length){c3.push("[*^$]="+cp+"*(?:''|\"\")")}if(!dq.querySelectorAll("[selected]").length){c3.push("\\["+cp+"*(?:value|"+b8+")")}if(!dq.querySelectorAll("[id~="+da+"-]").length){c3.push("~=")}if(!dq.querySelectorAll(":checked").length){c3.push(":checked")}if(!dq.querySelectorAll("a#"+da+"+*").length){c3.push(".#.+[+~]")}});cf(function(dr){var dq=dp.createElement("input");dq.setAttribute("type","hidden");dr.appendChild(dq).setAttribute("name","D");if(dr.querySelectorAll("[name=d]").length){c3.push("name"+cp+"*[*^$|!~]?=")}if(!dr.querySelectorAll(":enabled").length){c3.push(":enabled",":disabled")}dr.querySelectorAll("*,:x")
 ;c3.push(",.*:")})}if((dh.matchesSelector=cP.test((ce=co.matches||co.webkitMatchesSelector||co.mozMatchesSelector||co.oMatchesSelector||co.msMatchesSelector)))){cf(function(dq){dh.disconnectedMatch=ce.call(dq,"div");ce.call(dq,"[s!='']:x");df.push("!=",ck)})}c3=c3.length&&new RegExp(c3.join("|"));df=df.length&&new RegExp(df.join("|"));e=cP.test(co.compareDocumentPosition);cF=e||cP.test(co.contains)?function(dr,dq){var dt=dr.nodeType===9?dr.documentElement:dr,ds=dq&&dq.parentNode;return dr===ds||!!(ds&&ds.nodeType===1&&(dt.contains?dt.contains(ds):dr.compareDocumentPosition&&dr.compareDocumentPosition(ds)&16))}:function(dr,dq){if(dq){while((dq=dq.parentNode)){if(dq===dr){return true}}}return false};cE=e?function(dr,dq){if(dr===dq){cX=true;return 0}var ds=!dr.compareDocumentPosition-!dq.compareDocumentPosition;if(ds){return ds}ds=(dr.ownerDocument||dr)===(dq.ownerDocument||dq)?dr.compareDocumentPosition(dq):1;if(ds&1||(!dh.sortDetached&&dq.compareDocumentPosition(dr)===ds)){if(dr===dp
 ||dr.ownerDocument===cJ&&cF(cJ,dr)){return -1}if(dq===dp||dq.ownerDocument===cJ&&cF(cJ,dq)){return 1}return cI?(cd(cI,dr)-cd(cI,dq)):0}return ds&4?-1:1}:function(dr,dq){if(dr===dq){cX=true;return 0}var dx,du=0,dw=dr.parentNode,dt=dq.parentNode,ds=[dr],dv=[dq];if(!dw||!dt){return dr===dp?-1:dq===dp?1:dw?-1:dt?1:cI?(cd(cI,dr)-cd(cI,dq)):0}else{if(dw===dt){return ca(dr,dq)}}dx=dr;while((dx=dx.parentNode)){ds.unshift(dx)}dx=dq;while((dx=dx.parentNode)){dv.unshift(dx)}while(ds[du]===dv[du]){du++}return du?ca(ds[du],dv[du]):ds[du]===cJ?-1:dv[du]===cJ?1:0};return dp};cw.matches=function(i,e){return cw(i,null,null,e)};cw.matchesSelector=function(dn,dq){if((dn.ownerDocument||dn)!==cC){cZ(dn)}dq=dq.replace(ct,"='$1']");if(dh.matchesSelector&&c8&&(!df||!df.test(dq))&&(!c3||!c3.test(dq))){try{var i=ce.call(dn,dq);if(i||dh.disconnectedMatch||dn.document&&dn.document.nodeType!==11){return i}}catch(dp){}}return cw(dq,cC,null,[dn]).length>0};cw.contains=function(e,i){if((e.ownerDocument||e)!==cC){c
 Z(e)}return cF(e,i)};cw.attr=function(dn,e){if((dn.ownerDocument||dn)!==cC){cZ(dn)}var i=cn.attrHandle[e.toLowerCase()],dp=i&&cO.call(cn.attrHandle,e.toLowerCase())?i(dn,e,!c8):undefined;return dp!==undefined?dp:dh.attributes||!c8?dn.getAttribute(e):(dp=dn.getAttributeNode(e))&&dp.specified?dp.value:null};cw.error=function(e){throw new Error("Syntax error, unrecognized expression: "+e)};cw.uniqueSort=function(dp){var dq,dr=[],e=0,dn=0;cX=!dh.detectDuplicates;cI=!dh.sortStable&&dp.slice(0);dp.sort(cE);if(cX){while((dq=dp[dn++])){if(dq===dp[dn]){e=dr.push(dn)}}while(e--){dp.splice(dr[e],1)}}cI=null;return dp};cH=cw.getText=function(dr){var dq,dn="",dp=0,e=dr.nodeType;if(!e){while((dq=dr[dp++])){dn+=cH(dq)}}else{if(e===1||e===9||e===11){if(typeof dr.textContent==="string"){return dr.textContent}else{for(dr=dr.firstChild;dr;dr=dr.nextSibling){dn+=cH(dr)}}}else{if(e===3||e===4){return dr.nodeValue}}}return dn};cn=cw.selectors={cacheLength:50,createPseudo:cj,match:c2,attrHandle:{},find:{}
 ,relative:{">":{dir:"parentNode",first:true}," ":{dir:"parentNode"},"+":{dir:"previousSibling",first:true},"~":{dir:"previousSibling"}},preFilter:{ATTR:function(e){e[1]=e[1].replace(cs,c5);e[3]=(e[3]||e[4]||e[5]||"").replace(cs,c5);if(e[2]==="~="){e[3]=" "+e[3]+" "}return e.slice(0,4)},CHILD:function(e){e[1]=e[1].toLowerCase();if(e[1].slice(0,3)==="nth"){if(!e[3]){cw.error(e[0])}e[4]=+(e[4]?e[5]+(e[6]||1):2*(e[3]==="even"||e[3]==="odd"));e[5]=+((e[7]+e[8])||e[3]==="odd")}else{if(e[3]){cw.error(e[0])}}return e},PSEUDO:function(i){var e,dn=!i[6]&&i[2];if(c2.CHILD.test(i[0])){return null}if(i[3]){i[2]=i[4]||i[5]||""}else{if(dn&&cS.test(dn)&&(e=ci(dn,true))&&(e=dn.indexOf(")",dn.length-e)-dn.length)){i[0]=i[0].slice(0,e);i[2]=dn.slice(0,e)}}return i.slice(0,3)}},filter:{TAG:function(i){var e=i.replace(cs,c5).toLowerCase();return i==="*"?function(){return true}:function(dn){return dn.nodeName&&dn.nodeName.toLowerCase()===e}},CLASS:function(e){var i=b9[e+" "];return i||(i=new RegExp("(^|"
 +cp+")"+e+"("+cp+"|$)"))&&b9(e,function(dn){return i.test(typeof dn.className==="string"&&dn.className||typeof dn.getAttribute!=="undefined"&&dn.getAttribute("class")||"")})},ATTR:function(dn,i,e){return function(dq){var dp=cw.attr(dq,dn);if(dp==null){return i==="!="}if(!i){return true}dp+="";return i==="="?dp===e:i==="!="?dp!==e:i==="^="?e&&dp.indexOf(e)===0:i==="*="?e&&dp.indexOf(e)>-1:i==="$="?e&&dp.slice(-e.length)===e:i==="~="?(" "+dp.replace(cu," ")+" ").indexOf(e)>-1:i==="|="?dp===e||dp.slice(0,e.length+1)===e+"-":false}},CHILD:function(i,dq,dp,dr,dn){var dt=i.slice(0,3)!=="nth",e=i.slice(-4)!=="last",ds=dq==="of-type";return dr===1&&dn===0?function(du){return !!du.parentNode}:function(dA,dy,dD){var du,dG,dB,dF,dC,dx,dz=dt!==e?"nextSibling":"previousSibling",dE=dA.parentNode,dw=ds&&dA.nodeName.toLowerCase(),dv=!dD&&!ds;if(dE){if(dt){while(dz){dB=dA;while((dB=dB[dz])){if(ds?dB.nodeName.toLowerCase()===dw:dB.nodeType===1){return false}}dx=dz=i==="only"&&!dx&&"nextSibling"}retur
 n true}dx=[e?dE.firstChild:dE.lastChild];if(e&&dv){dG=dE[da]||(dE[da]={});du=dG[i]||[];dC=du[0]===di&&du[1];dF=du[0]===di&&du[2];dB=dC&&dE.childNodes[dC];while((dB=++dC&&dB&&dB[dz]||(dF=dC=0)||dx.pop())){if(dB.nodeType===1&&++dF&&dB===dA){dG[i]=[di,dC,dF];break}}}else{if(dv&&(du=(dA[da]||(dA[da]={}))[i])&&du[0]===di){dF=du[1]}else{while((dB=++dC&&dB&&dB[dz]||(dF=dC=0)||dx.pop())){if((ds?dB.nodeName.toLowerCase()===dw:dB.nodeType===1)&&++dF){if(dv){(dB[da]||(dB[da]={}))[i]=[di,dF]}if(dB===dA){break}}}}}dF-=dn;return dF===dr||(dF%dr===0&&dF/dr>=0)}}},PSEUDO:function(dp,dn){var e,i=cn.pseudos[dp]||cn.setFilters[dp.toLowerCase()]||cw.error("unsupported pseudo: "+dp);if(i[da]){return i(dn)}if(i.length>1){e=[dp,dp,"",dn];return cn.setFilters.hasOwnProperty(dp.toLowerCase())?cj(function(ds,du){var dr,dq=i(ds,dn),dt=dq.length;while(dt--){dr=cd(ds,dq[dt]);ds[dr]=!(du[dr]=dq[dt])}}):function(dq){return i(dq,0,e)}}return i}},pseudos:{not:cj(function(e){var i=[],dn=[],dp=cW(e.replace(cr,"$1"));
 return dp[da]?cj(function(dr,dw,du,ds){var dv,dq=dp(dr,null,ds,[]),dt=dr.length;while(dt--){if((dv=dq[dt])){dr[dt]=!(dw[dt]=dv)}}}):function(ds,dr,dq){i[0]=ds;dp(i,null,dq,dn);i[0]=null;return !dn.pop()}}),has:cj(function(e){return function(i){return cw(e,i).length>0}}),contains:cj(function(e){e=e.replace(cs,c5);return function(i){return(i.textContent||i.innerText||cH(i)).indexOf(e)>-1}}),lang:cj(function(e){if(!cU.test(e||"")){cw.error("unsupported lang: "+e)}e=e.replace(cs,c5).toLowerCase();return function(dn){var i;do{if((i=c8?dn.lang:dn.getAttribute("xml:lang")||dn.getAttribute("lang"))){i=i.toLowerCase();return i===e||i.indexOf(e+"-")===0}}while((dn=dn.parentNode)&&dn.nodeType===1);return false}}),target:function(e){var i=de.location&&de.location.hash;return i&&i.slice(1)===e.id},root:function(e){return e===co},focus:function(e){return e===cC.activeElement&&(!cC.hasFocus||cC.hasFocus())&&!!(e.type||e.href||~e.tabIndex)},enabled:function(e){return e.disabled===false},disabled:fu
 nction(e){return e.disabled===true},checked:function(e){var i=e.nodeName.toLowerCase();return(i==="input"&&!!e.checked)||(i==="option"&&!!e.selected)},selected:function(e){if(e.parentNode){e.parentNode.selectedIndex}return e.selected===true},empty:function(e){for(e=e.firstChild;e;e=e.nextSibling){if(e.nodeType<6){return false}}return true},parent:function(e){return !cn.pseudos.empty(e)},header:function(e){return cl.test(e.nodeName)},input:function(e){return cc.test(e.nodeName)},button:function(i){var e=i.nodeName.toLowerCase();return e==="input"&&i.type==="button"||e==="button"},text:function(i){var e;return i.nodeName.toLowerCase()==="input"&&i.type==="text"&&((e=i.getAttribute("type"))==null||e.toLowerCase()==="text")},first:c7(function(){return[0]}),last:c7(function(e,i){return[i-1]}),eq:c7(function(e,dn,i){return[i<0?i+dn:i]}),even:c7(function(e,dp){var dn=0;for(;dn<dp;dn+=2){e.push(dn)}return e}),odd:c7(function(e,dp){var dn=1;for(;dn<dp;dn+=2){e.push(dn)}return e}),lt:c7(funct
 ion(e,dq,dp){var dn=dp<0?dp+dq:dp;for(;--dn>=0;){e.push(dn)}return e}),gt:c7(function(e,dq,dp){var dn=dp<0?dp+dq:dp;for(;++dn<dq;){e.push(dn)}return e})}};cn.pseudos.nth=cn.pseudos.eq;for(cy in {radio:true,checkbox:true,file:true,password:true,image:true}){cn.pseudos[cy]=cx(cy)}for(cy in {submit:true,reset:true}){cn.pseudos[cy]=cb(cy)}function cV(){}cV.prototype=cn.filters=cn.pseudos;cn.setFilters=new cV();ci=cw.tokenize=function(dq,dv){var i,dr,dt,du,ds,dn,e,dp=c9[dq+" "];if(dp){return dv?0:dp.slice(0)}ds=dq;dn=[];e=cn.preFilter;while(ds){if(!i||(dr=cv.exec(ds))){if(dr){ds=ds.slice(dr[0].length)||ds}dn.push((dt=[]))}i=false;if((dr=cB.exec(ds))){i=dr.shift();dt.push({value:i,type:dr[0].replace(cr," ")});ds=ds.slice(i.length)}for(du in cn.filter){if((dr=c2[du].exec(ds))&&(!e[du]||(dr=e[du](dr)))){i=dr.shift();dt.push({value:i,type:du,matches:dr});ds=ds.slice(i.length)}}if(!i){break}}return dv?ds.length:ds?cw.error(dq):c9(dq,dn).slice(0)};function ch(dq){var dp=0,dn=dq.length,e="";for
 (;dp<dn;dp++){e+=dq[dp].value}return e}function cq(dq,dn,dp){var e=dn.dir,dr=dp&&e==="parentNode",i=c4++;return dn.first?function(du,dt,ds){while((du=du[e])){if(du.nodeType===1||dr){return dq(du,dt,ds)}}}:function(dw,du,dt){var dx,dv,ds=[di,i];if(dt){while((dw=dw[e])){if(dw.nodeType===1||dr){if(dq(dw,du,dt)){return true}}}}else{while((dw=dw[e])){if(dw.nodeType===1||dr){dv=dw[da]||(dw[da]={});if((dx=dv[e])&&dx[0]===di&&dx[1]===i){return(ds[2]=dx[2])}else{dv[e]=ds;if((ds[2]=dq(dw,du,dt))){return true}}}}}}}function dk(e){return e.length>1?function(dr,dq,dn){var dp=e.length;while(dp--){if(!e[dp](dr,dq,dn)){return false}}return true}:e[0]}function cz(dn,dr,dq){var dp=0,e=dr.length;for(;dp<e;dp++){cw(dn,dr[dp],dq)}return dq}function c0(e,dn,dp,dq,dt){var dr,dw=[],ds=0,du=e.length,dv=dn!=null;for(;ds<du;ds++){if((dr=e[ds])){if(!dp||dp(dr,dq,dt)){dw.push(dr);if(dv){dn.push(ds)}}}}return dw}function cg(dn,i,dq,dp,dr,e){if(dp&&!dp[da]){dp=cg(dp)}if(dr&&!dr[da]){dr=cg(dr,e)}return cj(function
 (dC,dz,du,dB){var dE,dA,dw,dv=[],dD=[],dt=dz.length,ds=dC||cz(i||"*",du.nodeType?[du]:du,[]),dx=dn&&(dC||!i)?c0(ds,dv,dn,du,dB):ds,dy=dq?dr||(dC?dn:dt||dp)?[]:dz:dx;if(dq){dq(dx,dy,du,dB)}if(dp){dE=c0(dy,dD);dp(dE,[],du,dB);dA=dE.length;while(dA--){if((dw=dE[dA])){dy[dD[dA]]=!(dx[dD[dA]]=dw)}}}if(dC){if(dr||dn){if(dr){dE=[];dA=dy.length;while(dA--){if((dw=dy[dA])){dE.push((dx[dA]=dw))}}dr(null,(dy=[]),dE,dB)}dA=dy.length;while(dA--){if((dw=dy[dA])&&(dE=dr?cd(dC,dw):dv[dA])>-1){dC[dE]=!(dz[dE]=dw)}}}}else{dy=c0(dy===dz?dy.splice(dt,dy.length):dy);if(dr){dr(null,dz,dy,dB)}else{b7.apply(dz,dy)}}})}function db(dt){var dn,dr,dp,ds=dt.length,dw=cn.relative[dt[0].type],dx=dw||cn.relative[" "],dq=dw?1:0,du=cq(function(i){return i===dn},dx,true),dv=cq(function(i){return cd(dn,i)>-1},dx,true),e=[function(dA,dz,dy){var i=(!dw&&(dy||dz!==dm))||((dn=dz).nodeType?du(dA,dz,dy):dv(dA,dz,dy));dn=null;return i}];for(;dq<ds;dq++){if((dr=cn.relative[dt[dq].type])){e=[cq(dk(e),dr)]}else{dr=cn.filter[dt[
 dq].type].apply(null,dt[dq].matches);if(dr[da]){dp=++dq;for(;dp<ds;dp++){if(cn.relative[dt[dp].type]){break}}return cg(dq>1&&dk(e),dq>1&&ch(dt.slice(0,dq-1).concat({value:dt[dq-2].type===" "?"*":""})).replace(cr,"$1"),dr,dq<dp&&db(dt.slice(dq,dp)),dp<ds&&db((dt=dt.slice(dp))),dp<ds&&ch(dt))}e.push(dr)}}return dk(e)}function cY(dp,dn){var e=dn.length>0,dq=dp.length>0,i=function(dA,du,dz,dy,dD){var dv,dw,dB,dF=0,dx="0",dr=dA&&[],dG=[],dE=dm,dt=dA||dq&&cn.find.TAG("*",dD),ds=(di+=dE==null?1:Math.random()||0.1),dC=dt.length;if(dD){dm=du!==cC&&du}for(;dx!==dC&&(dv=dt[dx])!=null;dx++){if(dq&&dv){dw=0;while((dB=dp[dw++])){if(dB(dv,du,dz)){dy.push(dv);break}}if(dD){di=ds}}if(e){if((dv=!dB&&dv)){dF--}if(dA){dr.push(dv)}}}dF+=dx;if(e&&dx!==dF){dw=0;while((dB=dn[dw++])){dB(dr,dG,du,dz)}if(dA){if(dF>0){while(dx--){if(!(dr[dx]||dG[dx])){dG[dx]=dd.call(dy)}}}dG=c0(dG)}b7.apply(dy,dG);if(dD&&!dA&&dG.length>0&&(dF+dn.length)>1){cw.uniqueSort(dy)}}if(dD){di=ds;dm=dE}return dr};return e?cj(i):i}cW=cw
 .compile=function(e,dp){var dq,dn=[],ds=[],dr=cG[e+" "];if(!dr){if(!dp){dp=ci(e)}dq=dp.length;while(dq--){dr=db(dp[dq]);if(dr[da]){dn.push(dr)}else{ds.push(dr)}}dr=cG(e,cY(ds,dn));dr.selector=e}return dr};dg=cw.select=function(dp,e,dq,dt){var dr,dw,dn,dx,du,dv=typeof dp==="function"&&dp,ds=!dt&&ci((dp=dv.selector||dp));dq=dq||[];if(ds.length===1){dw=ds[0]=ds[0].slice(0);if(dw.length>2&&(dn=dw[0]).type==="ID"&&dh.getById&&e.nodeType===9&&c8&&cn.relative[dw[1].type]){e=(cn.find.ID(dn.matches[0].replace(cs,c5),e)||[])[0];if(!e){return dq}else{if(dv){e=e.parentNode}}dp=dp.slice(dw.shift().value.length)}dr=c2.needsContext.test(dp)?0:dw.length;while(dr--){dn=dw[dr];if(cn.relative[(dx=dn.type)]){break}if((du=cn.find[dx])){if((dt=du(dn.matches[0].replace(cs,c5),c1.test(dw[0].type)&&cT(e.parentNode)||e))){dw.splice(dr,1);dp=dt.length&&ch(dw);if(!dp){b7.apply(dq,dt);return dq}break}}}}(dv||cW(dp,ds))(dt,e,!c8,dq,c1.test(dp)&&cT(e.parentNode)||e);return dq};dh.sortStable=da.split("").sort(cE).
 join("")===da;dh.detectDuplicates=!!cX;cZ();dh.sortDetached=cf(function(e){return e.compareDocumentPosition(cC.createElement("div"))&1});if(!cf(function(e){e.innerHTML="<a href='#'></a>";return e.firstChild.getAttribute("href")==="#"})){dj("type|href|height|width",function(i,e,dn){if(!dn){return i.getAttribute(e,e.toLowerCase()==="type"?1:2)}})}if(!dh.attributes||!cf(function(e){e.innerHTML="<input/>";e.firstChild.setAttribute("value","");return e.firstChild.getAttribute("value")===""})){dj("value",function(i,e,dn){if(!dn&&i.nodeName.toLowerCase()==="input"){return i.defaultValue}})}if(!cf(function(e){return e.getAttribute("disabled")==null})){dj(b8,function(i,e,dp){var dn;if(!dp){return i[e]===true?e.toLowerCase():(dn=i.getAttributeNode(e))&&dn.specified?dn.value:null}})}return cw})(a5);bI.find=m;bI.expr=m.selectors;bI.expr[":"]=bI.expr.pseudos;bI.unique=m.uniqueSort;bI.text=m.getText;bI.isXMLDoc=m.isXML;bI.contains=m.contains;var A=bI.expr.match.needsContext;var a=(/^<(\w+)\s*\/?>
 (?:<\/\1>|)$/);var aL=/^.[^:#\[\.,]*$/;function aR(b6,e,i){if(bI.isFunction(e)){return bI.grep(b6,function(b8,b7){return !!e.call(b8,b7,b8)!==i})}if(e.nodeType){return bI.grep(b6,function(b7){return(b7===e)!==i})}if(typeof e==="string"){if(aL.test(e)){return bI.filter(e,b6,i)}e=bI.filter(e,b6)}return bI.grep(b6,function(b7){return(bI.inArray(b7,e)>=0)!==i})}bI.filter=function(b7,e,b6){var i=e[0];if(b6){b7=":not("+b7+")"}return e.length===1&&i.nodeType===1?bI.find.matchesSelector(i,b7)?[i]:[]:bI.find.matches(b7,bI.grep(e,function(b8){return b8.nodeType===1}))};bI.fn.extend({find:function(b6){var b9,b8=[],b7=this,e=b7.length;if(typeof b6!=="string"){return this.pushStack(bI(b6).filter(function(){for(b9=0;b9<e;b9++){if(bI.contains(b7[b9],this)){return true}}}))}for(b9=0;b9<e;b9++){bI.find(b6,b7[b9],b8)}b8=this.pushStack(e>1?bI.unique(b8):b8);b8.selector=this.selector?this.selector+" "+b6:b6;return b8},filter:function(e){return this.pushStack(aR(this,e||[],false))},not:function(e){retur
 n this.pushStack(aR(this,e||[],true))},is:function(e){return !!aR(this,typeof e==="string"&&A.test(e)?bI(e):e||[],false).length}});var z,n=a5.document,bt=/^(?:\s*(<[\w\W]+>)[^>]*|#([\w-]*))$/,bV=bI.fn.init=function(e,b6){var i,b7;if(!e){return this}if(typeof e==="string"){if(e.charAt(0)==="<"&&e.charAt(e.length-1)===">"&&e.length>=3){i=[null,e,null]}else{i=bt.exec(e)}if(i&&(i[1]||!b6)){if(i[1]){b6=b6 instanceof bI?b6[0]:b6;bI.merge(this,bI.parseHTML(i[1],b6&&b6.nodeType?b6.ownerDocument||b6:n,true));if(a.test(i[1])&&bI.isPlainObject(b6)){for(i in b6){if(bI.isFunction(this[i])){this[i](b6[i])}else{this.attr(i,b6[i])}}}return this}else{b7=n.getElementById(i[2]);if(b7&&b7.parentNode){if(b7.id!==i[2]){return z.find(e)}this.length=1;this[0]=b7}this.context=n;this.selector=e;return this}}else{if(!b6||b6.jquery){return(b6||z).find(e)}else{return this.constructor(b6).find(e)}}}else{if(e.nodeType){this.context=this[0]=e;this.length=1;return this}else{if(bI.isFunction(e)){return typeof z.read
 y!=="undefined"?z.ready(e):e(bI)}}}if(e.selector!==undefined){this.selector=e.selector;this.context=e.context}return bI.makeArray(e,this)};bV.prototype=bI.fn;z=bI(n);var bv=/^(?:parents|prev(?:Until|All))/,bz={children:true,contents:true,next:true,prev:true};bI.extend({dir:function(b6,i,b8){var e=[],b7=b6[i];while(b7&&b7.nodeType!==9&&(b8===undefined||b7.nodeType!==1||!bI(b7).is(b8))){if(b7.nodeType===1){e.push(b7)}b7=b7[i]}return e},sibling:function(b6,i){var e=[];for(;b6;b6=b6.nextSibling){if(b6.nodeType===1&&b6!==i){e.push(b6)}}return e}});bI.fn.extend({has:function(b8){var b7,b6=bI(b8,this),e=b6.length;return this.filter(function(){for(b7=0;b7<e;b7++){if(bI.contains(this,b6[b7])){return true}}})},closest:function(b9,b8){var ca,b7=0,b6=this.length,e=[],cb=A.test(b9)||typeof b9!=="string"?bI(b9,b8||this.context):0;for(;b7<b6;b7++){for(ca=this[b7];ca&&ca!==b8;ca=ca.parentNode){if(ca.nodeType<11&&(cb?cb.index(ca)>-1:ca.nodeType===1&&bI.find.matchesSelector(ca,b9))){e.push(ca);break}
 }}return this.pushStack(e.length>1?bI.unique(e):e)},index:function(e){if(!e){return(this[0]&&this[0].parentNode)?this.first().prevAll().length:-1}if(typeof e==="string"){return bI.inArray(this[0],bI(e))}return bI.inArray(e.jquery?e[0]:e,this)},add:function(e,i){return this.pushStack(bI.unique(bI.merge(this.get(),bI(e,i))))},addBack:function(e){return this.add(e==null?this.prevObject:this.prevObject.filter(e))}});function aY(i,e){do{i=i[e]}while(i&&i.nodeType!==1);return i}bI.each({parent:function(i){var e=i.parentNode;return e&&e.nodeType!==11?e:null},parents:function(e){return bI.dir(e,"parentNode")},parentsUntil:function(b6,e,b7){return bI.dir(b6,"parentNode",b7)},next:function(e){return aY(e,"nextSibling")},prev:function(e){return aY(e,"previousSibling")},nextAll:function(e){return bI.dir(e,"nextSibling")},prevAll:function(e){return bI.dir(e,"previousSibling")},nextUntil:function(b6,e,b7){return bI.dir(b6,"nextSibling",b7)},prevUntil:function(b6,e,b7){return bI.dir(b6,"previousSi
 bling",b7)},siblings:function(e){return bI.sibling((e.parentNode||{}).firstChild,e)},children:function(e){return bI.sibling(e.firstChild)},contents:function(e){return bI.nodeName(e,"iframe")?e.contentDocument||e.contentWindow.document:bI.merge([],e.childNodes)}},function(e,i){bI.fn[e]=function(b8,b6){var b7=bI.map(this,i,b8);if(e.slice(-5)!=="Until"){b6=b8}if(b6&&typeof b6==="string"){b7=bI.filter(b6,b7)}if(this.length>1){if(!bz[e]){b7=bI.unique(b7)}if(bv.test(e)){b7=b7.reverse()}}return this.pushStack(b7)}});var aF=(/\S+/g);var b2={};function af(i){var e=b2[i]={};bI.each(i.match(aF)||[],function(b7,b6){e[b6]=true});return e}bI.Callbacks=function(ce){ce=typeof ce==="string"?(b2[ce]||af(ce)):bI.extend({},ce);var b8,b7,e,b9,ca,b6,cb=[],cc=!ce.once&&[],i=function(cf){b7=ce.memory&&cf;e=true;ca=b6||0;b6=0;b9=cb.length;b8=true;for(;cb&&ca<b9;ca++){if(cb[ca].apply(cf[0],cf[1])===false&&ce.stopOnFalse){b7=false;break}}b8=false;if(cb){if(cc){if(cc.length){i(cc.shift())}}else{if(b7){cb=[]}el
 se{cd.disable()}}}},cd={add:function(){if(cb){var cg=cb.length;(function cf(ch){bI.each(ch,function(cj,ci){var ck=bI.type(ci);if(ck==="function"){if(!ce.unique||!cd.has(ci)){cb.push(ci)}}else{if(ci&&ci.length&&ck!=="string"){cf(ci)}}})})(arguments);if(b8){b9=cb.length}else{if(b7){b6=cg;i(b7)}}}return this},remove:function(){if(cb){bI.each(arguments,function(ch,cf){var cg;while((cg=bI.inArray(cf,cb,cg))>-1){cb.splice(cg,1);if(b8){if(cg<=b9){b9--}if(cg<=ca){ca--}}}})}return this},has:function(cf){return cf?bI.inArray(cf,cb)>-1:!!(cb&&cb.length)},empty:function(){cb=[];b9=0;return this},disable:function(){cb=cc=b7=undefined;return this},disabled:function(){return !cb},lock:function(){cc=undefined;if(!b7){cd.disable()}return this},locked:function(){return !cc},fireWith:function(cg,cf){if(cb&&(!e||cc)){cf=cf||[];cf=[cg,cf.slice?cf.slice():cf];if(b8){cc.push(cf)}else{i(cf)}}return this},fire:function(){cd.fireWith(this,arguments);return this},fired:function(){return !!e}};return cd};bI.ex
 tend({Deferred:function(b6){var i=[["resolve","done",bI.Callbacks("once memory"),"resolved"],["reject","fail",bI.Callbacks("once memory"),"rejected"],["notify","progress",bI.Callbacks("memory")]],b7="pending",b8={state:function(){return b7},always:function(){e.done(arguments).fail(arguments);return this},then:function(){var b9=arguments;return bI.Deferred(function(ca){bI.each(i,function(cc,cb){var cd=bI.isFunction(b9[cc])&&b9[cc];e[cb[1]](function(){var ce=cd&&cd.apply(this,arguments);if(ce&&bI.isFunction(ce.promise)){ce.promise().done(ca.resolve).fail(ca.reject).progress(ca.notify)}else{ca[cb[0]+"With"](this===b8?ca.promise():this,cd?[ce]:arguments)}})});b9=null}).promise()},promise:function(b9){return b9!=null?bI.extend(b9,b8):b8}},e={};b8.pipe=b8.then;bI.each(i,function(ca,b9){var cc=b9[2],cb=b9[3];b8[b9[1]]=cc.add;if(cb){cc.add(function(){b7=cb},i[ca^1][2].disable,i[2][2].lock)}e[b9[0]]=function(){e[b9[0]+"With"](this===e?b8:this,arguments);return this};e[b9[0]+"With"]=cc.fireWi
 th});b8.promise(e);if(b6){b6.call(e,e)}return e},when:function(b9){var b7=0,cb=P.call(arguments),e=cb.length,b6=e!==1||(b9&&bI.isFunction(b9.promise))?e:0,ce=b6===1?b9:bI.Deferred(),b8=function(cg,ch,cf){return function(i){ch[cg]=this;cf[cg]=arguments.length>1?P.call(arguments):i;if(cf===cd){ce.notifyWith(ch,cf)}else{if(!(--b6)){ce.resolveWith(ch,cf)}}}},cd,ca,cc;if(e>1){cd=new Array(e);ca=new Array(e);cc=new Array(e);for(;b7<e;b7++){if(cb[b7]&&bI.isFunction(cb[b7].promise)){cb[b7].promise().done(b8(b7,cc,cb)).fail(ce.reject).progress(b8(b7,ca,cd))}else{--b6}}}if(!b6){ce.resolveWith(cc,cb)}return ce.promise()}});var ak;bI.fn.ready=function(e){bI.ready.promise().done(e);return this};bI.extend({isReady:false,readyWait:1,holdReady:function(e){if(e){bI.readyWait++}else{bI.ready(true)}},ready:function(e){if(e===true?--bI.readyWait:bI.isReady){return}if(!n.body){return setTimeout(bI.ready)}bI.isReady=true;if(e!==true&&--bI.readyWait>0){return}ak.resolveWith(n,[bI]);if(bI.fn.triggerHandler
 ){bI(n).triggerHandler("ready");bI(n).off("ready")}}});function bm(){if(n.addEventListener){n.removeEventListener("DOMContentLoaded",bZ,false);a5.removeEventListener("load",bZ,false)}else{n.detachEvent("onreadystatechange",bZ);a5.detachEvent("onload",bZ)}}function bZ(){if(n.addEventListener||event.type==="load"||n.readyState==="complete"){bm();bI.ready()}}bI.ready.promise=function(b8){if(!ak){ak=bI.Deferred();if(n.readyState==="complete"){setTimeout(bI.ready)}else{if(n.addEventListener){n.addEventListener("DOMContentLoaded",bZ,false);a5.addEventListener("load",bZ,false)}else{n.attachEvent("onreadystatechange",bZ);a5.attachEvent("onload",bZ);var b7=false;try{b7=a5.frameElement==null&&n.documentElement}catch(b6){}if(b7&&b7.doScroll){(function i(){if(!bI.isReady){try{b7.doScroll("left")}catch(b9){return setTimeout(i,50)}bm();bI.ready()}})()}}}}return ak.promise(b8)};var aC=typeof undefined;var bh;for(bh in bI(D)){break}D.ownLast=bh!=="0";D.inlineBlockNeedsLayout=false;bI(function(){var
  b6,b7,e,i;e=n.getElementsByTagName("body")[0];if(!e||!e.style){return}b7=n.createElement("div");i=n.createElement("div");i.style.cssText="position:absolute;border:0;width:0;height:0;top:0;left:-9999px";e.appendChild(i).appendChild(b7);if(typeof b7.style.zoom!==aC){b7.style.cssText="display:inline;margin:0;border:0;padding:1px;width:1px;zoom:1";D.inlineBlockNeedsLayout=b6=b7.offsetWidth===3;if(b6){e.style.zoom=1}}e.removeChild(i)});(function(){var b6=n.createElement("div");if(D.deleteExpando==null){D.deleteExpando=true;try{delete b6.test}catch(i){D.deleteExpando=false}}b6=null})();bI.acceptData=function(b6){var i=bI.noData[(b6.nodeName+" ").toLowerCase()],e=+b6.nodeType||1;return e!==1&&e!==9?false:!i||i!==true&&b6.getAttribute("classid")===i};var by=/^(?:\{[\w\W]*\}|\[[\w\W]*\])$/,aQ=/([A-Z])/g;function bA(b7,b6,b8){if(b8===undefined&&b7.nodeType===1){var i="data-"+b6.replace(aQ,"-$1").toLowerCase();b8=b7.getAttribute(i);if(typeof b8==="string"){try{b8=b8==="true"?true:b8==="false"
 ?false:b8==="null"?null:+b8+""===b8?+b8:by.test(b8)?bI.parseJSON(b8):b8}catch(b9){}bI.data(b7,b6,b8)}else{b8=undefined}}return b8}function Q(i){var e;for(e in i){if(e==="data"&&bI.isEmptyObject(i[e])){continue}if(e!=="toJSON"){return false}}return true}function bc(b7,i,b9,b8){if(!bI.acceptData(b7)){return}var cb,ca,cc=bI.expando,cd=b7.nodeType,e=cd?bI.cache:b7,b6=cd?b7[cc]:b7[cc]&&cc;if((!b6||!e[b6]||(!b8&&!e[b6].data))&&b9===undefined&&typeof i==="string"){return}if(!b6){if(cd){b6=b7[cc]=aP.pop()||bI.guid++}else{b6=cc}}if(!e[b6]){e[b6]=cd?{}:{toJSON:bI.noop}}if(typeof i==="object"||typeof i==="function"){if(b8){e[b6]=bI.extend(e[b6],i)}else{e[b6].data=bI.extend(e[b6].data,i)}}ca=e[b6];if(!b8){if(!ca.data){ca.data={}}ca=ca.data}if(b9!==undefined){ca[bI.camelCase(i)]=b9}if(typeof i==="string"){cb=ca[i];if(cb==null){cb=ca[bI.camelCase(i)]}}else{cb=ca}return cb}function ab(b9,b7,e){if(!bI.acceptData(b9)){return}var cb,b8,ca=b9.nodeType,b6=ca?bI.cache:b9,cc=ca?b9[bI.expando]:bI.expando;
 if(!b6[cc]){return}if(b7){cb=e?b6[cc]:b6[cc].data;if(cb){if(!bI.isArray(b7)){if(b7 in cb){b7=[b7]}else{b7=bI.camelCase(b7);if(b7 in cb){b7=[b7]}else{b7=b7.split(" ")}}}else{b7=b7.concat(bI.map(b7,bI.camelCase))}b8=b7.length;while(b8--){delete cb[b7[b8]]}if(e?!Q(cb):!bI.isEmptyObject(cb)){return}}}if(!e){delete b6[cc].data;if(!Q(b6[cc])){return}}if(ca){bI.cleanData([b9],true)}else{if(D.deleteExpando||b6!=b6.window){delete b6[cc]}else{b6[cc]=null}}}bI.extend({cache:{},noData:{"applet ":true,"embed ":true,"object ":"clsid:D27CDB6E-AE6D-11cf-96B8-444553540000"},hasData:function(e){e=e.nodeType?bI.cache[e[bI.expando]]:e[bI.expando];return !!e&&!Q(e)},data:function(i,e,b6){return bc(i,e,b6)},removeData:function(i,e){return ab(i,e)},_data:function(i,e,b6){return bc(i,e,b6,true)},_removeData:function(i,e){return ab(i,e,true)}});bI.fn.extend({data:function(b8,cb){var b7,b6,ca,b9=this[0],e=b9&&b9.attributes;if(b8===undefined){if(this.length){ca=bI.data(b9);if(b9.nodeType===1&&!bI._data(b9,"pa
 rsedAttrs")){b7=e.length;while(b7--){if(e[b7]){b6=e[b7].name;if(b6.indexOf("data-")===0){b6=bI.camelCase(b6.slice(5));bA(b9,b6,ca[b6])}}}bI._data(b9,"parsedAttrs",true)}}return ca}if(typeof b8==="object"){return this.each(function(){bI.data(this,b8)})}return arguments.length>1?this.each(function(){bI.data(this,b8,cb)}):b9?bA(b9,b8,bI.data(b9,b8)):undefined},removeData:function(e){return this.each(function(){bI.removeData(this,e)})}});bI.extend({queue:function(b6,i,b7){var e;if(b6){i=(i||"fx")+"queue";e=bI._data(b6,i);if(b7){if(!e||bI.isArray(b7)){e=bI._data(b6,i,bI.makeArray(b7))}else{e.push(b7)}}return e||[]}},dequeue:function(b9,b8){b8=b8||"fx";var i=bI.queue(b9,b8),ca=i.length,b7=i.shift(),e=bI._queueHooks(b9,b8),b6=function(){bI.dequeue(b9,b8)};if(b7==="inprogress"){b7=i.shift();ca--}if(b7){if(b8==="fx"){i.unshift("inprogress")}delete e.stop;b7.call(b9,b6,e)}if(!ca&&e){e.empty.fire()}},_queueHooks:function(b6,i){var e=i+"queueHooks";return bI._data(b6,e)||bI._data(b6,e,{empty:bI
 .Callbacks("once memory").add(function(){bI._removeData(b6,i+"queue");bI._removeData(b6,e)})})}});bI.fn.extend({queue:function(e,i){var b6=2;if(typeof e!=="string"){i=e;e="fx";b6--}if(arguments.length<b6){return bI.queue(this[0],e)}return i===undefined?this:this.each(function(){var b7=bI.queue(this,e,i);bI._queueHooks(this,e);if(e==="fx"&&b7[0]!=="inprogress"){bI.dequeue(this,e)}})},dequeue:function(e){return this.each(function(){bI.dequeue(this,e)})},clearQueue:function(e){return this.queue(e||"fx",[])},promise:function(b7,cb){var b6,b8=1,cc=bI.Deferred(),ca=this,e=this.length,b9=function(){if(!(--b8)){cc.resolveWith(ca,[ca])}};if(typeof b7!=="string"){cb=b7;b7=undefined}b7=b7||"fx";while(e--){b6=bI._data(ca[e],b7+"queueHooks");if(b6&&b6.empty){b8++;b6.empty.add(b9)}}b9();return cc.promise(cb)}});var aE=(/[+-]?(?:\d*\.|)\d+(?:[eE][+-]?\d+|)/).source;var bT=["Top","Right","Bottom","Left"];var S=function(i,e){i=e||i;return bI.css(i,"display")==="none"||!bI.contains(i.ownerDocument,i)
 };var aB=bI.access=function(e,ca,cc,cb,b8,ce,cd){var b7=0,b6=e.length,b9=cc==null;if(bI.type(cc)==="object"){b8=true;for(b7 in cc){bI.access(e,ca,b7,cc[b7],true,ce,cd)}}else{if(cb!==undefined){b8=true;if(!bI.isFunction(cb)){cd=true}if(b9){if(cd){ca.call(e,cb);ca=null}else{b9=ca;ca=function(cf,i,cg){return b9.call(bI(cf),cg)}}}if(ca){for(;b7<b6;b7++){ca(e[b7],cc,cd?cb:cb.call(e[b7],b7,ca(e[b7],cc)))}}}}return b8?e:b9?ca.call(e):b6?ca(e[0],cc):ce};var aM=(/^(?:checkbox|radio)$/i);(function(){var i=n.createElement("input"),b8=n.createElement("div"),b6=n.createDocumentFragment();b8.innerHTML="  <link/><table></table><a href='/a'>a</a><input type='checkbox'/>";D.leadingWhitespace=b8.firstChild.nodeType===3;D.tbody=!b8.getElementsByTagName("tbody").length;D.htmlSerialize=!!b8.getElementsByTagName("link").length;D.html5Clone=n.createElement("nav").cloneNode(true).outerHTML!=="<:nav></:nav>";i.type="checkbox";i.checked=true;b6.appendChild(i);D.appendChecked=i.checked;b8.innerHTML="<textarea
 >x</textarea>";D.noCloneChecked=!!b8.cloneNode(true).lastChild.defaultValue;b6.appendChild(b8);b8.innerHTML="<input type='radio' checked='checked' name='t'/>";D.checkClone=b8.cloneNode(true).cloneNode(true).lastChild.checked;D.noCloneEvent=true;if(b8.attachEvent){b8.attachEvent("onclick",function(){D.noCloneEvent=false});b8.cloneNode(true).click()}if(D.deleteExpando==null){D.deleteExpando=true;try{delete b8.test}catch(b7){D.deleteExpando=false}}})();(function(){var b6,e,b7=n.createElement("div");for(b6 in {submit:true,change:true,focusin:true}){e="on"+b6;if(!(D[b6+"Bubbles"]=e in a5)){b7.setAttribute(e,"t");D[b6+"Bubbles"]=b7.attributes[e].expando===false}}b7=null})();var bG=/^(?:input|select|textarea)$/i,a6=/^key/,bM=/^(?:mouse|pointer|contextmenu)|click/,bC=/^(?:focusinfocus|focusoutblur)$/,bx=/^([^.]*)(?:\.(.+)|)$/;function U(){return true}function Z(){return false}function am(){try{return n.activeElement}catch(e){}}bI.event={global:{},add:function(b8,cd,ci,ca,b9){var cb,cj,ck,b6
 ,cf,cc,ch,b7,cg,e,i,ce=bI._data(b8);if(!ce){return}if(ci.handler){b6=ci;ci=b6.handler;b9=b6.selector}if(!ci.guid){ci.guid=bI.guid++}if(!(cj=ce.events)){cj=ce.events={}}if(!(cc=ce.handle)){cc=ce.handle=function(cl){return typeof bI!==aC&&(!cl||bI.event.triggered!==cl.type)?bI.event.dispatch.apply(cc.elem,arguments):undefined};cc.elem=b8}cd=(cd||"").match(aF)||[""];ck=cd.length;while(ck--){cb=bx.exec(cd[ck])||[];cg=i=cb[1];e=(cb[2]||"").split(".").sort();if(!cg){continue}cf=bI.event.special[cg]||{};cg=(b9?cf.delegateType:cf.bindType)||cg;cf=bI.event.special[cg]||{};ch=bI.extend({type:cg,origType:i,data:ca,handler:ci,guid:ci.guid,selector:b9,needsContext:b9&&bI.expr.match.needsContext.test(b9),namespace:e.join(".")},b6);if(!(b7=cj[cg])){b7=cj[cg]=[];b7.delegateCount=0;if(!cf.setup||cf.setup.call(b8,ca,e,cc)===false){if(b8.addEventListener){b8.addEventListener(cg,cc,false)}else{if(b8.attachEvent){b8.attachEvent("on"+cg,cc)}}}}if(cf.add){cf.add.call(b8,ch);if(!ch.handler.guid){ch.handler
 .guid=ci.guid}}if(b9){b7.splice(b7.delegateCount++,0,ch)}else{b7.push(ch)}bI.event.global[cg]=true}b8=null},remove:function(b7,cd,ck,b8,cc){var ca,ch,cb,b9,cj,ci,cf,b6,cg,e,i,ce=bI.hasData(b7)&&bI._data(b7);if(!ce||!(ci=ce.events)){return}cd=(cd||"").match(aF)||[""];cj=cd.length;while(cj--){cb=bx.exec(cd[cj])||[];cg=i=cb[1];e=(cb[2]||"").split(".").sort();if(!cg){for(cg in ci){bI.event.remove(b7,cg+cd[cj],ck,b8,true)}continue}cf=bI.event.special[cg]||{};cg=(b8?cf.delegateType:cf.bindType)||cg;b6=ci[cg]||[];cb=cb[2]&&new RegExp("(^|\\.)"+e.join("\\.(?:.*\\.|)")+"(\\.|$)");b9=ca=b6.length;while(ca--){ch=b6[ca];if((cc||i===ch.origType)&&(!ck||ck.guid===ch.guid)&&(!cb||cb.test(ch.namespace))&&(!b8||b8===ch.selector||b8==="**"&&ch.selector)){b6.splice(ca,1);if(ch.selector){b6.delegateCount--}if(cf.remove){cf.remove.call(b7,ch)}}}if(b9&&!b6.length){if(!cf.teardown||cf.teardown.call(b7,e,ce.handle)===false){bI.removeEvent(b7,cg,ce.handle)}delete ci[cg]}}if(bI.isEmptyObject(ci)){delete ce.h
 andle;bI._removeData(b7,"events")}},trigger:function(b6,cd,b9,ck){var ce,b8,ci,cj,cg,cc,cb,ca=[b9||n],ch=K.call(b6,"type")?b6.type:b6,b7=K.call(b6,"namespace")?b6.namespace.split("."):[];ci=cc=b9=b9||n;if(b9.nodeType===3||b9.nodeType===8){return}if(bC.test(ch+bI.event.triggered)){return}if(ch.indexOf(".")>=0){b7=ch.split(".");ch=b7.shift();b7.sort()}b8=ch.indexOf(":")<0&&"on"+ch;b6=b6[bI.expando]?b6:new bI.Event(ch,typeof b6==="object"&&b6);b6.isTrigger=ck?2:3;b6.namespace=b7.join(".");b6.namespace_re=b6.namespace?new RegExp("(^|\\.)"+b7.join("\\.(?:.*\\.|)")+"(\\.|$)"):null;b6.result=undefined;if(!b6.target){b6.target=b9}cd=cd==null?[b6]:bI.makeArray(cd,[b6]);cg=bI.event.special[ch]||{};if(!ck&&cg.trigger&&cg.trigger.apply(b9,cd)===false){return}if(!ck&&!cg.noBubble&&!bI.isWindow(b9)){cj=cg.delegateType||ch;if(!bC.test(cj+ch)){ci=ci.parentNode}for(;ci;ci=ci.parentNode){ca.push(ci);cc=ci}if(cc===(b9.ownerDocument||n)){ca.push(cc.defaultView||cc.parentWindow||a5)}}cb=0;while((ci=ca[c
 b++])&&!b6.isPropagationStopped()){b6.type=cb>1?cj:cg.bindType||ch;ce=(bI._data(ci,"events")||{})[b6.type]&&bI._data(ci,"handle");if(ce){ce.apply(ci,cd)}ce=b8&&ci[b8];if(ce&&ce.apply&&bI.acceptData(ci)){b6.result=ce.apply(ci,cd);if(b6.result===false){b6.preventDefault()}}}b6.type=ch;if(!ck&&!b6.isDefaultPrevented()){if((!cg._default||cg._default.apply(ca.pop(),cd)===false)&&bI.acceptData(b9)){if(b8&&b9[ch]&&!bI.isWindow(b9)){cc=b9[b8];if(cc){b9[b8]=null}bI.event.triggered=ch;try{b9[ch]()}catch(cf){}bI.event.triggered=undefined;if(cc){b9[b8]=cc}}}}return b6.result},dispatch:function(e){e=bI.event.fix(e);var b9,ca,ce,b6,b8,cd=[],cc=P.call(arguments),b7=(bI._data(this,"events")||{})[e.type]||[],cb=bI.event.special[e.type]||{};cc[0]=e;e.delegateTarget=this;if(cb.preDispatch&&cb.preDispatch.call(this,e)===false){return}cd=bI.event.handlers.call(this,e,b7);b9=0;while((b6=cd[b9++])&&!e.isPropagationStopped()){e.currentTarget=b6.elem;b8=0;while((ce=b6.handlers[b8++])&&!e.isImmediatePropagat
 ionStopped()){if(!e.namespace_re||e.namespace_re.test(ce.namespace)){e.handleObj=ce;e.data=ce.data;ca=((bI.event.special[ce.origType]||{}).handle||ce.handler).apply(b6.elem,cc);if(ca!==undefined){if((e.result=ca)===false){e.preventDefault();e.stopPropagation()}}}}}if(cb.postDispatch){cb.postDispatch.call(this,e)}return e.result},handlers:function(e,b7){var b6,cc,ca,b9,cb=[],b8=b7.delegateCount,cd=e.target;if(b8&&cd.nodeType&&(!e.button||e.type!=="click")){for(;cd!=this;cd=cd.parentNode||this){if(cd.nodeType===1&&(cd.disabled!==true||e.type!=="click")){ca=[];for(b9=0;b9<b8;b9++){cc=b7[b9];b6=cc.selector+" ";if(ca[b6]===undefined){ca[b6]=cc.needsContext?bI(b6,this).index(cd)>=0:bI.find(b6,this,null,[cd]).length}if(ca[b6]){ca.push(cc)}}if(ca.length){cb.push({elem:cd,handlers:ca})}}}}if(b8<b7.length){cb.push({elem:this,handlers:b7.slice(b8)})}return cb},fix:function(b8){if(b8[bI.expando]){return b8}var b6,cb,ca,b7=b8.type,e=b8,b9=this.fixHooks[b7];if(!b9){this.fixHooks[b7]=b9=bM.test(b7
 )?this.mouseHooks:a6.test(b7)?this.keyHooks:{}}ca=b9.props?this.props.concat(b9.props):this.props;b8=new bI.Event(e);b6=ca.length;while(b6--){cb=ca[b6];b8[cb]=e[cb]}if(!b8.target){b8.target=e.srcElement||n}if(b8.target.nodeType===3){b8.target=b8.target.parentNode}b8.metaKey=!!b8.metaKey;return b9.filter?b9.filter(b8,e):b8},props:"altKey bubbles cancelable ctrlKey currentTarget eventPhase metaKey relatedTarget shiftKey target timeStamp view which".split(" "),fixHooks:{},keyHooks:{props:"char charCode key keyCode".split(" "),filter:function(i,e){if(i.which==null){i.which=e.charCode!=null?e.charCode:e.keyCode}return i}},mouseHooks:{props:"button buttons clientX clientY fromElement offsetX offsetY pageX pageY screenX screenY toElement".split(" "),filter:function(b7,b6){var e,b8,b9,i=b6.button,ca=b6.fromElement;if(b7.pageX==null&&b6.clientX!=null){b8=b7.target.ownerDocument||n;b9=b8.documentElement;e=b8.body;b7.pageX=b6.clientX+(b9&&b9.scrollLeft||e&&e.scrollLeft||0)-(b9&&b9.clientLeft||
 e&&e.clientLeft||0);b7.pageY=b6.clientY+(b9&&b9.scrollTop||e&&e.scrollTop||0)-(b9&&b9.clientTop||e&&e.clientTop||0)}if(!b7.relatedTarget&&ca){b7.relatedTarget=ca===b7.target?b6.toElement:ca}if(!b7.which&&i!==undefined){b7.which=(i&1?1:(i&2?3:(i&4?2:0)))}return b7}},special:{load:{noBubble:true},focus:{trigger:function(){if(this!==am()&&this.focus){try{this.focus();return false}catch(i){}}},delegateType:"focusin"},blur:{trigger:function(){if(this===am()&&this.blur){this.blur();return false}},delegateType:"focusout"},click:{trigger:function(){if(bI.nodeName(this,"input")&&this.type==="checkbox"&&this.click){this.click();return false}},_default:function(e){return bI.nodeName(e.target,"a")}},beforeunload:{postDispatch:function(e){if(e.result!==undefined&&e.originalEvent){e.originalEvent.returnValue=e.result}}}},simulate:function(b6,b8,b7,i){var b9=bI.extend(new bI.Event(),b7,{type:b6,isSimulated:true,originalEvent:{}});if(i){bI.event.trigger(b9,null,b8)}else{bI.event.dispatch.call(b8,b9
 )}if(b9.isDefaultPrevented()){b7.preventDefault()}}};bI.removeEvent=n.removeEventListener?function(i,e,b6){if(i.removeEventListener){i.removeEventListener(e,b6,false)}}:function(b6,i,b7){var e="on"+i;if(b6.detachEvent){if(typeof b6[e]===aC){b6[e]=null}b6.detachEvent(e,b7)}};bI.Event=function(i,e){if(!(this instanceof bI.Event)){return new bI.Event(i,e)}if(i&&i.type){this.originalEvent=i;this.type=i.type;this.isDefaultPrevented=i.defaultPrevented||i.defaultPrevented===undefined&&i.returnValue===false?U:Z}else{this.type=i}if(e){bI.extend(this,e)}this.timeStamp=i&&i.timeStamp||bI.now();this[bI.expando]=true};bI.Event.prototype={isDefaultPrevented:Z,isPropagationStopped:Z,isImmediatePropagationStopped:Z,preventDefault:function(){var i=this.originalEvent;this.isDefaultPrevented=U;if(!i){return}if(i.preventDefault){i.preventDefault()}else{i.returnValue=false}},stopPropagation:function(){var i=this.originalEvent;this.isPropagationStopped=U;if(!i){return}if(i.stopPropagation){i.stopPropagat
 ion()}i.cancelBubble=true},stopImmediatePropagation:function(){var i=this.originalEvent;this.isImmediatePropagationStopped=U;if(i&&i.stopImmediatePropagation){i.stopImmediatePropagation()}this.stopPropagation()}};bI.each({mouseenter:"mouseover",mouseleave:"mouseout",pointerenter:"pointerover",pointerleave:"pointerout"},function(i,e){bI.event.special[i]={delegateType:e,bindType:e,handle:function(b8){var b6,ca=this,b9=b8.relatedTarget,b7=b8.handleObj;if(!b9||(b9!==ca&&!bI.contains(ca,b9))){b8.type=b7.origType;b6=b7.handler.apply(this,arguments);b8.type=e}return b6}}});if(!D.submitBubbles){bI.event.special.submit={setup:function(){if(bI.nodeName(this,"form")){return false}bI.event.add(this,"click._submit keypress._submit",function(b7){var b6=b7.target,i=bI.nodeName(b6,"input")||bI.nodeName(b6,"button")?b6.form:undefined;if(i&&!bI._data(i,"submitBubbles")){bI.event.add(i,"submit._submit",function(e){e._submit_bubble=true});bI._data(i,"submitBubbles",true)}})},postDispatch:function(e){if
 (e._submit_bubble){delete e._submit_bubble;if(this.parentNode&&!e.isTrigger){bI.event.simulate("submit",this.parentNode,e,true)}}},teardown:function(){if(bI.nodeName(this,"form")){return false}bI.event.remove(this,"._submit")}}}if(!D.changeBubbles){bI.event.special.change={setup:function(){if(bG.test(this.nodeName)){if(this.type==="checkbox"||this.type==="radio"){bI.event.add(this,"propertychange._change",function(e){if(e.originalEvent.propertyName==="checked"){this._just_changed=true}});bI.event.add(this,"click._change",function(e){if(this._just_changed&&!e.isTrigger){this._just_changed=false}bI.event.simulate("change",this,e,true)})}return false}bI.event.add(this,"beforeactivate._change",function(b6){var i=b6.target;if(bG.test(i.nodeName)&&!bI._data(i,"changeBubbles")){bI.event.add(i,"change._change",function(e){if(this.parentNode&&!e.isSimulated&&!e.isTrigger){bI.event.simulate("change",this.parentNode,e,true)}});bI._data(i,"changeBubbles",true)}})},handle:function(i){var e=i.tar
 get;if(this!==e||i.isSimulated||i.isTrigger||(e.type!=="radio"&&e.type!=="checkbox")){return i.handleObj.handler.apply(this,arguments)}},teardown:function(){bI.event.remove(this,"._change");return !bG.test(this.nodeName)}}}if(!D.focusinBubbles){bI.each({focus:"focusin",blur:"focusout"},function(b6,e){var i=function(b7){bI.event.simulate(e,b7.target,bI.event.fix(b7),true)};bI.event.special[e]={setup:function(){var b8=this.ownerDocument||this,b7=bI._data(b8,e);if(!b7){b8.addEventListener(b6,i,true)}bI._data(b8,e,(b7||0)+1)},teardown:function(){var b8=this.ownerDocument||this,b7=bI._data(b8,e)-1;if(!b7){b8.removeEventListener(b6,i,true);bI._removeData(b8,e)}else{bI._data(b8,e,b7)}}}})}bI.fn.extend({on:function(b6,e,b9,b8,i){var b7,ca;if(typeof b6==="object"){if(typeof e!=="string"){b9=b9||e;e=undefined}for(b7 in b6){this.on(b7,e,b9,b6[b7],i)}return this}if(b9==null&&b8==null){b8=e;b9=e=undefined}else{if(b8==null){if(typeof e==="string"){b8=b9;b9=undefined}else{b8=b9;b9=e;e=undefined}}}
 if(b8===false){b8=Z}else{if(!b8){return this}}if(i===1){ca=b8;b8=function(cb){bI().off(cb);return ca.apply(this,arguments)};b8.guid=ca.guid||(ca.guid=bI.guid++)}return this.each(function(){bI.event.add(this,b6,b8,b9,e)})},one:function(i,e,b7,b6){return this.on(i,e,b7,b6,1)},off:function(b6,e,b8){var i,b7;if(b6&&b6.preventDefault&&b6.handleObj){i=b6.handleObj;bI(b6.delegateTarget).off(i.namespace?i.origType+"."+i.namespace:i.origType,i.selector,i.handler);return this}if(typeof b6==="object"){for(b7 in b6){this.off(b7,e,b6[b7])}return this}if(e===false||typeof e==="function"){b8=e;e=undefined}if(b8===false){b8=Z}return this.each(function(){bI.event.remove(this,b6,b8,e)})},trigger:function(e,i){return this.each(function(){bI.event.trigger(e,i,this)})},triggerHandler:function(e,b6){var i=this[0];if(i){return bI.event.trigger(e,b6,i,true)}}});function B(e){var b6=d.split("|"),i=e.createDocumentFragment();if(i.createElement){while(b6.length){i.createElement(b6.pop())}}return i}var d="abbr
 |article|aside|audio|bdi|canvas|data|datalist|details|figcaption|figure|footer|header|hgroup|mark|meter|nav|output|progress|section|summary|time|video",aD=/ jQuery\d+="(?:null|\d+)"/g,M=new RegExp("<(?:"+d+")[\\s/>]","i"),b5=/^\s+/,aH=/<(?!area|br|col|embed|hr|img|input|link|meta|param)(([\w:]+)[^>]*)\/>/gi,o=/<([\w:]+)/,b0=/<tbody/i,L=/<|&#?\w+;/,an=/<(?:script|style|link)/i,bW=/checked\s*(?:[^=]|=\s*.checked.)/i,bB=/^$|\/(?:java|ecma)script/i,ar=/^true\/(.*)/,aO=/^\s*<!(?:\[CDATA\[|--)|(?:\]\]|--)>\s*$/g,W={option:[1,"<select multiple='multiple'>","</select>"],legend:[1,"<fieldset>","</fieldset>"],area:[1,"<map>","</map>"],param:[1,"<object>","</object>"],thead:[1,"<table>","</table>"],tr:[2,"<table><tbody>","</tbody></table>"],col:[2,"<table><tbody></tbody><colgroup>","</colgroup></table>"],td:[3,"<table><tbody><tr>","</tr></tbody></table>"],_default:D.htmlSerialize?[0,"",""]:[1,"X<div>","</div>"]},aT=B(n),k=aT.appendChild(n.createElement("div"));W.optgroup=W.option;W.tbody=W.tfo
 ot=W.colgroup=W.caption=W.thead;W.th=W.td;function l(b8,e){var b6,b9,b7=0,ca=typeof b8.getElementsByTagName!==aC?b8.getElementsByTagName(e||"*"):typeof b8.querySelectorAll!==aC?b8.querySelectorAll(e||"*"):undefined;if(!ca){for(ca=[],b6=b8.childNodes||b8;(b9=b6[b7])!=null;b7++){if(!e||bI.nodeName(b9,e)){ca.push(b9)}else{bI.merge(ca,l(b9,e))}}}return e===undefined||e&&bI.nodeName(b8,e)?bI.merge([b8],ca):ca}function bY(e){if(aM.test(e.type)){e.defaultChecked=e.checked}}function a3(i,e){return bI.nodeName(i,"table")&&bI.nodeName(e.nodeType!==11?e:e.firstChild,"tr")?i.getElementsByTagName("tbody")[0]||i.appendChild(i.ownerDocument.createElement("tbody")):i}function u(e){e.type=(bI.find.attr(e,"type")!==null)+"/"+e.type;return e}function bf(i){var e=ar.exec(i.type);if(e){i.type=e[1]}else{i.removeAttribute("type")}return i}function bu(e,b7){var b8,b6=0;for(;(b8=e[b6])!=null;b6++){bI._data(b8,"globalEval",!b7||bI._data(b7[b6],"globalEval"))}}function at(cc,b6){if(b6.nodeType!==1||!bI.hasDat
 a(cc)){return}var b9,b8,e,cb=bI._data(cc),ca=bI._data(b6,cb),b7=cb.events;if(b7){delete ca.handle;ca.events={};for(b9 in b7){for(b8=0,e=b7[b9].length;b8<e;b8++){bI.event.add(b6,b9,b7[b9][b8])}}}if(ca.data){ca.data=bI.extend({},ca.data)}}function T(b8,i){var b9,b7,b6;if(i.nodeType!==1){return}b9=i.nodeName.toLowerCase();if(!D.noCloneEvent&&i[bI.expando]){b6=bI._data(i);for(b7 in b6.events){bI.removeEvent(i,b7,b6.handle)}i.removeAttribute(bI.expando)}if(b9==="script"&&i.text!==b8.text){u(i).text=b8.text;bf(i)}else{if(b9==="object"){if(i.parentNode){i.outerHTML=b8.outerHTML}if(D.html5Clone&&(b8.innerHTML&&!bI.trim(i.innerHTML))){i.innerHTML=b8.innerHTML}}else{if(b9==="input"&&aM.test(b8.type)){i.defaultChecked=i.checked=b8.checked;if(i.value!==b8.value){i.value=b8.value}}else{if(b9==="option"){i.defaultSelected=i.selected=b8.defaultSelected}else{if(b9==="input"||b9==="textarea"){i.defaultValue=b8.defaultValue}}}}}}bI.extend({clone:function(b6,b8,e){var ca,b7,cd,b9,cb,cc=bI.contains(b6.
 ownerDocument,b6);if(D.html5Clone||bI.isXMLDoc(b6)||!M.test("<"+b6.nodeName+">")){cd=b6.cloneNode(true)}else{k.innerHTML=b6.outerHTML;k.removeChild(cd=k.firstChild)}if((!D.noCloneEvent||!D.noCloneChecked)&&(b6.nodeType===1||b6.nodeType===11)&&!bI.isXMLDoc(b6)){ca=l(cd);cb=l(b6);for(b9=0;(b7=cb[b9])!=null;++b9){if(ca[b9]){T(b7,ca[b9])}}}if(b8){if(e){cb=cb||l(b6);ca=ca||l(cd);for(b9=0;(b7=cb[b9])!=null;b9++){at(b7,ca[b9])}}else{at(b6,cd)}}ca=l(cd,"script");if(ca.length>0){bu(ca,!cc&&l(b6,"script"))}ca=cb=b7=null;return cd},buildFragment:function(b6,b8,cd,ci){var ce,ca,cc,ch,cj,cg,b7,cb=b6.length,b9=B(b8),e=[],cf=0;for(;cf<cb;cf++){ca=b6[cf];if(ca||ca===0){if(bI.type(ca)==="object"){bI.merge(e,ca.nodeType?[ca]:ca)}else{if(!L.test(ca)){e.push(b8.createTextNode(ca))}else{ch=ch||b9.appendChild(b8.createElement("div"));cj=(o.exec(ca)||["",""])[1].toLowerCase();b7=W[cj]||W._default;ch.innerHTML=b7[1]+ca.replace(aH,"<$1></$2>")+b7[2];ce=b7[0];while(ce--){ch=ch.lastChild}if(!D.leadingWhitespa
 ce&&b5.test(ca)){e.push(b8.createTextNode(b5.exec(ca)[0]))}if(!D.tbody){ca=cj==="table"&&!b0.test(ca)?ch.firstChild:b7[1]==="<table>"&&!b0.test(ca)?ch:0;ce=ca&&ca.childNodes.length;while(ce--){if(bI.nodeName((cg=ca.childNodes[ce]),"tbody")&&!cg.childNodes.length){ca.removeChild(cg)}}}bI.merge(e,ch.childNodes);ch.textContent="";while(ch.firstChild){ch.removeChild(ch.firstChild)}ch=b9.lastChild}}}}if(ch){b9.removeChild(ch)}if(!D.appendChecked){bI.grep(l(e,"input"),bY)}cf=0;while((ca=e[cf++])){if(ci&&bI.inArray(ca,ci)!==-1){continue}cc=bI.contains(ca.ownerDocument,ca);ch=l(b9.appendChild(ca),"script");if(cc){bu(ch)}if(cd){ce=0;while((ca=ch[ce++])){if(bB.test(ca.type||"")){cd.push(ca)}}}}ch=null;return b9},cleanData:function(b6,ce){var b8,cd,b7,b9,ca=0,cf=bI.expando,e=bI.cache,cb=D.deleteExpando,cc=bI.event.special;for(;(b8=b6[ca])!=null;ca++){if(ce||bI.acceptData(b8)){b7=b8[cf];b9=b7&&e[b7];if(b9){if(b9.events){for(cd in b9.events){if(cc[cd]){bI.event.remove(b8,cd)}else{bI.removeEvent(
 b8,cd,b9.handle)}}}if(e[b7]){delete e[b7];if(cb){delete b8[cf]}else{if(typeof b8.removeAttribute!==aC){b8.removeAttribute(cf)}else{b8[cf]=null}}aP.push(b7)}}}}}});bI.fn.extend({text:function(e){return aB(this,function(i){return i===undefined?bI.text(this):this.empty().append((this[0]&&this[0].ownerDocument||n).createTextNode(i))},null,e,arguments.length)},append:function(){return this.domManip(arguments,function(e){if(this.nodeType===1||this.nodeType===11||this.nodeType===9){var i=a3(this,e);i.appendChild(e)}})},prepend:function(){return this.domManip(arguments,function(e){if(this.nodeType===1||this.nodeType===11||this.nodeType===9){var i=a3(this,e);i.insertBefore(e,i.firstChild)}})},before:function(){return this.domManip(arguments,function(e){if(this.parentNode){this.parentNode.insertBefore(e,this)}})},after:function(){return this.domManip(arguments,function(e){if(this.parentNode){this.parentNode.insertBefore(e,this.nextSibling)}})},remove:function(e,b9){var b8,b6=e?bI.filter(e,thi
 s):this,b7=0;for(;(b8=b6[b7])!=null;b7++){if(!b9&&b8.nodeType===1){bI.cleanData(l(b8))}if(b8.parentNode){if(b9&&bI.contains(b8.ownerDocument,b8)){bu(l(b8,"script"))}b8.parentNode.removeChild(b8)}}return this},empty:function(){var b6,e=0;for(;(b6=this[e])!=null;e++){if(b6.nodeType===1){bI.cleanData(l(b6,false))}while(b6.firstChild){b6.removeChild(b6.firstChild)}if(b6.options&&bI.nodeName(b6,"select")){b6.options.length=0}}return this},clone:function(i,e){i=i==null?false:i;e=e==null?i:e;return this.map(function(){return bI.clone(this,i,e)})},html:function(e){return aB(this,function(b9){var b8=this[0]||{},b7=0,b6=this.length;if(b9===undefined){return b8.nodeType===1?b8.innerHTML.replace(aD,""):undefined}if(typeof b9==="string"&&!an.test(b9)&&(D.htmlSerialize||!M.test(b9))&&(D.leadingWhitespace||!b5.test(b9))&&!W[(o.exec(b9)||["",""])[1].toLowerCase()]){b9=b9.replace(aH,"<$1></$2>");try{for(;b7<b6;b7++){b8=this[b7]||{};if(b8.nodeType===1){bI.cleanData(l(b8,false));b8.innerHTML=b9}}b8=0}
 catch(ca){}}if(b8){this.empty().append(b9)}},null,e,arguments.length)},replaceWith:function(){var e=arguments[0];this.domManip(arguments,function(i){e=this.parentNode;bI.cleanData(l(this));if(e){e.replaceChild(i,this)}});return e&&(e.length||e.nodeType)?this:this.remove()},detach:function(e){return this.remove(e,true)},domManip:function(cd,ci){cd=az.apply([],cd);var cb,b7,e,b9,cg,cc,ca=0,b8=this.length,cf=this,ch=b8-1,ce=cd[0],b6=bI.isFunction(ce);if(b6||(b8>1&&typeof ce==="string"&&!D.checkClone&&bW.test(ce))){return this.each(function(cj){var i=cf.eq(cj);if(b6){cd[0]=ce.call(this,cj,i.html())}i.domManip(cd,ci)})}if(b8){cc=bI.buildFragment(cd,this[0].ownerDocument,false,this);cb=cc.firstChild;if(cc.childNodes.length===1){cc=cb}if(cb){b9=bI.map(l(cc,"script"),u);e=b9.length;for(;ca<b8;ca++){b7=cc;if(ca!==ch){b7=bI.clone(b7,true,true);if(e){bI.merge(b9,l(b7,"script"))}}ci.call(this[ca],b7,ca)}if(e){cg=b9[b9.length-1].ownerDocument;bI.map(b9,bf);for(ca=0;ca<e;ca++){b7=b9[ca];if(bB.tes
 t(b7.type||"")&&!bI._data(b7,"globalEval")&&bI.contains(cg,b7)){if(b7.src){if(bI._evalUrl){bI._evalUrl(b7.src)}}else{bI.globalEval((b7.text||b7.textContent||b7.innerHTML||"").replace(aO,""))}}}}cc=cb=null}}return this}});bI.each({appendTo:"append",prependTo:"prepend",insertBefore:"before",insertAfter:"after",replaceAll:"replaceWith"},function(e,i){bI.fn[e]=function(b6){var b7,b9=0,b8=[],cb=bI(b6),ca=cb.length-1;for(;b9<=ca;b9++){b7=b9===ca?this:this.clone(true);bI(cb[b9])[i](b7);x.apply(b8,b7.get())}return this.pushStack(b8)}});var aI,bl={};function a4(e,b8){var i,b6=bI(b8.createElement(e)).appendTo(b8.body),b7=a5.getDefaultComputedStyle&&(i=a5.getDefaultComputedStyle(b6[0]))?i.display:bI.css(b6[0],"display");b6.detach();return b7}function a0(b6){var i=n,e=bl[b6];if(!e){e=a4(b6,i);if(e==="none"||!e){aI=(aI||bI("<iframe frameborder='0' width='0' height='0'/>")).appendTo(i.documentElement);i=(aI[0].contentWindow||aI[0].contentDocument).document;i.write();i.close();e=a4(b6,i);aI.detach
 ()}bl[b6]=e}return e}(function(){var e;D.shrinkWrapBlocks=function(){if(e!=null){return e}e=false;var b7,i,b6;i=n.getElementsByTagName("body")[0];if(!i||!i.style){return}b7=n.createElement("div");b6=n.createElement("div");b6.style.cssText="position:absolute;border:0;width:0;height:0;top:0;left:-9999px";i.appendChild(b6).appendChild(b7);if(typeof b7.style.zoom!==aC){b7.style.cssText="-webkit-box-sizing:content-box;-moz-box-sizing:content-box;box-sizing:content-box;display:block;margin:0;border:0;padding:1px;width:1px;zoom:1";b7.appendChild(n.createElement("div")).style.width="5px";e=b7.offsetWidth!==3}i.removeChild(b6);return e}})();var aZ=(/^margin/);var Y=new RegExp("^("+aE+")(?!px)[a-z%]+$","i");var bq,G,bo=/^(top|right|bottom|left)$/;if(a5.getComputedStyle){bq=function(e){if(e.ownerDocument.defaultView.opener){return e.ownerDocument.defaultView.getComputedStyle(e,null)}return a5.getComputedStyle(e,null)};G=function(cb,i,ca){var b8,b7,b9,e,b6=cb.style;ca=ca||bq(cb);e=ca?ca.getProp
 ertyValue(i)||ca[i]:undefined;if(ca){if(e===""&&!bI.contains(cb.ownerDocument,cb)){e=bI.style(cb,i)}if(Y.test(e)&&aZ.test(i)){b8=b6.width;b7=b6.minWidth;b9=b6.maxWidth;b6.minWidth=b6.maxWidth=b6.width=e;e=ca.width;b6.width=b8;b6.minWidth=b7;b6.maxWidth=b9}}return e===undefined?e:e+""}}else{if(n.documentElement.currentStyle){bq=function(e){return e.currentStyle};G=function(ca,b7,b9){var cb,i,e,b6,b8=ca.style;b9=b9||bq(ca);b6=b9?b9[b7]:undefined;if(b6==null&&b8&&b8[b7]){b6=b8[b7]}if(Y.test(b6)&&!bo.test(b7)){cb=b8.left;i=ca.runtimeStyle;e=i&&i.left;if(e){i.left=ca.currentStyle.left}b8.left=b7==="fontSize"?"1em":b6;b6=b8.pixelLeft+"px";b8.left=cb;if(e){i.left=e}}return b6===undefined?b6:b6+""||"auto"}}}function a7(e,i){return{get:function(){var b6=e();if(b6==null){return}if(b6){delete this.get;return}return(this.get=i).apply(this,arguments)}}}(function(){var cb,b9,b7,ca,b6,b8,i;cb=n.createElement("div");cb.innerHTML="  <link/><table></table><a href='/a'>a</a><input type='checkbox'/>";b
 7=cb.getElementsByTagName("a")[0];b9=b7&&b7.style;if(!b9){return}b9.cssText="float:left;opacity:.5";D.opacity=b9.opacity==="0.5";D.cssFloat=!!b9.cssFloat;cb.style.backgroundClip="content-box";cb.cloneNode(true).style.backgroundClip="";D.clearCloneStyle=cb.style.backgroundClip==="content-box";D.boxSizing=b9.boxSizing===""||b9.MozBoxSizing===""||b9.WebkitBoxSizing==="";bI.extend(D,{reliableHiddenOffsets:function(){if(b8==null){e()}return b8},boxSizingReliable:function(){if(b6==null){e()}return b6},pixelPosition:function(){if(ca==null){e()}return ca},reliableMarginRight:function(){if(i==null){e()}return i}});function e(){var cf,cc,cd,ce;cc=n.getElementsByTagName("body")[0];if(!cc||!cc.style){return}cf=n.createElement("div");cd=n.createElement("div");cd.style.cssText="position:absolute;border:0;width:0;height:0;top:0;left:-9999px";cc.appendChild(cd).appendChild(cf);cf.style.cssText="-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box;display:block;margin-top:1
 %;top:1%;border:1px;padding:1px;width:4px;position:absolute";ca=b6=false;i=true;if(a5.getComputedStyle){ca=(a5.getComputedStyle(cf,null)||{}).top!=="1%";b6=(a5.getComputedStyle(cf,null)||{width:"4px"}).width==="4px";ce=cf.appendChild(n.createElement("div"));ce.style.cssText=cf.style.cssText="-webkit-box-sizing:content-box;-moz-box-sizing:content-box;box-sizing:content-box;display:block;margin:0;border:0;padding:0";ce.style.marginRight=ce.style.width="0";cf.style.width="1px";i=!parseFloat((a5.getComputedStyle(ce,null)||{}).marginRight);cf.removeChild(ce)}cf.innerHTML="<table><tr><td></td><td>t</td></tr></table>";ce=cf.getElementsByTagName("td");ce[0].style.cssText="margin:0;border:0;padding:0;display:none";b8=ce[0].offsetHeight===0;if(b8){ce[0].style.display="";ce[1].style.display="none";b8=ce[0].offsetHeight===0}cc.removeChild(cd)}})();bI.swap=function(b9,b8,ca,b7){var b6,i,e={};for(i in b8){e[i]=b9.style[i];b9.style[i]=b8[i]}b6=ca.apply(b9,b7||[]);for(i in b8){b9.style[i]=e[i]}retu
 rn b6};var bj=/alpha\([^)]*\)/i,aU=/opacity\s*=\s*([^)]*)/,H=/^(none|table(?!-c[ea]).+)/,bb=new RegExp("^("+aE+")(.*)$","i"),V=new RegExp("^([+-])=("+aE+")","i"),be={position:"absolute",visibility:"hidden",display:"block"},bD={letterSpacing:"0",fontWeight:"400"},aw=["Webkit","O","Moz","ms"];function c(b8,b6){if(b6 in b8){return b6}var b9=b6.charAt(0).toUpperCase()+b6.slice(1),e=b6,b7=aw.length;while(b7--){b6=aw[b7]+b9;if(b6 in b8){return b6}}return e}function s(ca,e){var cb,b8,b9,i=[],b6=0,b7=ca.length;for(;b6<b7;b6++){b8=ca[b6];if(!b8.style){continue}i[b6]=bI._data(b8,"olddisplay");cb=b8.style.display;if(e){if(!i[b6]&&cb==="none"){b8.style.display=""}if(b8.style.display===""&&S(b8)){i[b6]=bI._data(b8,"olddisplay",a0(b8.nodeName))}}else{b9=S(b8);if(cb&&cb!=="none"||!b9){bI._data(b8,"olddisplay",b9?cb:bI.css(b8,"display"))}}}for(b6=0;b6<b7;b6++){b8=ca[b6];if(!b8.style){continue}if(!e||b8.style.display==="none"||b8.style.display===""){b8.style.display=e?i[b6]||"":"none"}}return ca}fun
 ction aN(e,b6,b7){var i=bb.exec(b6);return i?Math.max(0,i[1]-(b7||0))+(i[2]||"px"):b6}function ax(b9,b6,e,cb,b8){var b7=e===(cb?"border":"content")?4:b6==="width"?1:0,ca=0;for(;b7<4;b7+=2){if(e==="margin"){ca+=bI.css(b9,e+bT[b7],true,b8)}if(cb){if(e==="content"){ca-=bI.css(b9,"padding"+bT[b7],true,b8)}if(e!=="margin"){ca-=bI.css(b9,"border"+bT[b7]+"Width",true,b8)}}else{ca+=bI.css(b9,"padding"+bT[b7],true,b8);if(e!=="padding"){ca+=bI.css(b9,"border"+bT[b7]+"Width",true,b8)}}}return ca}function v(b8,i,e){var b7=true,b9=i==="width"?b8.offsetWidth:b8.offsetHeight,b6=bq(b8),ca=D.boxSizing&&bI.css(b8,"boxSizing",false,b6)==="border-box";if(b9<=0||b9==null){b9=G(b8,i,b6);if(b9<0||b9==null){b9=b8.style[i]}if(Y.test(b9)){return b9}b7=ca&&(D.boxSizingReliable()||b9===b8.style[i]);b9=parseFloat(b9)||0}return(b9+ax(b8,i,e||(ca?"border":"content"),b7,b6))+"px"}bI.extend({cssHooks:{opacity:{get:function(b6,i){if(i){var e=G(b6,"opacity");return e===""?"1":e}}}},cssNumber:{columnCount:true,fillOpa
 city:true,flexGrow:true,flexShrink:true,fontWeight:true,lineHeight:true,opacity:true,order:true,orphans:true,widows:true,zIndex:true,zoom:true},cssProps:{"float":D.cssFloat?"cssFloat":"styleFloat"},style:function(b7,b6,cd,b8){if(!b7||b7.nodeType===3||b7.nodeType===8||!b7.style){return}var cb,cc,ce,b9=bI.camelCase(b6),i=b7.style;b6=bI.cssProps[b9]||(bI.cssProps[b9]=c(i,b9));ce=bI.cssHooks[b6]||bI.cssHooks[b9];if(cd!==undefined){cc=typeof cd;if(cc==="string"&&(cb=V.exec(cd))){cd=(cb[1]+1)*cb[2]+parseFloat(bI.css(b7,b6));cc="number"}if(cd==null||cd!==cd){return}if(cc==="number"&&!bI.cssNumber[b9]){cd+="px"}if(!D.clearCloneStyle&&cd===""&&b6.indexOf("background")===0){i[b6]="inherit"}if(!ce||!("set" in ce)||(cd=ce.set(b7,cd,b8))!==undefined){try{i[b6]=cd}catch(ca){}}}else{if(ce&&"get" in ce&&(cb=ce.get(b7,false,b8))!==undefined){return cb}return i[b6]}},css:function(ca,b8,i,b9){var b7,cb,e,b6=bI.camelCase(b8);b8=bI.cssProps[b6]||(bI.cssProps[b6]=c(ca.style,b6));e=bI.cssHooks[b8]||bI.css
 Hooks[b6];if(e&&"get" in e){cb=e.get(ca,true,i)}if(cb===undefined){cb=G(ca,b8,b9)}if(cb==="normal"&&b8 in bD){cb=bD[b8]}if(i===""||i){b7=parseFloat(cb);return i===true||bI.isNumeric(b7)?b7||0:cb}return cb}});bI.each(["height","width"],function(b6,e){bI.cssHooks[e]={get:function(b8,b7,i){if(b7){return H.test(bI.css(b8,"display"))&&b8.offsetWidth===0?bI.swap(b8,be,function(){return v(b8,e,i)}):v(b8,e,i)}},set:function(b8,b9,i){var b7=i&&bq(b8);return aN(b8,b9,i?ax(b8,e,i,D.boxSizing&&bI.css(b8,"boxSizing",false,b7)==="border-box",b7):0)}}});if(!D.opacity){bI.cssHooks.opacity={get:function(i,e){return aU.test((e&&i.currentStyle?i.currentStyle.filter:i.style.filter)||"")?(0.01*parseFloat(RegExp.$1))+"":e?"1":""},set:function(b8,b9){var b7=b8.style,i=b8.currentStyle,e=bI.isNumeric(b9)?"alpha(opacity="+b9*100+")":"",b6=i&&i.filter||b7.filter||"";b7.zoom=1;if((b9>=1||b9==="")&&bI.trim(b6.replace(bj,""))===""&&b7.removeAttribute){b7.removeAttribute("filter");if(b9===""||i&&!i.filter){return
 }}b7.filter=bj.test(b6)?b6.replace(bj,e):b6+" "+e}}}bI.cssHooks.marginRight=a7(D.reliableMarginRight,function(i,e){if(e){return bI.swap(i,{display:"inline-block"},G,[i,"marginRight"])}});bI.each({margin:"",padding:"",border:"Width"},function(e,i){bI.cssHooks[e+i]={expand:function(b8){var b7=0,b6={},b9=typeof b8==="string"?b8.split(" "):[b8];for(;b7<4;b7++){b6[e+bT[b7]+i]=b9[b7]||b9[b7-2]||b9[0]}return b6}};if(!aZ.test(e)){bI.cssHooks[e+i].set=aN}});bI.fn.extend({css:function(e,i){return aB(this,function(ca,b7,cb){var b9,b6,cc={},b8=0;if(bI.isArray(b7)){b9=bq(ca);b6=b7.length;for(;b8<b6;b8++){cc[b7[b8]]=bI.css(ca,b7[b8],false,b9)}return cc}return cb!==undefined?bI.style(ca,b7,cb):bI.css(ca,b7)},e,i,arguments.length>1)},show:function(){return s(this,true)},hide:function(){return s(this)},toggle:function(e){if(typeof e==="boolean"){return e?this.show():this.hide()}return this.each(function(){if(S(this)){bI(this).show()}else{bI(this).hide()}})}});function J(b6,i,b8,e,b7){return new J.pr
 ototype.init(b6,i,b8,e,b7)}bI.Tween=J;J.prototype={constructor:J,init:function(b7,i,b9,e,b8,b6){this.elem=b7;this.prop=b9;this.easing=b8||"swing";this.options=i;this.start=this.now=this.cur();this.end=e;this.unit=b6||(bI.cssNumber[b9]?"":"px")},cur:function(){var e=J.propHooks[this.prop];return e&&e.get?e.get(this):J.propHooks._default.get(this)},run:function(b6){var i,e=J.propHooks[this.prop];if(this.options.duration){this.pos=i=bI.easing[this.easing](b6,this.options.duration*b6,0,1,this.options.duration)}else{this.pos=i=b6}this.now=(this.end-this.start)*i+this.start;if(this.options.step){this.options.step.call(this.elem,this.now,this)}if(e&&e.set){e.set(this)}else{J.propHooks._default.set(this)}return this}};J.prototype.init.prototype=J.prototype;J.propHooks={_default:{get:function(i){var e;if(i.elem[i.prop]!=null&&(!i.elem.style||i.elem.style[i.prop]==null)){return i.elem[i.prop]}e=bI.css(i.elem,i.prop,"");return !e||e==="auto"?0:e},set:function(e){if(bI.fx.step[e.prop]){bI.fx.st
 ep[e.prop](e)}else{if(e.elem.style&&(e.elem.style[bI.cssProps[e.prop]]!=null||bI.cssHooks[e.prop])){bI.style(e.elem,e.prop,e.now+e.unit)}else{e.elem[e.prop]=e.now}}}}};J.propHooks.scrollTop=J.propHooks.scrollLeft={set:function(e){if(e.elem.nodeType&&e.elem.parentNode){e.elem[e.prop]=e.now}}};bI.easing={linear:function(e){return e},swing:function(e){return 0.5-Math.cos(e*Math.PI)/2}};bI.fx=J.prototype.init;bI.fx.step={};var N,ae,bR=/^(?:toggle|show|hide)$/,bJ=new RegExp("^(?:([+-])=|)("+aE+")([a-z%]*)$","i"),bP=/queueHooks$/,aG=[h],a2={"*":[function(e,ca){var cc=this.createTween(e,ca),b8=cc.cur(),b7=bJ.exec(ca),cb=b7&&b7[3]||(bI.cssNumber[e]?"":"px"),i=(bI.cssNumber[e]||cb!=="px"&&+b8)&&bJ.exec(bI.css(cc.elem,e)),b6=1,b9=20;if(i&&i[3]!==cb){cb=cb||i[3];b7=b7||[];i=+b8||1;do{b6=b6||".5";i=i/b6;bI.style(cc.elem,e,i+cb)}while(b6!==(b6=cc.cur()/b8)&&b6!==1&&--b9)}if(b7){i=cc.start=+i||+b8||0;cc.unit=cb;cc.end=b7[1]?i+(b7[1]+1)*b7[2]:+b7[2]}return cc}]};function bn(){setTimeout(function()
 {N=undefined});return(N=bI.now())}function bH(b7,b9){var b8,e={height:b7},b6=0;b9=b9?1:0;for(;b6<4;b6+=2-b9){b8=bT[b6];e["margin"+b8]=e["padding"+b8]=b7}if(b9){e.opacity=e.width=b7}return e}function bd(b8,ca,b7){var i,b9=(a2[ca]||[]).concat(a2["*"]),e=0,b6=b9.length;for(;e<b6;e++){if((i=b9[e].call(b7,ca,b8))){return i}}}function h(b7,cc,e){var b6,cf,b9,ci,cj,cg,cb,ce,b8=this,cd={},i=b7.style,ca=b7.nodeType&&S(b7),ch=bI._data(b7,"fxshow");if(!e.queue){cj=bI._queueHooks(b7,"fx");if(cj.unqueued==null){cj.unqueued=0;cg=cj.empty.fire;cj.empty.fire=function(){if(!cj.unqueued){cg()}}}cj.unqueued++;b8.always(function(){b8.always(function(){cj.unqueued--;if(!bI.queue(b7,"fx").length){cj.empty.fire()}})})}if(b7.nodeType===1&&("height" in cc||"width" in cc)){e.overflow=[i.overflow,i.overflowX,i.overflowY];cb=bI.css(b7,"display");ce=cb==="none"?bI._data(b7,"olddisplay")||a0(b7.nodeName):cb;if(ce==="inline"&&bI.css(b7,"float")==="none"){if(!D.inlineBlockNeedsLayout||a0(b7.nodeName)==="inline"){i
 .display="inline-block"}else{i.zoom=1}}}if(e.overflow){i.overflow="hidden";if(!D.shrinkWrapBlocks()){b8.always(function(){i.overflow=e.overflow[0];i.overflowX=e.overflow[1];i.overflowY=e.overflow[2]})}}for(b6 in cc){cf=cc[b6];if(bR.exec(cf)){delete cc[b6];b9=b9||cf==="toggle";if(cf===(ca?"hide":"show")){if(cf==="show"&&ch&&ch[b6]!==undefined){ca=true}else{continue}}cd[b6]=ch&&ch[b6]||bI.style(b7,b6)}else{cb=undefined}}if(!bI.isEmptyObject(cd)){if(ch){if("hidden" in ch){ca=ch.hidden}}else{ch=bI._data(b7,"fxshow",{})}if(b9){ch.hidden=!ca}if(ca){bI(b7).show()}else{b8.done(function(){bI(b7).hide()})}b8.done(function(){var ck;bI._removeData(b7,"fxshow");for(ck in cd){bI.style(b7,ck,cd[ck])}});for(b6 in cd){ci=bd(ca?ch[b6]:0,b6,b8);if(!(b6 in ch)){ch[b6]=ci.start;if(ca){ci.end=ci.start;ci.start=b6==="width"||b6==="height"?1:0}}}}else{if((cb==="none"?a0(b7.nodeName):cb)==="inline"){i.display=cb}}}function ao(b7,b9){var b6,i,ca,b8,e;for(b6 in b7){i=bI.camelCase(b6);ca=b9[i];b8=b7[b6];if(bI.
 isArray(b8)){ca=b8[1];b8=b7[b6]=b8[0]}if(b6!==i){b7[i]=b8;delete b7[b6]}e=bI.cssHooks[i];if(e&&"expand" in e){b8=e.expand(b8);delete b7[i];for(b6 in b8){if(!(b6 in b7)){b7[b6]=b8[b6];b9[b6]=ca}}}else{b9[i]=ca}}}function f(b6,ca,cd){var ce,e,b9=0,i=aG.length,cc=bI.Deferred().always(function(){delete b8.elem}),b8=function(){if(e){return false}var ck=N||bn(),ch=Math.max(0,b7.startTime+b7.duration-ck),cf=ch/b7.duration||0,cj=1-cf,cg=0,ci=b7.tweens.length;for(;cg<ci;cg++){b7.tweens[cg].run(cj)}cc.notifyWith(b6,[b7,cj,ch]);if(cj<1&&ci){return ch}else{cc.resolveWith(b6,[b7]);return false}},b7=cc.promise({elem:b6,props:bI.extend({},ca),opts:bI.extend(true,{specialEasing:{}},cd),originalProperties:ca,originalOptions:cd,startTime:N||bn(),duration:cd.duration,tweens:[],createTween:function(ch,cf){var cg=bI.Tween(b6,b7.opts,ch,cf,b7.opts.specialEasing[ch]||b7.opts.easing);b7.tweens.push(cg);return cg},stop:function(cg){var cf=0,ch=cg?b7.tweens.length:0;if(e){return this}e=true;for(;cf<ch;cf++){
 b7.tweens[cf].run(1)}if(cg){cc.resolveWith(b6,[b7,cg])}else{cc.rejectWith(b6,[b7,cg])}return this}}),cb=b7.props;ao(cb,b7.opts.specialEasing);for(;b9<i;b9++){ce=aG[b9].call(b7,b6,cb,b7.opts);if(ce){return ce}}bI.map(cb,bd,b7);if(bI.isFunction(b7.opts.start)){b7.opts.start.call(b6,b7)}bI.fx.timer(bI.extend(b8,{elem:b6,anim:b7,queue:b7.opts.queue}));return b7.progress(b7.opts.progress).done(b7.opts.done,b7.opts.complete).fail(b7.opts.fail).always(b7.opts.always)}bI.Animation=bI.extend(f,{tweener:function(i,b8){if(bI.isFunction(i)){b8=i;i=["*"]}else{i=i.split(" ")}var b7,e=0,b6=i.length;for(;e<b6;e++){b7=i[e];a2[b7]=a2[b7]||[];a2[b7].unshift(b8)}},prefilter:function(i,e){if(e){aG.unshift(i)}else{aG.push(i)}}});bI.speed=function(b6,b7,i){var e=b6&&typeof b6==="object"?bI.extend({},b6):{complete:i||!i&&b7||bI.isFunction(b6)&&b6,duration:b6,easing:i&&b7||b7&&!bI.isFunction(b7)&&b7};e.duration=bI.fx.off?0:typeof e.duration==="number"?e.duration:e.duration in bI.fx.speeds?bI.fx.speeds[e.dur
 ation]:bI.fx.speeds._default;if(e.queue==null||e.queue===true){e.queue="fx"}e.old=e.complete;e.complete=function(){if(bI.isFunction(e.old)){e.old.call(this)}if(e.queue){bI.dequeue(this,e.queue)}};return e};bI.fn.extend({fadeTo:function(e,b7,b6,i){return this.filter(S).css("opacity",0).show().end().animate({opacity:b7},e,b6,i)},animate:function(ca,b7,b9,b8){var b6=bI.isEmptyObject(ca),e=bI.speed(b7,b9,b8),i=function(){var cb=f(this,bI.extend({},ca),e);if(b6||bI._data(this,"finish")){cb.stop(true)}};i.finish=i;return b6||e.queue===false?this.each(i):this.queue(e.queue,i)},stop:function(b6,i,e){var b7=function(b8){var b9=b8.stop;delete b8.stop;b9(e)};if(typeof b6!=="string"){e=i;i=b6;b6=undefined}if(i&&b6!==false){this.queue(b6||"fx",[])}return this.each(function(){var cb=true,b8=b6!=null&&b6+"queueHooks",ca=bI.timers,b9=bI._data(this);if(b8){if(b9[b8]&&b9[b8].stop){b7(b9[b8])}}else{for(b8 in b9){if(b9[b8]&&b9[b8].stop&&bP.test(b8)){b7(b9[b8])}}}for(b8=ca.length;b8--;){if(ca[b8].elem==
 =this&&(b6==null||ca[b8].queue===b6)){ca[b8].anim.stop(e);cb=false;ca.splice(b8,1)}}if(cb||!e){bI.dequeue(this,b6)}})},finish:function(e){if(e!==false){e=e||"fx"}return this.each(function(){var b7,ca=bI._data(this),b6=ca[e+"queue"],i=ca[e+"queueHooks"],b9=bI.timers,b8=b6?b6.length:0;ca.finish=true;bI.queue(this,e,[]);if(i&&i.stop){i.stop.call(this,true)}for(b7=b9.length;b7--;){if(b9[b7].elem===this&&b9[b7].queue===e){b9[b7].anim.stop(true);b9.splice(b7,1)}}for(b7=0;b7<b8;b7++){if(b6[b7]&&b6[b7].finish){b6[b7].finish.call(this)}}delete ca.finish})}});bI.each(["toggle","show","hide"],function(b6,e){var b7=bI.fn[e];bI.fn[e]=function(i,b9,b8){return i==null||typeof i==="boolean"?b7.apply(this,arguments):this.animate(bH(e,true),i,b9,b8)}});bI.each({slideDown:bH("show"),slideUp:bH("hide"),slideToggle:bH("toggle"),fadeIn:{opacity:"show"},fadeOut:{opacity:"hide"},fadeToggle:{opacity:"toggle"}},function(e,i){bI.fn[e]=function(b6,b8,b7){return this.animate(i,b6,b8,b7)}});bI.timers=[];bI.fx.ti
 ck=function(){var b7,b6=bI.timers,e=0;N=bI.now();for(;e<b6.length;e++){b7=b6[e];if(!b7()&&b6[e]===b7){b6.splice(e--,1)}}if(!b6.length){bI.fx.stop()}N=undefined};bI.fx.timer=function(e){bI.timers.push(e);if(e()){bI.fx.start()}else{bI.timers.pop()}};bI.fx.interval=13;bI.fx.start=function(){if(!ae){ae=setInterval(bI.fx.tick,bI.fx.interval)}};bI.fx.stop=function(){clearInterval(ae);ae=null};bI.fx.speeds={slow:600,fast:200,_default:400};bI.fn.delay=function(i,e){i=bI.fx?bI.fx.speeds[i]||i:i;e=e||"fx";return this.queue(e,function(b7,b6){var b8=setTimeout(b7,i);b6.stop=function(){clearTimeout(b8)}})};(function(){var b6,b8,e,i,b7;b8=n.createElement("div");b8.setAttribute("className","t");b8.innerHTML="  <link/><table></table><a href='/a'>a</a><input type='checkbox'/>";i=b8.getElementsByTagName("a")[0];e=n.createElement("select");b7=e.appendChild(n.createElement("option"));b6=b8.getElementsByTagName("input")[0];i.style.cssText="top:1px";D.getSetAttribute=b8.className!=="t";D.style=/top/.test
 (i.getAttribute("style"));D.hrefNormalized=i.getAttribute("href")==="/a";D.checkOn=!!b6.value;D.optSelected=b7.selected;D.enctype=!!n.createElement("form").enctype;e.disabled=true;D.optDisabled=!b7.disabled;b6=n.createElement("input");b6.setAttribute("value","");D.input=b6.getAttribute("value")==="";b6.value="t";b6.setAttribute("type","radio");D.radioValue=b6.value==="t"})();var al=/\r/g;bI.fn.extend({val:function(b7){var e,i,b8,b6=this[0];if(!arguments.length){if(b6){e=bI.valHooks[b6.type]||bI.valHooks[b6.nodeName.toLowerCase()];if(e&&"get" in e&&(i=e.get(b6,"value"))!==undefined){return i}i=b6.value;return typeof i==="string"?i.replace(al,""):i==null?"":i}return}b8=bI.isFunction(b7);return this.each(function(b9){var ca;if(this.nodeType!==1){return}if(b8){ca=b7.call(this,b9,bI(this).val())}else{ca=b7}if(ca==null){ca=""}else{if(typeof ca==="number"){ca+=""}else{if(bI.isArray(ca)){ca=bI.map(ca,function(cb){return cb==null?"":cb+""})}}}e=bI.valHooks[this.type]||bI.valHooks[this.nodeNa
 me.toLowerCase()];if(!e||!("set" in e)||e.set(this,ca,"value")===undefined){this.value=ca}})}});bI.extend({valHooks:{option:{get:function(e){var i=bI.find.attr(e,"value");return i!=null?i:bI.trim(bI.text(e))}},select:{get:function(e){var cb,b7,cd=e.options,b9=e.selectedIndex,b8=e.type==="select-one"||b9<0,cc=b8?null:[],ca=b8?b9+1:cd.length,b6=b9<0?ca:b8?b9:0;for(;b6<ca;b6++){b7=cd[b6];if((b7.selected||b6===b9)&&(D.optDisabled?!b7.disabled:b7.getAttribute("disabled")===null)&&(!b7.parentNode.disabled||!bI.nodeName(b7.parentNode,"optgroup"))){cb=bI(b7).val();if(b8){return cb}cc.push(cb)}}return cc},set:function(ca,cb){var cc,b9,b7=ca.options,e=bI.makeArray(cb),b8=b7.length;while(b8--){b9=b7[b8];if(bI.inArray(bI.valHooks.option.get(b9),e)>=0){try{b9.selected=cc=true}catch(b6){b9.scrollHeight}}else{b9.selected=false}}if(!cc){ca.selectedIndex=-1}return b7}}}});bI.each(["radio","checkbox"],function(){bI.valHooks[this]={set:function(e,i){if(bI.isArray(i)){return(e.checked=bI.inArray(bI(e).
 val(),i)>=0)}}};if(!D.checkOn){bI.valHooks[this].get=function(e){return e.getAttribute("value")===null?"on":e.value}}});var ba,b3,bO=bI.expr.attrHandle,aq=/^(?:checked|selected)$/i,bN=D.getSetAttribute,bF=D.input;bI.fn.extend({attr:function(e,i){return aB(this,bI.attr,e,i,arguments.length>1)},removeAttr:function(e){return this.each(function(){bI.removeAttr(this,e)})}});bI.extend({attr:function(b8,b7,b9){var e,b6,i=b8.nodeType;if(!b8||i===3||i===8||i===2){return}if(typeof b8.getAttribute===aC){return bI.prop(b8,b7,b9)}if(i!==1||!bI.isXMLDoc(b8)){b7=b7.toLowerCase();e=bI.attrHooks[b7]||(bI.expr.match.bool.test(b7)?b3:ba)}if(b9!==undefined){if(b9===null){bI.removeAttr(b8,b7)}else{if(e&&"set" in e&&(b6=e.set(b8,b9,b7))!==undefined){return b6}else{b8.setAttribute(b7,b9+"");return b9}}}else{if(e&&"get" in e&&(b6=e.get(b8,b7))!==null){return b6}else{b6=bI.find.attr(b8,b7);return b6==null?undefined:b6}}},removeAttr:function(b7,b9){var e,b8,b6=0,ca=b9&&b9.match(aF);if(ca&&b7.nodeType===1){wh
 ile((e=ca[b6++])){b8=bI.propFix[e]||e;if(bI.expr.match.bool.test(e)){if(bF&&bN||!aq.test(e)){b7[b8]=false}else{b7[bI.camelCase("default-"+e)]=b7[b8]=false}}else{bI.attr(b7,e,"")}b7.removeAttribute(bN?e:b8)}}},attrHooks:{type:{set:function(e,i){if(!D.radioValue&&i==="radio"&&bI.nodeName(e,"input")){var b6=e.value;e.setAttribute("type",i);if(b6){e.value=b6}return i}}}}});b3={set:function(i,b6,e){if(b6===false){bI.removeAttr(i,e)}else{if(bF&&bN||!aq.test(e)){i.setAttribute(!bN&&bI.propFix[e]||e,e)}else{i[bI.camelCase("default-"+e)]=i[e]=true}}return e}};bI.each(bI.expr.match.bool.source.match(/\w+/g),function(b7,b6){var e=bO[b6]||bI.find.attr;bO[b6]=bF&&bN||!aq.test(b6)?function(b9,b8,cb){var i,ca;if(!cb){ca=bO[b8];bO[b8]=i;i=e(b9,b8,cb)!=null?b8.toLowerCase():null;bO[b8]=ca}return i}:function(b8,i,b9){if(!b9){return b8[bI.camelCase("default-"+i)]?i.toLowerCase():null}}});if(!bF||!bN){bI.attrHooks.value={set:function(i,b6,e){if(bI.nodeName(i,"input")){i.defaultValue=b6}else{return ba&&
 ba.set(i,b6,e)}}}}if(!bN){ba={set:function(b6,b7,i){var e=b6.getAttributeNode(i);if(!e){b6.setAttributeNode((e=b6.ownerDocument.createAttribute(i)))}e.value=b7+="";if(i==="value"||b7===b6.getAttribute(i)){return b7}}};bO.id=bO.name=bO.coords=function(b6,i,b7){var e;if(!b7){return(e=b6.getAttributeNode(i))&&e.value!==""?e.value:null}};bI.valHooks.button={get:function(b6,i){var e=b6.getAttributeNode(i);if(e&&e.specified){return e.value}},set:ba.set};bI.attrHooks.contenteditable={set:function(i,b6,e){ba.set(i,b6===""?false:b6,e)}};bI.each(["width","height"],function(b6,e){bI.attrHooks[e]={set:function(i,b7){if(b7===""){i.setAttribute(e,"auto");return b7}}}})}if(!D.style){bI.attrHooks.style={get:function(e){return e.style.cssText||undefined},set:function(e,i){return(e.style.cssText=i+"")}}}var aJ=/^(?:input|select|textarea|button|object)$/i,F=/^(?:a|area)$/i;bI.fn.extend({prop:function(e,i){return aB(this,bI.prop,e,i,arguments.length>1)},removeProp:function(e){e=bI.propFix[e]||e;return 
 this.each(function(){try{this[e]=undefined;delete this[e]}catch(i){}})}});bI.extend({propFix:{"for":"htmlFor","class":"className"},prop:function(b9,b7,ca){var b6,e,b8,i=b9.nodeType;if(!b9||i===3||i===8||i===2){return}b8=i!==1||!bI.isXMLDoc(b9);if(b8){b7=bI.propFix[b7]||b7;e=bI.propHooks[b7]}if(ca!==undefined){return e&&"set" in e&&(b6=e.set(b9,ca,b7))!==undefined?b6:(b9[b7]=ca)}else{return e&&"get" in e&&(b6=e.get(b9,b7))!==null?b6:b9[b7]}},propHooks:{tabIndex:{get:function(i){var e=bI.find.attr(i,"tabindex");return e?parseInt(e,10):aJ.test(i.nodeName)||F.test(i.nodeName)&&i.href?0:-1}}}});if(!D.hrefNormalized){bI.each(["href","src"],function(b6,e){bI.propHooks[e]={get:function(i){return i.getAttribute(e,4)}}})}if(!D.optSelected){bI.propHooks.selected={get:function(i){var e=i.parentNode;if(e){e.selectedIndex;if(e.parentNode){e.parentNode.selectedIndex}}return null}}}bI.each(["tabIndex","readOnly","maxLength","cellSpacing","cellPadding","rowSpan","colSpan","useMap","frameBorder","con
 tentEditable"],function(){bI.propFix[this.toLowerCase()]=this});if(!D.enctype){bI.propFix.enctype="encoding"}var bL=/[\t\r\n\f]/g;bI.fn.extend({addClass:function(cd){var b7,b6,ce,cb,b8,e,b9=0,ca=this.length,cc=typeof cd==="string"&&cd;if(bI.isFunction(cd)){return this.each(function(i){bI(this).addClass(cd.call(this,i,this.className))})}if(cc){b7=(cd||"").match(aF)||[];for(;b9<ca;b9++){b6=this[b9];ce=b6.nodeType===1&&(b6.className?(" "+b6.className+" ").replace(bL," "):" ");if(ce){b8=0;while((cb=b7[b8++])){if(ce.indexOf(" "+cb+" ")<0){ce+=cb+" "}}e=bI.trim(ce);if(b6.className!==e){b6.className=e}}}}return this},removeClass:function(cd){var b7,b6,ce,cb,b8,e,b9=0,ca=this.length,cc=arguments.length===0||typeof cd==="string"&&cd;if(bI.isFunction(cd)){return this.each(function(i){bI(this).removeClass(cd.call(this,i,this.className))})}if(cc){b7=(cd||"").match(aF)||[];for(;b9<ca;b9++){b6=this[b9];ce=b6.nodeType===1&&(b6.className?(" "+b6.className+" ").replace(bL," "):"");if(ce){b8=0;while(
 (cb=b7[b8++])){while(ce.indexOf(" "+cb+" ")>=0){ce=ce.replace(" "+cb+" "," ")}}e=cd?bI.trim(ce):"";if(b6.className!==e){b6.className=e}}}}return this},toggleClass:function(b6,e){var i=typeof b6;if(typeof e==="boolean"&&i==="string"){return e?this.addClass(b6):this.removeClass(b6)}if(bI.isFunction(b6)){return this.each(function(b7){bI(this).toggleClass(b6.call(this,b7,this.className,e),e)})}return this.each(function(){if(i==="string"){var b9,b8=0,b7=bI(this),ca=b6.match(aF)||[];while((b9=ca[b8++])){if(b7.hasClass(b9)){b7.removeClass(b9)}else{b7.addClass(b9)}}}else{if(i===aC||i==="boolean"){if(this.className){bI._data(this,"__className__",this.className)}this.className=this.className||b6===false?"":bI._data(this,"__className__")||""}}})},hasClass:function(e){var b8=" "+e+" 

<TRUNCATED>

[24/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/roles/sensor-test-mode/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/sensor-test-mode/index.html b/site/current-book/metron-deployment/roles/sensor-test-mode/index.html
deleted file mode 100644
index bf88d45..0000000
--- a/site/current-book/metron-deployment/roles/sensor-test-mode/index.html
+++ /dev/null
@@ -1,345 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Sensor Test Mode</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../../css/site.css" />
-    <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Sensor Test Mode</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-down"></i>
-        Roles</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/monit/index.html" title="Monit">
-          <i class="none"></i>
-        Monit</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
-          <i class="none"></i>
-        Opentaxii</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
-          <i class="none"></i>
-        Pcap_replay</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
-          <i class="none"></i>
-        Sensor-stubs</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Sensor-test-mode</a>
-          </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Sensor Test Mode</h1>
-<p>A role that configures each of the sensors to produce the maximum amount of telemetry data. This role is useful only for testing. It can be useful to support functional, performance, and load testing of Apache Metron.</p>
-<p>The role does the following to maximize the amount of telemetry data produced by each Metron sensor.</p>
-
-<ul>
-  
-<li>Plays a packet capture file through a network interface to simulate live network traffic.</li>
-  
-<li>Configures <a class="externalLink" href="https://tools.netsa.cert.org/yaf/yaf.html">YAF</a> with <tt>idle-timeout=0</tt>. This causes a flow record to be produced for every network packet received.</li>
-  
-<li>Configures <a class="externalLink" href="https://www.snort.org/">Snort</a> to produce an alert for every network packet received.</li>
-</ul>
-<div class="section">
-<h2><a name="Getting_Started"></a>Getting Started</h2>
-<p>To enable the <tt>sensor-test-mode</tt> role apply the role to the <tt>sensors</tt> host group in your Ansible playbook.</p>
-
-<div class="source">
-<div class="source">
-<pre>- hosts: sensors
-  roles:
-    - role: sensor-test-mode
-</pre></div></div>
-<p>The role has also been added to the default <tt>metron_install.yml</tt> playbook so that it can be turned on/off with a property in both the local Virtualbox and the remote EC2 deployments.</p>
-
-<div class="source">
-<div class="source">
-<pre>sensor_test_mode: True
-</pre></div></div></div>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/vagrant/codelab-platform/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/vagrant/codelab-platform/index.html b/site/current-book/metron-deployment/vagrant/codelab-platform/index.html
deleted file mode 100644
index e4b8eca..0000000
--- a/site/current-book/metron-deployment/vagrant/codelab-platform/index.html
+++ /dev/null
@@ -1,383 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Developer Image for Apache Metron on Virtualbox</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../../css/site.css" />
-    <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Developer Image for Apache Metron on Virtualbox</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                                                                                
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-down"></i>
-        Vagrant</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Codelab-platform</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/fastcapa-test-platform/index.html" title="Fastcapa-test-platform">
-          <i class="none"></i>
-        Fastcapa-test-platform</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/full-dev-platform/index.html" title="Full-dev-platform">
-          <i class="none"></i>
-        Full-dev-platform</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/quick-dev-platform/index.html" title="Quick-dev-platform">
-          <i class="none"></i>
-        Quick-dev-platform</a>
-            </li>
-              </ul>
-        </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                                            
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Developer Image for Apache Metron on Virtualbox</h1>
-<p>This image is a fully functional Metron installation that has been pre-loaded with Ambari, HDP and Metron.</p>
-<div class="section">
-<h2><a name="Getting_Started"></a>Getting Started</h2>
-<div class="section">
-<h3><a name="Prerequisites"></a>Prerequisites</h3>
-<p>As with the Singlenode Full Image, the computer used to deploy Apache Metron will need to have the following components installed.</p>
-
-<ul>
-  
-<li><a class="externalLink" href="https://github.com/ansible/ansible">Ansible</a> (2.0.0.2 or 2.2.2.0)</li>
-  
-<li><a class="externalLink" href="https://www.vagrantup.com">Vagrant</a> 1.8.1</li>
-  
-<li><a class="externalLink" href="https://virtualbox.org">Virtualbox</a> 5.0.16</li>
-  
-<li>Python 2.7.11</li>
-  
-<li>Maven 3.3.9</li>
-</ul></div>
-<div class="section">
-<h3><a name="Ensure_vagrant_hostmanager_is_installed"></a>Ensure vagrant hostmanager is installed</h3>
-<p>To check and make sure you have the plugin installed execute the following:</p>
-
-<div class="source">
-<div class="source">
-<pre> vagrant plugin list
-</pre></div></div>
-<p>If you have it installed you should see the following listed in the output:</p>
-
-<div class="source">
-<div class="source">
-<pre>  vagrant-hostmanager (1.8.1)
-</pre></div></div>
-<p>If it is not installed, you can install it with the following command:</p>
-
-<div class="source">
-<div class="source">
-<pre>  vagrant plugin install vagrant-hostmanager
-</pre></div></div></div>
-<div class="section">
-<h3><a name="Launch_the_Metron_Development_Image"></a>Launch the Metron Development Image</h3>
-<p>Start the image with the following commands:</p>
-
-<div class="source">
-<div class="source">
-<pre>  cd metron-deployment/vagrant/codelab-platform
-  ./run.sh
-</pre></div></div></div>
-<div class="section">
-<h3><a name="Work_with_Metron"></a>Work with Metron</h3>
-<p>As you build out new capabilities for Metron, you will need to re-deploy the Storm topologies. To do so, first HALT the running Storm topologies and then run:</p>
-
-<div class="source">
-<div class="source">
-<pre>./run_enrichment_role.sh
-</pre></div></div>
-<p>Remember Navigate to the following resources to explore your newly minted Apache Metron environment.</p>
-
-<ul>
-  
-<li><a class="externalLink" href="http://node1:5000">Metron</a></li>
-  
-<li><a class="externalLink" href="http://node1:8080">Ambari</a></li>
-  
-<li><a class="externalLink" href="http://node1:2812">Services</a></li>
-</ul>
-<p>Connecting to the host through SSH is as simple as running the following command.</p>
-
-<div class="source">
-<div class="source">
-<pre>   vagrant ssh
-</pre></div></div></div></div>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/vagrant/fastcapa-test-platform/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/vagrant/fastcapa-test-platform/index.html b/site/current-book/metron-deployment/vagrant/fastcapa-test-platform/index.html
deleted file mode 100644
index 2fcb617..0000000
--- a/site/current-book/metron-deployment/vagrant/fastcapa-test-platform/index.html
+++ /dev/null
@@ -1,416 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Fastcapa Test Environment</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../../css/site.css" />
-    <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Fastcapa Test Environment</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-down"></i>
-        Vagrant</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Fastcapa-test-platform</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/full-dev-platform/index.html" title="Full-dev-platform">
-          <i class="none"></i>
-        Full-dev-platform</a>
-            </li>
-              </ul>
-        </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Fastcapa Test Environment</h1>
-<p>Provides a test environment for the development and testing of Fastcapa. The environment is automatically validated after it is created to ensure that Fastcapa is behaving correctly.</p>
-<p>Two virtualized nodes are launched with Vagrant that can communicate with one another over a private network. </p>
-
-<ul>
-  
-<li>The <tt>source</tt> node uses Metron&#x2019;s <tt>pcap_replay</tt> functionality to transmit raw network packet data over a private network.</li>
-  
-<li>The <tt>sink</tt> node is running <tt>fastcapa</tt> and is capturing these network packets.</li>
-  
-<li>Fastcapa then transforms and bundles the packets into a message.</li>
-  
-<li>The message is sent to a Kafka broker running on the <tt>source</tt> node.</li>
-</ul>
-<div class="section">
-<h2><a name="Getting_Started"></a>Getting Started</h2>
-<p>The Fastcapa test environment can be executed on different operating systems. There is a sub-directory for each operating system that Fastcapa can be tested on. </p>
-<p>To run, simply execute <tt>vagrant up</tt> within the appropriate directory. For example, to run the tests on CentOS 7.1 then execute the following commands.</p>
-
-<div class="source">
-<div class="source">
-<pre>cd centos-7.1
-vagrant up
-</pre></div></div>
-<p>Automated tests are executed after provisioning completes to ensure that Fastcapa and the rest of the environment is functioning properly. If you see something like the following, then the tests have passed.</p>
-
-<div class="source">
-<div class="source">
-<pre>$ vagrant up
-==&gt; source: Running provisioner: ansible...
-    source: Running ansible-playbook...
-...
-TASK [debug] *******************************************************************
-ok: [source] =&gt; {
-    &quot;msg&quot;: &quot;Successfully received packets sent from pcap-replay!&quot;
-}
-...
-TASK [debug] *******************************************************************
-ok: [source] =&gt; {
-    &quot;msg&quot;: &quot;Successfully received a Kafka message from fastcapa!&quot;
-}
-</pre></div></div>
-<p>If the deployment process fails mid-course, running <tt>vagrant provision</tt> will continue the process from where it left off. This can sometimes occur when the VM reboots as part of the deployment process. The error might look like the following.</p>
-
-<div class="source">
-<div class="source">
-<pre>TASK [fastcapa : Restart for modified kernel params] ***************************
-fatal: [sink]: UNREACHABLE! =&gt; {&quot;changed&quot;: false, &quot;msg&quot;: &quot;Failed to connect to the host via ssh: Shared connection to 127.0.0.1 closed.\r\n&quot;, &quot;unreachable&quot;: true}
-	to retry, use: --limit @/Users/nallen/Development/metron/metron-deployment/vagrant/fastcapa-test-platform/playbook.retry
-
-PLAY RECAP *********************************************************************
-sink                       : ok=11   changed=9    unreachable=1    failed=0
-source                     : ok=29   changed=25   unreachable=0    failed=0
-
-Ansible failed to complete successfully. Any error output should be
-visible above. Please fix these errors and try again.
-</pre></div></div></div>
-<div class="section">
-<h2><a name="Going_Deeper"></a>Going Deeper</h2>
-<p>This section will outline in more detail the environment and how to interact with it.</p>
-<div class="section">
-<h3><a name="source"></a><tt>source</tt></h3>
-<p>To validate that the <tt>source</tt> node is functioning properly, run the following commands.</p>
-<p>First, ensure that the <tt>pcap-replay</tt> service is running.</p>
-
-<div class="source">
-<div class="source">
-<pre>vagrant ssh source
-sudo service pcap-replay status
-</pre></div></div>
-<p>Use <tt>tcpdump</tt> to ensure that the raw packet data is being sent over the private network. Enter &#x2018;CTRL-C&#x2019; to kill the <tt>tcpdump</tt> process once you are able to see that packets are being sent.</p>
-
-<div class="source">
-<div class="source">
-<pre>sudo yum -y install tcpdump
-sudo tcpdump -i enp0s8
-</pre></div></div></div>
-<div class="section">
-<h3><a name="sink"></a><tt>sink</tt></h3>
-<p>Next validate that the <tt>sink</tt> is functioning properly. Run the following commands starting from the host operating system. </p>
-<p>First, ensure that the <tt>fastcapa</tt> service is running.</p>
-
-<div class="source">
-<div class="source">
-<pre>vagrant ssh sink
-service fastcapa status
-</pre></div></div>
-<p>Ensure that the raw network packet data is being received by Kafka.</p>
-
-<div class="source">
-<div class="source">
-<pre>/usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic pcap
-</pre></div></div>
-<p>Enter &#x2018;CTRL-C&#x2019; to kill the <tt>kafka-console-consumer</tt> process once you are able to see that packets are being sent. These packets will appear to be gibberish in the console. This is the raw binary network packet data after all.</p></div></div>
-<div class="section">
-<h2><a name="FAQ"></a>FAQ</h2>
-<div class="section">
-<h3><a name="Error_Message:_Timed_out_while_waiting_for_the_machine_to_boot"></a>Error Message: Timed out while waiting for the machine to boot</h3>
-
-<div class="source">
-<div class="source">
-<pre>Timed out while waiting for the machine to boot. This means that
-Vagrant was unable to communicate with the guest machine within
-the configured (&quot;config.vm.boot_timeout&quot; value) time period.
-If you look above, you should be able to see the error(s) that
-Vagrant had when attempting to connect to the machine. These errors
-are usually good hints as to what may be wrong.
-If you're using a custom box, make sure that networking is properly
-working and you're able to connect to the machine. It is a common
-problem that networking isn't setup properly in these boxes.
-Verify that authentication configurations are also setup properly,
-as well.
-If the box appears to be booting properly, you may want to increase
-the timeout (&quot;config.vm.boot_timeout&quot;) value.
-&#x279c;  centos-7.4 git:(master) &#x2717; vagrant status
-Current machine states:
-source                    running (virtualbox)
-sink                      not created (virtualbox)
-</pre></div></div>
-<p>If you are unable to launch any of the Fastcapa test environments, which results in a message like the one above, then you may need to upgrade your version of Virtualbox. Success has been reported with versions of VirtualBox 5.1.22+.</p></div></div>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/vagrant/full-dev-platform/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/vagrant/full-dev-platform/index.html b/site/current-book/metron-deployment/vagrant/full-dev-platform/index.html
deleted file mode 100644
index bee92c5..0000000
--- a/site/current-book/metron-deployment/vagrant/full-dev-platform/index.html
+++ /dev/null
@@ -1,420 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Full Development Platform</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../../css/site.css" />
-    <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Full Development Platform</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-down"></i>
-        Vagrant</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/fastcapa-test-platform/index.html" title="Fastcapa-test-platform">
-          <i class="none"></i>
-        Fastcapa-test-platform</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Full-dev-platform</a>
-          </li>
-              </ul>
-        </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Full Development Platform</h1>
-<p>This project fully automates the provisioning and deployment of Apache Metron and all necessary prerequisites on a single, virtualized host running on Virtualbox. </p>
-<p>Metron is composed of many components and installing all of these on a single host, especially a virtualized one, will greatly stress the resources of the host. The host will require at least 8 GB of RAM and a fair amount of patience. It is highly recommended that you shut down all unnecessary services. To that end the vagrant file configuration defaults to disabling solr and yaf.</p>
-<div class="section">
-<h2><a name="Getting_Started"></a>Getting Started</h2>
-<div class="section">
-<h3><a name="Prerequisites"></a>Prerequisites</h3>
-<p>The computer used to deploy Apache Metron will need to have the following components installed.</p>
-
-<ul>
-  
-<li><a class="externalLink" href="https://github.com/ansible/ansible">Ansible</a> (2.0.0.2 or 2.2.2.0)</li>
-  
-<li><a class="externalLink" href="https://www.docker.com/community-edition">Docker</a></li>
-  
-<li><a class="externalLink" href="https://www.vagrantup.com">Vagrant</a> 1.8+</li>
-  
-<li><a class="externalLink" href="https://github.com/devopsgroup-io/vagrant-hostmanager">Vagrant Hostmanager Plugin</a></li>
-  
-<li><a class="externalLink" href="https://virtualbox.org">Virtualbox</a> 5.0+</li>
-  
-<li>Python 2.7</li>
-  
-<li>Maven 3.3.9</li>
-</ul>
-<div class="section">
-<h4><a name="macOS"></a>macOS</h4>
-<p>Any platform that supports these tools is suitable, but the following instructions cover installation on macOS. The easiest means of installing these tools on a Mac is to use the excellent <a class="externalLink" href="http://brew.sh/">Homebrew</a> project.</p>
-
-<ol style="list-style-type: decimal">
-  
-<li>
-<p>Install Homebrew by following the instructions at <a class="externalLink" href="http://brew.sh/">Homebrew</a>.</p></li>
-  
-<li>
-<p>Run the following command in a terminal to install all of the required tools.</p>
-  
-<div class="source">
-<div class="source">
-<pre>brew cask install vagrant virtualbox docker
-brew cask install caskroom/versions/java8
-brew install maven@3.3 git
-pip install ansible==2.2.2.0
-vagrant plugin install vagrant-hostmanager
-open /Applications/Docker.app
-</pre></div></div></li>
-</ol></div></div>
-<div class="section">
-<h3><a name="Deploy_Metron"></a>Deploy Metron</h3>
-
-<ol style="list-style-type: decimal">
-  
-<li>
-<p>Ensure that the Docker service is running.</p></li>
-  
-<li>
-<p>Deploy Metron</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron-deployment/vagrant/full-dev-platform
-vagrant up
-</pre></div></div>
-<p>Should the process fail before completing the deployment, the following command will continue the deployment process without re-instantiating the host.</p>
-  
-<div class="source">
-<div class="source">
-<pre>vagrant provision
-</pre></div></div></li>
-</ol></div>
-<div class="section">
-<h3><a name="Explore_Metron"></a>Explore Metron</h3>
-<p>Navigate to the following resources to explore your newly minted Apache Metron environment.</p>
-
-<ul>
-  
-<li><a class="externalLink" href="http://node1:8080">Ambari</a></li>
-  
-<li><a class="externalLink" href="http://node1:5000">Metron</a></li>
-  
-<li><a class="externalLink" href="http://node1:2812">Services</a></li>
-</ul>
-<p>Connecting to the host through SSH is as simple as running the following command.</p>
-
-<div class="source">
-<div class="source">
-<pre>vagrant ssh
-</pre></div></div></div>
-<div class="section">
-<h3><a name="Working_with_Metron"></a>Working with Metron</h3>
-<p>In addition to re-running the entire provisioning play book, you may now re-run an individual Ansible tag or a collection of tags in the following ways. The following commands will re-run the <tt>web</tt> role on the Vagrant image. This will install components (if necessary) and start the UI.</p>
-
-<div class="source">
-<div class="source">
-<pre>./run_ansible_role.sh web
-</pre></div></div>
-<p>or</p>
-
-<div class="source">
-<div class="source">
-<pre>vagrant --ansible-tags=&quot;web&quot; provision
-</pre></div></div>
-<div class="section">
-<h4><a name="Using_Tags"></a>Using Tags</h4>
-<p>A collection of tags is specified as a comma separated list.</p>
-
-<div class="source">
-<div class="source">
-<pre>./run_ansible_role.sh &quot;sensors,enrichment&quot;
-
-</pre></div></div>
-<p>Tags are listed in the playbooks, some frequently used tags:</p>
-
-<ul>
-  
-<li><tt>hdp-install</tt> - Install HDP</li>
-  
-<li><tt>hdp-deploy</tt> - Deploy and Start HDP Services (will start all Hadoop Services)</li>
-  
-<li><tt>sensors</tt> - Deploy and Start Sensors.</li>
-  
-<li><tt>enrichment</tt> - Deploy and Start Enrichment Topology.</li>
-</ul>
-<p>Note also that there is a convenience script <tt>./run_enrichment_role.sh</tt> which executes Vagrant with the <tt>enrichment</tt> tag.</p></div></div></div>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/vagrant/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/vagrant/index.html b/site/current-book/metron-deployment/vagrant/index.html
deleted file mode 100644
index 744ea5b..0000000
--- a/site/current-book/metron-deployment/vagrant/index.html
+++ /dev/null
@@ -1,305 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Vagrant Deployment</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../css/site.css" />
-    <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Vagrant Deployment</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                              
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Vagrant</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/vagrant/fastcapa-test-platform/index.html" title="Fastcapa-test-platform">
-          <i class="none"></i>
-        Fastcapa-test-platform</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/vagrant/full-dev-platform/index.html" title="Full-dev-platform">
-          <i class="none"></i>
-        Full-dev-platform</a>
-            </li>
-              </ul>
-        </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Vagrant Deployment</h1>
-<p><a name="Vagrant_Deployment"></a></p>
-
-<ul>
-  
-<li>Fast CAPA Test Platform</li>
-  
-<li>Full Dev Platform</li>
-</ul>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>


[07/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/use-cases/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/use-cases/index.html b/site/current-book/use-cases/index.html
index e8c6b28..684cc14 100644
--- a/site/current-book/use-cases/index.html
+++ b/site/current-book/use-cases/index.html
@@ -1,227 +1,114 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/use-cases/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Worked Examples</title>
-    <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../css/site.css" />
     <link rel="stylesheet" href="../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Worked Examples</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Worked Examples</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                              
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Use-cases</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../use-cases/forensic_clustering/index.html" title="Forensic_clustering">
-          <i class="none"></i>
-        Forensic_clustering</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../use-cases/geographic_login_outliers/index.html" title="Geographic_login_outliers">
-          <i class="none"></i>
-        Geographic_login_outliers</a>
-            </li>
-              </ul>
-        </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Use-cases</a>
+    <ul class="nav nav-list">
+    <li><a href="../use-cases/forensic_clustering/index.html" title="Forensic_clustering"><span class="none"></span>Forensic_clustering</a></li>
+    <li><a href="../use-cases/geographic_login_outliers/index.html" title="Geographic_login_outliers"><span class="none"></span>Geographic_login_outliers</a></li>
+    <li><a href="../use-cases/typosquat_detection/index.html" title="Typosquat_detection"><span class="none"></span>Typosquat_detection</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Worked Examples</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Worked Examples</h1>
 <p><a name="Worked_Examples"></a></p>
 <p>The following are worked examples of use-cases that showcase some (or many) component(s) of Metron.</p>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/use-cases/typosquat_detection/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/use-cases/typosquat_detection/index.html b/site/current-book/use-cases/typosquat_detection/index.html
new file mode 100644
index 0000000..03c3581
--- /dev/null
+++ b/site/current-book/use-cases/typosquat_detection/index.html
@@ -0,0 +1,545 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/use-cases/typosquat_detection/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Problem Statement</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Problem Statement</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-down"></span>Use-cases</a>
+    <ul class="nav nav-list">
+    <li><a href="../../use-cases/forensic_clustering/index.html" title="Forensic_clustering"><span class="none"></span>Forensic_clustering</a></li>
+    <li><a href="../../use-cases/geographic_login_outliers/index.html" title="Geographic_login_outliers"><span class="none"></span>Geographic_login_outliers</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Typosquat_detection</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Problem Statement</h1>
+<p><a name="Problem_Statement"></a></p>
+<p><a class="externalLink" href="https://en.wikipedia.org/wiki/Typosquatting">Typosquatting</a> is a form of cybersquatting which relies on likely typos to trick unsuspecting users to visit possibly malicious URLs.  In the best case, this is a mischievous joke as in the following RickRoll: <a class="externalLink" href="http://www.latlmes.com/breaking/apache-metron-named-best-software-by-asf-1">http://www.latlmes.com/breaking/apache-metron-named-best-software-by-asf-1</a>. In the worst case, however, it can be overtly malicious as Bitcoin users found out in <a class="externalLink" href="https://nakedsecurity.sophos.com/2014/03/24/bitcoin-user-loses-10k-to-typosquatters/">2016</a> when thousands of dollars of Bitcoin was stolen as part of a phishing attack which used typosquatting.</p>
+<p>It is therefore of use for us to detect so called typosquatting attacks as they appear over the network.  We have had for some time, through the flatfile loader and open source typosquatting generation tools such as <a class="externalLink" href="https://github.com/elceef/dnstwist">DNS Twist</a>, the ability to generated potential typosquatted domains, import them into HBase and look them up via <tt>ENRICHMENT_EXISTS</tt>.</p>
+<p>There are some challenges with this approach, though entirely viable:</p>
+<ul>
+
+<li>Even for modest numbers of domains, the number of records can grow quite large.  The Top Alexa 10k domains has on the order of 3 million potential typosquatted domains.</li>
+<li>It still requires a network hop if out of cache.</li>
+</ul>
+<p><a name="The_Tools_Metron_Provides"></a></p>
+<h1>The Tools Metron Provides</h1>
+<div class="section">
+<h2><a name="Bloom_Filters"></a>Bloom Filters</h2>
+<p>It would be nice to have a local solution for these types of problems that may tradeoff accuracy for better locality and space.  Those who have been following the general theme of Metron&#x2019;s analytics philosophy will see that we are likely in the domain where a probabalistic sketching data structure is in order.  In this case, we are asking simple existence queries, so a <a class="externalLink" href="https://en.wikipedia.org/wiki/Bloom_filter">Bloom Filter</a> fits well here.</p>
+<p>In Metron, we have the ability to create, add and merge bloom filters via:</p>
+<ul>
+
+<li><tt>BLOOM_INIT( size, fpp)</tt> - Creates a bloom filter to handle <tt>size</tt> number of elements with <tt>fpp</tt> probability of false positives (<tt>0 &lt; fpp &lt; 1</tt>).</li>
+<li><tt>BLOOM_ADD( filter, object)</tt> - Add an item to an existing bloom filter.</li>
+<li><tt>BLOOM_MERGE( filters )</tt> - Merge a <tt>filters</tt>, a list of Bloom Filters.</li>
+</ul></div>
+<div class="section">
+<h2><a name="Typosquatting_Domain_Generation"></a>Typosquatting Domain Generation</h2>
+<p>Now that we have a suitable data structure, we need a way to generate potential typosquatted domains for a given domain.  Following the good work of <a class="externalLink" href="https://github.com/elceef/dnstwist">DNS Twist</a>, we have ported their set of typosquatting strategies to Metron:</p>
+<ul>
+
+<li>Bitsquatting - See <a class="externalLink" href="http://dinaburg.org/bitsquatting.html">here</a></li>
+<li>Homoglyphs - Substituting characters for ascii or unicode analogues which are visually similar (e.g. <tt>latlmes.com</tt> for <tt>latimes.com</tt> as above)</li>
+<li>Subdomain - Making part of the domain a subdomain (e.g. <tt>am.azon.com</tt>)</li>
+<li>Hyphenation</li>
+<li>Insertion</li>
+<li>Addition</li>
+<li>Omission</li>
+<li>Repetition</li>
+<li>Replacement</li>
+<li>Transposition</li>
+<li>Vowel swapping</li>
+</ul>
+<p>The Stellar function in Metron is <tt>DOMAIN_TYPOSQUAT( domain )</tt>.  It is recommended to remove the TLD from the domain.  You can see it in action here with our rick roll example above:</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; 'latlmes' in DOMAIN_TYPOSQUAT( 'latimes')
+true
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Generating_Summaries"></a>Generating Summaries</h2>
+<p>We need a way to generate the summary sketches from flat data for this to work.  This is similar to, but somewhat different from, loading flat data into HBase.  Instead of each row in the file being loaded generating a record in HBase, what we want is for each record to contribute to the summary sketch and at the end to write out the summary sketch.</p>
+<p>For this purpose, we have a new utility <tt>$METRON_HOME/bin/flatfile_summarizer.sh</tt> to accompany <tt>$METRON_HOME/bin/flatfile_loader.sh</tt>.  The same extractor config is used, but we have 3 new configuration options:</p>
+<ul>
+
+<li><tt>state_init</tt> - Allows a state object to be initialized.  This is a string, so a single expression is created.  The output of this expression will be available as the <tt>state</tt> variable.</li>
+<li><tt>state_update</tt> - Allows a state object to be updated.  This is a map, so you can have temporary variables here.  Note that you can reference the <tt>state</tt> variable from this.</li>
+<li><tt>state_merge</tt> - Allows a list of states to be merged. This is a string, so a single expression.  There is a special field called <tt>states</tt> available, which is a list of the states (one per thread).  If this is not in existence, the number of threads is bound to one.</li>
+</ul>
+<p>Just as with <tt>flatfile_loader.sh</tt>, you can specify the number of threads (via <tt>-p</tt>) and batch size (via <tt>-b</tt>), but now you have the opportunity to specify the output destination (via <tt>-o</tt>) and the output mode (via <tt>-om</tt>). The current output modes are:</p>
+<ul>
+
+<li><tt>LOCAL</tt> - The default, to local disk.</li>
+<li><tt>CONSOLE</tt> - Write the object summarized out to std out.  This is useful to get summary statistics about the data imported. For instance, you could determine how many typosquatted domains there are for the Alexa 10k.</li>
+<li><tt>HDFS</tt></li>
+</ul></div>
+<div class="section">
+<h2><a name="Reading_Summaries_In-Stream"></a>Reading Summaries In-Stream</h2>
+<p>These summaries are immutable data and are stored in HDFS.  We want to read them in and cache them for later, so a new stellar function called <tt>OBJECT_GET( hdfs_path )</tt> will allow you to read the data from HDFS and deserialize the data into an object which can be used.  Subsequent calls for the next 24 hours (by default, defaults of the cache can be changed in the global config) will be read from the static cache.</p>
+<p>For instance, if you have used the <tt>flatfile_summarizer.sh</tt> utility described above to write out an object to <tt>/apps/metron/objects/alexa_10k_filter.ser</tt>, you can read and deserialize this object and use the bloom filter to determine if the domain <tt>goggle</tt> is a typosquatted domain:</p>
+
+<div>
+<div>
+<pre class="source">BLOOM_EXISTS( OBJECT_GET('/apps/metron/objects/alexa_10k_filter.ser'), 'goggle')
+</pre></div></div>
+
+<p><a name="Example"></a></p>
+<h1>Example</h1>
+<p>In the following demo, we will:</p>
+<ul>
+
+<li>Generate summary data from the top 10k Alexa domains in a Bloom Filter</li>
+<li>Use this to detect potential typosquatting instances in proxy data</li>
+</ul></div>
+<div class="section">
+<h2><a name="Preliminaries"></a>Preliminaries</h2>
+<p>We assume that the following environment variables are set:</p>
+<ul>
+
+<li><tt>METRON_HOME</tt> - the home directory for metron</li>
+<li><tt>ZOOKEEPER</tt> - The zookeeper quorum (comma separated with port specified: e.g. <tt>node1:2181</tt> for full-dev)</li>
+<li><tt>BROKERLIST</tt> - The Kafka broker list (comma separated with port specified: e.g. <tt>node1:6667</tt> for full-dev)</li>
+<li><tt>ES_HOST</tt> - The elasticsearch master (and port) e.g. <tt>node1:9200</tt> for full-dev.</li>
+</ul>
+<p>Also, this does not assume that you are using a kerberized cluster.  If you are, then the parser start command will adjust slightly to include the security protocol.</p>
+<p>Before editing configurations, be sure to pull the configs from zookeeper locally via</p>
+
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/zk_load_configs.sh --mode PULL -z $ZOOKEEPER -o $METRON_HOME/config/zookeeper/ -f
+</pre></div></div>
+
+<p>If you are doing this on full-dev, I&#x2019;d recommend stopping existing parsers and the profiler to free up resources.  You can do this in Ambari.</p></div>
+<div class="section">
+<h2><a name="Install_Squid_Proxy"></a>Install Squid Proxy</h2>
+<p>Before starting, we&#x2019;re going to need to install and start squid by executing the following commands:</p>
+<ul>
+
+<li><tt>yum install -y squid</tt></li>
+<li><tt>service squid start</tt></li>
+</ul></div>
+<div class="section">
+<h2><a name="Retrieve_Alexa_Data"></a>Retrieve Alexa Data</h2>
+<p>From the Metron access node in <tt>~</tt>, retrieve the <a class="externalLink" href="https://en.wikipedia.org/wiki/Alexa_Internet">Alexa top domains</a> data via:</p>
+
+<div>
+<div>
+<pre class="source">cd ~
+wget http://s3.amazonaws.com/alexa-static/top-1m.csv.zip
+unzip top-1m.csv.zip
+head -n 10000 top-1m.csv &gt; top-10k.csv
+</pre></div></div>
+
+<p>You should now have a file <tt>~/top-10k.csv</tt> which contains the top 10,000 domains as per <a class="externalLink" href="https://en.wikipedia.org/wiki/Alexa_Internet">Alexa</a>.</p></div>
+<div class="section">
+<h2><a name="Summarize"></a>Summarize</h2>
+<div class="section">
+<h3><a name="Configure_the_Bloom_Filter"></a>Configure the Bloom Filter</h3>
+<p>In order to configure the bloom filter, we need to know two things:</p>
+<ol style="list-style-type: decimal">
+
+<li>Roughly how many elements are going into the bloom filter (an upper bound will do)</li>
+<li>What kind of false positive probability do we want?</li>
+</ol>
+<p>Both of these are going to inform how large the bloom filter is going to be.  We can decide 2, but 1 is going to require some computation.  Let&#x2019;s use the <tt>CONSOLE</tt> output mode of the <tt>flatfile_summarizer.sh</tt> to count the number of typosquatted domains across the entire document.</p>
+<p>Create a file <tt>~/extractor_count.json</tt> with the following content:</p>
+
+<div>
+<div>
+<pre class="source">{
+  &quot;config&quot; : {
+    &quot;columns&quot; : {
+       &quot;rank&quot; : 0,
+       &quot;domain&quot; : 1
+    },
+    &quot;value_transform&quot; : {
+       &quot;domain&quot; : &quot;DOMAIN_REMOVE_TLD(domain)&quot;
+    },
+    &quot;value_filter&quot; : &quot;LENGTH(domain) &gt; 0&quot;,
+    &quot;state_init&quot; : &quot;0L&quot;,
+    &quot;state_update&quot; : {
+       &quot;state&quot; : &quot;state + LENGTH( DOMAIN_TYPOSQUAT( domain ))&quot;
+                     },
+    &quot;state_merge&quot; : &quot;REDUCE(states, (s, x) -&gt; s + x, 0)&quot;,
+    &quot;separator&quot; : &quot;,&quot;
+  },
+  &quot;extractor&quot; : &quot;CSV&quot;
+}
+</pre></div></div>
+
+<p>In this extractor config we are using the CSV extractor with the following config properties:</p>
+<ul>
+
+<li><tt>columns</tt> - Indicates the schema of the CSV.  There are 2 columns, <tt>rank</tt> at the first position and <tt>domain</tt> at the second position.</li>
+<li><tt>separator</tt> - Use a comma to separate the columns.</li>
+<li><tt>value_transform</tt> - For each row, transform each <tt>domain</tt> column by removing the TLD.</li>
+<li><tt>value_filter</tt> - Only consider non-empty domains</li>
+<li><tt>state_init</tt> - Initialize the state, a long integer, to 0.</li>
+<li><tt>state_update</tt> - For each row in the CSV, update the state, which is the running partial sum, with the number of typosquatted domains for the domain</li>
+<li><tt>state_merge</tt> - For each thread, we have a partial sum, we want to merge the partial sums into the total.</li>
+</ul>
+<p>We can run this via:</p>
+
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/flatfile_summarizer.sh -i ~/top-10k.csv -e ~/extractor_count.json -p 5 -om CONSOLE
+</pre></div></div>
+
+<p>The output should be something like:</p>
+
+<div>
+<div>
+<pre class="source">17/12/22 17:05:19 WARN extractor.TransformFilterExtractorDecorator: Unable to setup zookeeper client - zk_quorum url not provided. **This will limit some Stellar functionality**
+
+Processing /root/top-10k.csv
+17/12/22 17:05:20 WARN resolver.BaseFunctionResolver: Using System classloader
+Processed 9999 - \
+3496552
+</pre></div></div>
+
+<p>So, we the total number of possible elements in the bloom filter summary, <tt>3,496,552</tt>.</p></div>
+<div class="section">
+<h3><a name="Generate_the_Bloom_Filter"></a>Generate the Bloom Filter</h3>
+<p>Now we can generate the bloom filter on HDFS.  As before, we will adapt our previous extractor config to generate the bloom filter rather than the sum of the typosquatted domains.</p>
+<p>Create a file <tt>~/extractor_filter.json</tt> with the following contents:</p>
+
+<div>
+<div>
+<pre class="source">{
+  &quot;config&quot; : {
+    &quot;columns&quot; : {
+       &quot;rank&quot; : 0,
+       &quot;domain&quot; : 1
+    },
+    &quot;value_transform&quot; : {
+       &quot;domain&quot; : &quot;DOMAIN_REMOVE_TLD(domain)&quot;
+    },
+    &quot;value_filter&quot; : &quot;LENGTH(domain) &gt; 0&quot;,
+    &quot;state_init&quot; : &quot;BLOOM_INIT(3496552, 0.001)&quot;,
+    &quot;state_update&quot; : {
+       &quot;state&quot; : &quot;REDUCE( DOMAIN_TYPOSQUAT( domain ), (s, x) -&gt; BLOOM_ADD(s, x), state)&quot;
+                     },
+    &quot;state_merge&quot; : &quot;BLOOM_MERGE(states)&quot;,
+    &quot;separator&quot; : &quot;,&quot;
+  },
+  &quot;extractor&quot; : &quot;CSV&quot;
+}
+</pre></div></div>
+
+<p>Most of the configs are the same, but there are three that are different:</p>
+<ul>
+
+<li><tt>state_init</tt> - We have changed our state to be a bloom filter, initialized with
+<ul>
+
+<li><tt>3496552</tt> - the size calculated in the previous step</li>
+<li><tt>0.001</tt> - The false positive probability (<tt>0.1%</tt>)</li>
+</ul>
+</li>
+<li><tt>state_update</tt> - Update the bloom filter (the <tt>state</tt> variable) with each typosquatted domain</li>
+<li><tt>state_merge</tt> - Merge the bloom filters generated per thread into a final, single bloom filter to be written.</li>
+</ul>
+<p>Now we can generate the bloom filter in HDFS at <tt>/tmp/reference/alexa10k_filter.ser</tt> via</p>
+
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/flatfile_summarizer.sh -i ~/top-10k.csv -o /tmp/reference/alexa10k_filter.ser -e ~/extractor_filter.json -p 5 -om HDFS
+</pre></div></div>
+
+<p>You can try out the object to ensure it functions as expected via the Stellar REPL (<tt>$METRON_HOME/bin/stellar -z $ZOOKEEPER</tt>):</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'gogle')
+true
+[Stellar]&gt;&gt;&gt; BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'google')
+false
+[Stellar]&gt;&gt;&gt; BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'github')
+false
+[Stellar]&gt;&gt;&gt; BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'gituub')
+true
+</pre></div></div>
+
+<p>Notice the lag on the first call is more substantial than the subsequent calls as they are pulled from the cache.</p></div></div>
+<div class="section">
+<h2><a name="Parser"></a>Parser</h2>
+<p>Start the squid parser via:</p>
+<ul>
+
+<li>Create the squid topic: <tt>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic squid --partitions 1 --replication-factor 1</tt></li>
+<li>Start the squid parser: <tt>$METRON_HOME/bin/start_parser_topology.sh -z $ZOOKEEPER -s squid</tt></li>
+</ul></div>
+<div class="section">
+<h2><a name="Set_up_Enrichment.2C_Threat_Intel_and_Threat_Triage"></a>Set up Enrichment, Threat Intel and Threat Triage</h2>
+<p>Now that we have squid parser running, we should create an enrichment to add a field <tt>is_potential_typosquat</tt> which determines if the domain is potentially a typosquatted domain.  Furthermore, we should set an alert if it&#x2019;s so and triage those messages.</p>
+<p>We can do this by creating <tt>$METRON_HOME/config/zookeeper/enrichments/squid.json</tt> with the following content:</p>
+
+<div>
+<div>
+<pre class="source">{
+  &quot;enrichment&quot;: {
+    &quot;fieldMap&quot;: {
+      &quot;stellar&quot; : {
+        &quot;config&quot; : [
+          &quot;domain_without_tld := DOMAIN_REMOVE_TLD(domain_without_subdomains)&quot;,
+          &quot;is_potential_typosquat := BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), domain_without_tld)&quot;,
+          &quot;domain_without_tld := null&quot;
+        ]
+      }
+   }
+  ,&quot;fieldToTypeMap&quot;: { }
+  },
+  &quot;threatIntel&quot;: {
+    &quot;fieldMap&quot;: {
+      &quot;stellar&quot; : {
+        &quot;config&quot; : [
+          &quot;is_alert := (exists(is_alert) &amp;&amp; is_alert) || is_potential_typosquat&quot;
+        ]
+      }
+
+    },
+    &quot;fieldToTypeMap&quot;: { },
+    &quot;triageConfig&quot; : {
+      &quot;riskLevelRules&quot; : [
+        {
+          &quot;name&quot; : &quot;Alexa 10k Typosquat Bloom&quot;,
+          &quot;comment&quot; : &quot;Inspect a bloom filter with potentially typosquatted domains from the top Alexa 10k&quot;,
+          &quot;rule&quot; : &quot;is_potential_typosquat != null &amp;&amp; is_potential_typosquat&quot;,
+          &quot;score&quot; : 10,
+          &quot;reason&quot; : &quot;FORMAT('%s is a potential typosquatted domain from the top 10k domains from alexa', domain_without_subdomains)&quot;
+        }
+      ],
+      &quot;aggregator&quot; : &quot;MAX&quot;
+    }
+  }
+}
+
+</pre></div></div>
+
+<p>As you can see, following the pattern of enrichments the following are done:</p>
+<ul>
+
+<li>A new field <tt>is_potential_typosquat</tt> is created which indicates whether the domain sans TLD and subdomains is a typosquatted domain according to our bloom filter of the top 10k Alexa domains</li>
+<li><tt>is_alert</tt> is updated based on the <tt>is_potential_typosquat</tt> field</li>
+<li>A new threat triage rule is added to give the analyst sufficient context if this alert triggers and a score of 10.</li>
+</ul>
+<p>Push the configs via <tt>$METRON_HOME/bin/zk_load_configs.sh -m PUSH -i $METRON_HOME/config/zookeeper -z $ZOOKEEPER</tt></p></div>
+<div class="section">
+<h2><a name="Setup_Indices"></a>Setup Indices</h2>
+<p>We have to adjust the mappings for the indices we just created to add the <tt>alert</tt> nested property and ensure each of our properties gets the right type.  One does not want to rely entirely on elasticsearch to guess the right types.  We will do that by specifying a template:</p>
+
+<div>
+<div>
+<pre class="source">curl -XPOST &quot;http://$ES_HOST/_template/squid_index&quot; -d '{
+          &quot;template&quot;: &quot;squid_index*&quot;,
+          &quot;mappings&quot;: {
+             &quot;squid_doc&quot;: {
+               &quot;dynamic_templates&quot;: [
+                 {
+                   &quot;timestamps&quot;: {
+                     &quot;match&quot;: &quot;*:ts&quot;,
+                     &quot;match_mapping_type&quot;: &quot;*&quot;,
+                     &quot;mapping&quot;: {
+                     &quot;type&quot;: &quot;date&quot;,
+                     &quot;format&quot;: &quot;epoch_millis&quot;
+                     }
+                                 }
+                 },
+                 {
+                   &quot;threat_triage_score&quot;: {
+                     &quot;mapping&quot;: {
+                       &quot;type&quot;: &quot;float&quot;
+                     },
+                     &quot;match&quot;: &quot;threat:triage:*score&quot;,
+                     &quot;match_mapping_type&quot;: &quot;*&quot;
+                   }
+                 },
+                 {
+                   &quot;threat_triage_reason&quot;: {
+                     &quot;mapping&quot;: {
+                       &quot;type&quot;: &quot;text&quot;,
+                       &quot;fielddata&quot;: &quot;true&quot;
+                     },
+                     &quot;match&quot;: &quot;threat:triage:rules:*:reason&quot;,
+                     &quot;match_mapping_type&quot;: &quot;*&quot;
+                   }
+                 }
+               ],
+               &quot;properties&quot; : {
+                 &quot;action&quot; : {
+                   &quot;type&quot; : &quot;text&quot;,&quot;fielddata&quot; : true
+                 },
+                 &quot;bytes&quot; : {
+                   &quot;type&quot; : &quot;long&quot;
+                 },
+                 &quot;code&quot; : {
+                   &quot;type&quot; : &quot;long&quot;
+                 },
+                 &quot;domain_without_subdomains&quot; : {
+                   &quot;type&quot; : &quot;text&quot;,&quot;fielddata&quot; : true
+                 },
+                 &quot;elapsed&quot; : {
+                   &quot;type&quot; : &quot;long&quot;
+                 },
+                 &quot;full_hostname&quot; : {
+                   &quot;type&quot; : &quot;text&quot;,&quot;fielddata&quot; : true
+                 },
+                 &quot;guid&quot; : {
+                   &quot;type&quot; : &quot;keyword&quot;
+                 },
+                 &quot;ip_dst_addr&quot; : {
+                   &quot;type&quot; : &quot;ip&quot;
+                 },
+                 &quot;ip_src_addr&quot; : {
+                   &quot;type&quot; : &quot;ip&quot;
+                 },
+                 &quot;is_alert&quot; : {
+                   &quot;type&quot; : &quot;text&quot;,&quot;fielddata&quot; : true
+                 },
+                 &quot;is_potential_typosquat&quot; : {
+                   &quot;type&quot; : &quot;boolean&quot;
+                 },
+                 &quot;method&quot; : {
+                   &quot;type&quot; : &quot;text&quot;,&quot;fielddata&quot; : true
+                 },
+                 &quot;original_text&quot; : {
+                   &quot;type&quot; : &quot;text&quot;,&quot;fielddata&quot; : true
+                 },
+                 &quot;source:type&quot; : {
+                   &quot;type&quot; : &quot;keyword&quot;
+                 },
+                 &quot;timestamp&quot; : {
+                   &quot;type&quot; : &quot;date&quot;,
+                   &quot;format&quot;: &quot;epoch_millis&quot;
+                 },
+                 &quot;url&quot; : {
+                   &quot;type&quot; : &quot;text&quot;,&quot;fielddata&quot; : true
+                 },
+                 &quot;alert&quot; : {
+                   &quot;type&quot; : &quot;nested&quot;
+                 }
+              }
+           }
+   }
+}'
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Generate_Sample_Data"></a>Generate Sample Data</h2>
+<p>We can now use <tt>squidclient</tt> to visit a regular domain and typosquatted domain and send the data to kafka:</p>
+
+<div>
+<div>
+<pre class="source">squidclient http://www.github.com
+squidclient http://gituub.com/apache/metron
+cat /var/log/squid/access.log | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list $BROKERLIST --topic squid
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Investigate_via_the_Alerts_UI"></a>Investigate via the Alerts UI</h2>
+<p>We should now have data in our elasticsearch indices, so let&#x2019;s investigate via the alerts UI.<br />
+We can visit the Alerts UI (find the link from Ambari if not on full-dev.  If on full-dev, go <a class="externalLink" href="http://node1:4201/alerts-list">here</a>.</p>
+<p>From there you should see the following data from squid with one as an alert and the other not</p>
+<p><img src="squid_search.png" alt="Alerts" /></p>
+<p>Now, if you drill down into the alert, you can see our fields and the reasons specified</p>
+<p><img src="drill_down.png" alt="Drill Down" /></p></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/documentation/index.md
----------------------------------------------------------------------
diff --git a/site/documentation/index.md b/site/documentation/index.md
index e44953c..585f656 100644
--- a/site/documentation/index.md
+++ b/site/documentation/index.md
@@ -74,24 +74,24 @@ title: Apache Metron Documentation
 <section class="events-section info-block darken" id="releases">
     <div class="wrapper downloads">
         <div class="text-center">
-            <h4>Current Release: 0.4.2</h4>
+            <h4>Current Release: 0.5.0</h4>
         </div>
         <div class="content-960 hover-btn text-center">
             <ul>
                 <li>
-                <a href="http://www.apache.org/dyn/closer.cgi/metron/0.4.2/apache-metron-0.4.2.tar.gz" target="new">
-                    apache-metron-0.4.2.tar.gz
+                <a href="http://www.apache.org/dyn/closer.cgi/metron/0.5.0/apache-metron-0.5.0.tar.gz" target="new">
+                    apache-metron-0.5.0.tar.gz
                 </a>
-                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.4.2/apache-metron-0.4.2.tar.gz.asc" target="new">
+                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.asc" target="new">
                     PGP
                  </a>]
-                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.4.2/apache-metron-0.4.2.tar.gz.sha1" target="new">
+                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha1" target="new">
                     SHA
                  </a>]
-                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.4.2/apache-metron-0.4.2.tar.gz.sha512" target="new">
+                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha512" target="new">
                     SHA512
                  </a>]
-                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.4.2/apache-metron-0.4.2.tar.gz.md5" target="new">
+                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.md5" target="new">
                     MD5
                   </a>]
                  [ <a href="/current-book/index.html" target="new"> Book </a>]


[45/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/ae1d3eb9
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/ae1d3eb9
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/ae1d3eb9

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: ae1d3eb9adfa678553c59c8abdc3fe90aaf1a5ae
Parents: 1b9828e
Author: justinleet <ju...@gmail.com>
Authored: Fri Jun 8 08:18:15 2018 -0400
Committer: leet <le...@apache.org>
Committed: Fri Jun 8 08:18:15 2018 -0400

----------------------------------------------------------------------
 site/current-book/CONTRIBUTING.html             |   157 +
 site/current-book/Upgrading.html                |   354 +-
 .../css/apache-maven-fluido-1.3.0.min.css       |     9 -
 .../css/apache-maven-fluido-1.7.min.css         |    17 +
 site/current-book/css/maven-base.css            |   168 +
 site/current-book/css/maven-theme.css           |   161 +
 site/current-book/css/print.css                 |    11 +-
 .../fonts/glyphicons-halflings-regular.eot      |   Bin 0 -> 35283 bytes
 .../fonts/glyphicons-halflings-regular.svg      |   229 +
 .../fonts/glyphicons-halflings-regular.ttf      |   Bin 0 -> 55016 bytes
 .../fonts/glyphicons-halflings-regular.woff     |   Bin 0 -> 41793 bytes
 .../images/apache-maven-project-2.png           |   Bin 33442 -> 43073 bytes
 site/current-book/images/collapsed.gif          |   Bin 0 -> 53 bytes
 site/current-book/images/expanded.gif           |   Bin 0 -> 52 bytes
 site/current-book/images/external.png           |   Bin 0 -> 230 bytes
 site/current-book/images/icon_error_sml.gif     |   Bin 633 -> 1010 bytes
 site/current-book/images/icon_info_sml.gif      |   Bin 638 -> 606 bytes
 site/current-book/images/icon_success_sml.gif   |   Bin 604 -> 990 bytes
 site/current-book/images/icon_warning_sml.gif   |   Bin 625 -> 576 bytes
 site/current-book/images/newwindow.png          |   Bin 0 -> 220 bytes
 site/current-book/index.html                    |   347 +-
 .../js/apache-maven-fluido-1.3.0.min.js         |    21 -
 .../js/apache-maven-fluido-1.7.min.js           |    25 +
 site/current-book/metron-analytics/index.html   |   303 +-
 .../metron-maas-service/index.html              |   451 +-
 .../metron-profiler-client/index.html           |   888 +-
 .../metron-analytics/metron-profiler/index.html |  1338 +-
 .../metron-statistics/HLLP.html                 | 10962 ++++++-----------
 .../metron-statistics/index.html                |   870 +-
 .../metron-contrib/metron-docker/index.html     |   540 +-
 .../metron-performance/index.html               |   317 +
 .../Kerberos-ambari-setup.html                  |   372 +-
 .../Kerberos-manual-setup.html                  |  1218 +-
 .../metron-deployment/amazon-ec2/index.html     |   755 +-
 .../metron-deployment/ansible/index.html        |   128 +
 .../metron-deployment/ansible/roles/index.html  |   140 +
 .../ansible/roles/opentaxii/index.html          |   301 +
 .../ansible/roles/pcap_replay/index.html        |   172 +
 .../ansible/roles/sensor-stubs/index.html       |   221 +
 .../ansible/roles/sensor-test-mode/index.html   |   156 +
 .../development/centos6/index.html              |   233 +
 .../development/fastcapa/index.html             |   249 +
 .../metron-deployment/development/index.html    |   137 +
 .../development/ubuntu14/index.html             |   233 +
 site/current-book/metron-deployment/index.html  |   691 +-
 .../metron-deployment/other-examples/index.html |   363 +-
 .../manual-install/Manual_Install_CentOS6.html  |  1352 +-
 .../ambari/elasticsearch-mpack/index.html       |   190 +
 .../packaging/ambari/index.html                 |  1150 +-
 .../packaging/ambari/metron-mpack/index.html    |   337 +
 .../packaging/docker/ansible-docker/index.html  |   432 +-
 .../packaging/docker/deb-docker/index.html      |   194 +
 .../packaging/docker/rpm-docker/index.html      |   430 +-
 .../packaging/packer-build/index.html           |   382 +-
 .../metron-deployment/roles/index.html          |   332 -
 .../metron-deployment/roles/monit/index.html    |   388 -
 .../roles/opentaxii/index.html                  |   480 -
 .../roles/pcap_replay/index.html                |   355 -
 .../roles/sensor-stubs/index.html               |   401 -
 .../roles/sensor-test-mode/index.html           |   345 -
 .../vagrant/codelab-platform/index.html         |   383 -
 .../vagrant/fastcapa-test-platform/index.html   |   416 -
 .../vagrant/full-dev-platform/index.html        |   420 -
 .../metron-deployment/vagrant/index.html        |   305 -
 .../vagrant/quick-dev-platform/index.html       |   406 -
 .../metron-interface/metron-alerts/index.html   |   434 +-
 .../metron-interface/metron-config/index.html   |   415 +-
 .../metron-interface/metron-rest/index.html     |  2661 ++--
 .../Performance-tuning-guide.html               |  1123 +-
 site/current-book/metron-platform/index.html    |   346 +-
 .../metron-platform/metron-api/index.html       |   387 +-
 .../metron-platform/metron-common/index.html    |   877 +-
 .../metron-data-management/index.html           |  1348 +-
 .../metron-elasticsearch/index.html             |   706 +-
 .../metron-enrichment/Performance.html          |   802 ++
 .../metron-enrichment/index.html                |   760 +-
 .../metron-platform/metron-indexing/index.html  |   547 +-
 .../metron-management/index.html                |  1178 +-
 .../metron-parsers/3rdPartyParser.html          |   467 +
 .../metron-platform/metron-parsers/index.html   |   870 +-
 .../metron-parsers/parser-testing.html          |   434 +-
 .../metron-pcap-backend/index.html              |   503 +-
 .../metron-platform/metron-writer/index.html    |   377 +-
 .../metron-sensors/bro-plugin-kafka/index.html  |   519 -
 .../metron-sensors/fastcapa/index.html          |  1239 +-
 site/current-book/metron-sensors/index.html     |   299 +-
 .../metron-sensors/pycapa/index.html            |   605 +-
 .../stellar-3rd-party-example/index.html        |   263 +-
 .../stellar-common/3rdPartyStellar.html         |   330 +-
 .../metron-stellar/stellar-common/index.html    |  3603 ++----
 .../metron-stellar/stellar-zeppelin/index.html  |   239 +
 .../use-cases/forensic_clustering/index.html    |   517 +-
 .../geographic_login_outliers/index.html        |   442 +-
 site/current-book/use-cases/index.html          |   283 +-
 .../use-cases/typosquat_detection/index.html    |   545 +
 site/documentation/index.md                     |    14 +-
 96 files changed, 22303 insertions(+), 32065 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/CONTRIBUTING.html
----------------------------------------------------------------------
diff --git a/site/current-book/CONTRIBUTING.html b/site/current-book/CONTRIBUTING.html
new file mode 100644
index 0000000..bb3119e
--- /dev/null
+++ b/site/current-book/CONTRIBUTING.html
@@ -0,0 +1,157 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/CONTRIBUTING.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; How To Contribute</title>
+    <link rel="stylesheet" href="./css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="./css/site.css" />
+    <link rel="stylesheet" href="./css/print.css" media="print" />
+    <script type="text/javascript" src="./js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">How To Contribute</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>How To Contribute</h1>
+<p><a name="How_To_Contribute"></a></p>
+<p>As an open source project, Metron welcomes contributions of all forms. There are several great ways to contribute!</p>
+<ul>
+
+<li><a href="#Contributing_a_Code_Change">Contributing a Code Change</a></li>
+<li>Reviewing pull requests on our GitHub page. Check out current open <a class="externalLink" href="https://github.com/apache/metron/pulls">Pull Requests</a></li>
+<li>Improve our documentation. Our docs are self contained in the project in README files. Doc changes are the same process as a code change. See <a href="#Contributing_a_Code_Change">Contributing a Code Change</a></li>
+<li>Contributing to or starting discussions on the mailing lists. Both the user and dev lists are great places to give and receive help, or provide feedback. See <a class="externalLink" href="http://metron.apache.org/community/#mailinglist">Mailing Lists</a></li>
+<li>Filing tickets for features, enhancements, and bugs to our JIRA. Take a look at <a class="externalLink" href="https://cwiki.apache.org/confluence/display/METRON/Reporting+Issues">Reporting Issues</a> and the <a class="externalLink" href="https://issues.apache.org/jira/projects/METRON">Metron JIRA</a></li>
+</ul>
+<div class="section">
+<h2><a name="Contributing_A_Code_Change"></a>Contributing A Code Change</h2>
+<ol style="list-style-type: decimal">
+
+<li>Open a <a class="externalLink" href="https://issues.apache.org/jira/projects/METRON">JIRA ticket</a> associated with your change, if one doesn&#x2019;t already exist. Assign it to yourself and give a good description.
+<ul>
+
+<li>Feel free to ask questions on the lists and collaborate!</li>
+</ul>
+</li>
+<li>Implement your change
+<ul>
+
+<li>We recommend setting up <a class="externalLink" href="https://docs.travis-ci.com">Travis CI</a> on your personal Github repo to handle long running testing. If the Travis build fails, you&#x2019;ll want to look into it. See <a class="externalLink" href="https://docs.travis-ci.com/user/getting-started/#To-get-started-with-Travis-CI">Getting started</a> for instructions.</li>
+</ul>
+</li>
+<li>Open a GitHub <a class="externalLink" href="https://github.com/apache/metron/pulls">Pull Request</a> with your change
+<ul>
+
+<li>Fork the Metron repo. Look at <a class="externalLink" href="https://help.github.com/articles/fork-a-repo/">Fork a repo</a></li>
+<li>Make the PR. See <a class="externalLink" href="https://help.github.com/articles/creating-a-pull-request-from-a-fork/">Creating a pull request from a fork</a></li>
+<li>Make sure the PR name starts with your JIRA ticket number (METRON-XXXX).</li>
+</ul>
+</li>
+<li>Iterate on your change with reviewers until it&#x2019;s merged into master.</li>
+</ol></div>
+<div class="section">
+<h2><a name="Development_Guidelines"></a>Development Guidelines</h2>
+<p>The full guidelines can be found on the <a class="externalLink" href="https://cwiki.apache.org/confluence/display/METRON/Development+Guidelines">Metron wiki</a>.  They boil down to</p>
+<ol style="list-style-type: decimal">
+
+<li>Make sure you&#x2019;ve tested your change.</li>
+<li>Make sure you&#x2019;ve documented your change.</li>
+<li>Follow the <a class="externalLink" href="https://google.github.io/styleguide/javaguide.html">Google Java Style Guide</a>
+<ul>
+
+<li>If the file is a different style follow that style.</li>
+</ul>
+</li>
+<li>Be open to feedback.</li>
+</ol></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/Upgrading.html
----------------------------------------------------------------------
diff --git a/site/current-book/Upgrading.html b/site/current-book/Upgrading.html
index 78fe2ac..fac4db1 100644
--- a/site/current-book/Upgrading.html
+++ b/site/current-book/Upgrading.html
@@ -1,199 +1,109 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/Upgrading.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Upgrading</title>
-    <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="./css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="./css/site.css" />
     <link rel="stylesheet" href="./css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="./js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Upgrading</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Upgrading</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Upgrading</a>
-          </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Upgrading</a></li>
+    <li><a href="metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Upgrading</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Upgrading</h1>
 <p><a name="Upgrading"></a></p>
 <p>This document constitutes a per-version listing of changes of configuration which are non-backwards compatible.</p>
 <div class="section">
+<h2><a name="a0.4.2_to_0.5.0"></a>0.4.2 to 0.5.0</h2>
+<div class="section">
+<h3><a name="METRON-941:_native_PaloAlto_parser_corrupts_message_when_having_a_comma_in_the_payload"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-941">METRON-941: native PaloAlto parser corrupts message when having a comma in the payload</a></h3>
+<p>While modifying the PaloAlto log parser to support logs from newer PAN-OS version and to not break when a message payload contains a comma, some field names were changed to extend the coverage, fix some duplicate names and change some field names to the Metron standard message format.</p>
+<p>Installations making use of this parser should check, if the resulting messages still meet their expectations and adjust downstream configurations (i.e. ElasticSearch template) accordingly.</p>
+<p><i>Note:</i> Previously, the samples for the test contained a full syslog line (including syslog header). This did - and will continue to - create a broken &#x201c;domain&#x201d; field in the parsed message. It is recommended to only feed the syslog message part to the parser for now.</p></div></div>
+<div class="section">
 <h2><a name="a0.4.1_to_0.4.2"></a>0.4.1 to 0.4.2</h2>
 <div class="section">
 <h3><a name="METRON-1277:_STELLAR_Add_Match_functionality_to_language"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-1277">METRON-1277: STELLAR Add Match functionality to language</a></h3>
-<p>As we continue to evolve the Stellar language, it is possible that new keywords will be added to the language. This may cause compatablity issues where these reserved words and symbols are used in existing scripts.</p>
+<p>As we continue to evolve the Stellar language, it is possible that new keywords will be added to the language.  This may cause compatablity issues where these reserved words and symbols are used in existing scripts.</p>
 <p>Adding <tt>match</tt> to the Stellar lanaguage has introduced the following new reserved keywords and symbols:</p>
 <p><tt>match</tt>, <tt>default</tt>, <tt>{</tt>, <tt>}</tt>, &#x2018;=&gt;&#x2019;</p>
 <p>Any stellar expressions which use these keywords not in quotes will need to be modified.</p></div>
@@ -203,21 +113,22 @@
 <p>Two steps must be done for each sensor, but not on each index for each sensor.</p>
 <p>First is to update the Elasticsearch template for each sensor, so any new indices have the field:</p>
 
-<div class="source">
-<div class="source">
-<pre>export ELASTICSEARCH=&quot;node1&quot;
+<div>
+<div>
+<pre class="source">export ELASTICSEARCH=&quot;node1&quot;
 export SENSOR=&quot;bro&quot;
 curl -XGET &quot;http://${ELASTICSEARCH}:9200/_template/${SENSOR}_index*?pretty=true&quot; -o &quot;${SENSOR}.template&quot;
-sed -i '' '2d;$d' ./${SENSOR}.template
-sed -i '' '/&quot;properties&quot; : {/ a\
+sed -i '2d;$d' ./${SENSOR}.template
+sed -i '/&quot;properties&quot; : {/ a\
 &quot;alert&quot;: { &quot;type&quot;: &quot;nested&quot;},' ${SENSOR}.template
 curl -XPUT &quot;http://${ELASTICSEARCH}:9200/_template/${SENSOR}_index&quot; -d @${SENSOR}.template
 </pre></div></div>
-<p>To update existing indexes, update Elasticsearch mappings with the new field for each sensor. Make sure to set the ELASTICSEARCH variable appropriately.</p>
 
-<div class="source">
-<div class="source">
-<pre>curl -XPUT &quot;http://${ELASTICSEARCH}:9200/${SENSOR}_index*/_mapping/${SENSOR}_doc&quot; -d '
+<p>To update existing indexes, update Elasticsearch mappings with the new field for each sensor.  Make sure to set the ELASTICSEARCH variable appropriately.</p>
+
+<div>
+<div>
+<pre class="source">curl -XPUT &quot;http://${ELASTICSEARCH}:9200/${SENSOR}_index*/_mapping/${SENSOR}_doc&quot; -d '
 {
         &quot;properties&quot; : {
           &quot;alert&quot; : {
@@ -228,10 +139,11 @@ curl -XPUT &quot;http://${ELASTICSEARCH}:9200/_template/${SENSOR}_index&quot; -d
 '
 rm ${SENSOR}.template
 </pre></div></div>
+
 <p>For a more detailed description, please see metron-platform/metron-elasticsearch/README.md</p></div>
 <div class="section">
 <h3><a name="Description"></a>Description</h3>
-<p>In the 0.4.2 release, </p></div></div>
+<p>In the 0.4.2 release,</p></div></div>
 <div class="section">
 <h2><a name="a0.3.1_to_0.4.0"></a>0.3.1 to 0.4.0</h2>
 <div class="section">
@@ -242,15 +154,13 @@ rm ${SENSOR}.template
 <div class="section">
 <h2><a name="a0.3.0_to_0.3.1"></a>0.3.0 to 0.3.1</h2>
 <div class="section">
-<h3><a name="METRON-664:_Make_the_index_configuration_per-writer_with_enableddisabled"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-664">METRON-664: Make the index configuration per-writer with enabled/disabled</a></h3>
+<h3><a name="METRON-664:_Make_the_index_configuration_per-writer_with_enabled.2Fdisabled"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-664">METRON-664: Make the index configuration per-writer with enabled/disabled</a></h3>
 <div class="section">
 <h4><a name="Description"></a>Description</h4>
 <p>As of 0.3.0 the indexing configuration</p>
-
 <ul>
-  
+
 <li>Is held in the enrichment configuration for a sensor</li>
-  
 <li>Has properties which control every writers (i.e. HDFS, solr or elasticsearch).</li>
 </ul>
 <p>In the 0.3.1 release, this configuration has been broken out and control for individual writers are separated.</p>
@@ -260,18 +170,19 @@ rm ${SENSOR}.template
 <p>Migrate the configurations from each sensor enrichment configuration and create appropriate configurations for indexing.</p>
 <p>For instance, if a sensor enrichment config for sensor <tt>foo</tt> is in <tt>$METRON_HOME/config/zookeeper/enrichments/foo.json</tt> and looks like</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;index&quot; : &quot;foo&quot;,
   &quot;batchSize&quot; : 100
 }
 </pre></div></div>
+
 <p>You would create a file to configure each writer for sensor <tt>foo</tt> called <tt>$METRON_HOME/config/zookeeper/indexing/foo.json</tt> with the contents</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;elasticsearch&quot; : {
     &quot;index&quot; : &quot;foo&quot;,
     &quot;batchSize&quot; : 100,
@@ -283,25 +194,27 @@ rm ${SENSOR}.template
     &quot;enabled&quot; : true
   }
 }
-</pre></div></div></div></div>
+</pre></div></div>
+</div></div>
 <div class="section">
 <h3><a name="METRON-675:_Make_Threat_Triage_rules_able_to_be_assigned_names_and_comments"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-675">METRON-675: Make Threat Triage rules able to be assigned names and comments</a></h3>
 <div class="section">
 <h4><a name="Description"></a>Description</h4>
 <p>As of 0.3.0, threat triage rules were defined as a simple Map associating a Stellar expression with a score. As of 0.3.1, due to the fact that there may be many threat triage rules, we have made the rules more complex. To help organize these, we have made the threat triage objects in their own right that contain optional name and optional comment fields.</p>
-<p>This essentially makes the risk level rules slightly more complex. The format goes from:</p>
+<p>This essentially makes the risk level rules slightly more complex.  The format goes from:</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;riskLevelRules&quot; : {
+<div>
+<div>
+<pre class="source">&quot;riskLevelRules&quot; : {
     &quot;stellar expression&quot; : numeric score
 }
 </pre></div></div>
+
 <p>to:</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;riskLevelRules&quot; : [
+<div>
+<div>
+<pre class="source">&quot;riskLevelRules&quot; : [
      {
         &quot;name&quot; : &quot;optional name&quot;,
         &quot;comment&quot; : &quot;optional comment&quot;,
@@ -309,7 +222,8 @@ rm ${SENSOR}.template
         &quot;score&quot; : numeric score
      }
 ]
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Migration"></a>Migration</h4>
 <p>For every sensor enrichment configuration, you will need to migrate the <tt>riskLevelRules</tt> section to move from a map to a list of risk level rule objects.</p></div></div>
@@ -324,23 +238,25 @@ rm ${SENSOR}.template
 <p>While new installs will not require any additional steps, in an existing install a script must be run to retrieve and load the initial data.</p>
 <p>The shell script <tt>geo_enrichment_load.sh</tt> will retrieve MaxMind GeoLite2 data and load data into HDFS, and update the configuration to point to this data. In most cases the following usage will grab the data appropriately:</p>
 
-<div class="source">
-<div class="source">
-<pre>$METRON_HOME/bin/geo_enrichment_load.sh -z &lt;zk_server&gt;:&lt;zk_port&gt;
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/geo_enrichment_load.sh -z &lt;zk_server&gt;:&lt;zk_port&gt;
 </pre></div></div>
-<p>Additional options, including changing the source file location (which can be a file:// location if the GeoIP data is already downloaded), are available with the -h flag and are also detailed in the metron-data-management README.me  file.</p>
+
+<p>Additional options, including changing the source file location (which can be a <a class="externalLink" href="file://">file://</a> location if the GeoIP data is already downloaded), are available with the -h flag and are also detailed in the metron-data-management README.me file.</p>
 <p>One caveat is that this script will NOT update on disk config files. It is recommended to retrieve the configuration using</p>
 
-<div class="source">
-<div class="source">
-<pre>$METRON_HOME/bin/zk_load_configs.sh -z &lt;zk_server&gt;:&lt;zk_port&gt; -m DUMP
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/zk_load_configs.sh -z &lt;zk_server&gt;:&lt;zk_port&gt; -m DUMP
 </pre></div></div>
+
 <p>The new config will be <tt>geo.hdfs.file</tt> in the global section of the configuration. Append this key-value into the global.json in the config directory. A PUSH is unnecessary</p></div></div>
 <div class="section">
 <h3><a name="METRON-684:_Decouple_Timestamp_calculation_from_PROFILE_GET"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-684">METRON-684: Decouple Timestamp calculation from PROFILE_GET</a></h3>
 <div class="section">
 <h4><a name="Description"></a>Description</h4>
-<p>During 0.3.1 we decoupled specifying durations for calls to the profiler into a separate function. The consequence is that existing calls to <tt>PROFILE_GET</tt> will need to migrate.</p></div>
+<p>During 0.3.1 we decoupled specifying durations for calls to the profiler into a separate function.  The consequence is that existing calls to <tt>PROFILE_GET</tt> will need to migrate.</p></div>
 <div class="section">
 <h4><a name="Migration"></a>Migration</h4>
 <p>Existing calls to <tt>PROFILE_GET</tt> will need to change from <tt>PROFILE_GET('profile', 'entity', duration, 'durationUnits')</tt> to <tt>PROFILE_GET('profile', 'entity', PROFILE_FIXED(duration, 'durationUnits'))</tt></p></div></div></div>
@@ -350,9 +266,7 @@ rm ${SENSOR}.template
 <h3><a name="METRON-447:_Monit_fails_to_reload_when_upgrading_from_0.2.0BETA_to_master"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-447">METRON-447: Monit fails to reload when upgrading from 0.2.0BETA to master</a></h3>
 <div class="section">
 <h4><a name="Description"></a>Description</h4>
-<p><tt>/etc/monit.d/enrichment-elasticsearch.monit</tt> was renamed to <tt>/etc/monit.d/indexing-elasticsearch.monit</tt>, however the old file isn&#x2019;t removed via ansible, which causes the below error during an upgrade: <tt>Starting monit: /etc/monit.d/enrichment-elasticsearch.monit:18: Service
-name conflict, enrichment already defined
-'/usr/local/monit/status_enrichment_topology.sh'</tt></p></div></div>
+<p><tt>/etc/monit.d/enrichment-elasticsearch.monit</tt> was renamed to <tt>/etc/monit.d/indexing-elasticsearch.monit</tt>, however the old file isn&#x2019;t removed via ansible, which causes the below error during an upgrade: <tt>Starting monit: /etc/monit.d/enrichment-elasticsearch.monit:18: Service name conflict, enrichment already defined '/usr/local/monit/status_enrichment_topology.sh'</tt></p></div></div>
 <div class="section">
 <h3><a name="METRON-448:Upgrading_via_Ansible_deployment_does_not_add_topology.classpath"></a><a class="externalLink" href="https://issues.apache.org/jira/browse/METRON-448">METRON-448:Upgrading via Ansible deployment does not add topology.classpath </a></h3>
 <div class="section">
@@ -361,23 +275,17 @@ name conflict, enrichment already defined
 <div class="section">
 <h4><a name="Workaround"></a>Workaround</h4>
 <p>Set the <tt>topology.classpath</tt> property for storm in Ambari to <tt>/etc/hbase/conf:/etc/hadoop/conf</tt></p></div></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[16/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-indexing/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-indexing/index.html b/site/current-book/metron-platform/metron-indexing/index.html
index 3d79323..6d4b8d6 100644
--- a/site/current-book/metron-platform/metron-indexing/index.html
+++ b/site/current-book/metron-platform/metron-indexing/index.html
@@ -1,378 +1,200 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-indexing/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Indexing</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Indexing</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Indexing</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Indexing</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Indexing</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Indexing</h1>
 <p><a name="Indexing"></a></p>
 <div class="section">
 <h2><a name="Introduction"></a>Introduction</h2>
 <p>The <tt>indexing</tt> topology is a topology dedicated to taking the data from the enrichment topology that have been enriched and storing the data in one or more supported indices</p>
-
 <ul>
-  
+
 <li>HDFS as rolled text files, one JSON blob per line</li>
-  
 <li>Elasticsearch</li>
-  
 <li>Solr</li>
 </ul>
 <p>By default, this topology writes out to both HDFS and one of Elasticsearch and Solr.</p>
 <p>Indices are written in batch and the batch size and batch timeout are specified in the <a href="#Sensor_Indexing_Configuration">Sensor Indexing Configuration</a> via the <tt>batchSize</tt> and <tt>batchTimeout</tt> parameters. These configs are variable by sensor type.</p></div>
 <div class="section">
+<h2><a name="Minimal_Assumptions_for_Message_Structure"></a>Minimal Assumptions for Message Structure</h2>
+<p>At minimum, a message should have a <tt>source.type</tt> field. Without this field, the message tuple will be failed and not written with an appropriate error indicated in the Storm UI and logs.</p></div>
+<div class="section">
 <h2><a name="Indexing_Architecture"></a>Indexing Architecture</h2>
 <p><img src="../../images/indexing_arch.png" alt="Architecture" /></p>
-<p>The indexing topology is extremely simple. Data is ingested into kafka and sent to </p>
-
+<p>The indexing topology is extremely simple.  Data is ingested into kafka and sent to</p>
 <ul>
-  
+
 <li>An indexing bolt configured to write to either elasticsearch or Solr</li>
-  
 <li>An indexing bolt configured to write to HDFS under <tt>/apps/metron/enrichment/indexed</tt></li>
 </ul>
 <p>By default, errors during indexing are sent back into the <tt>indexing</tt> kafka queue so that they can be indexed and archived.</p></div>
 <div class="section">
 <h2><a name="Sensor_Indexing_Configuration"></a>Sensor Indexing Configuration</h2>
-<p>The sensor specific configuration is intended to configure the indexing used for a given sensor type (e.g. <tt>snort</tt>). </p>
-<p>Just like the global config, the format is a JSON stored in zookeeper and on disk at <tt>$METRON_HOME/config/zookeeper/indexing</tt>. Within the sensor-specific configuration, you can configure the individual writers. The writers currently supported are:</p>
-
+<p>The sensor specific configuration is intended to configure the indexing used for a given sensor type (e.g. <tt>snort</tt>).</p>
+<p>Just like the global config, the format is a JSON stored in zookeeper and on disk at <tt>$METRON_HOME/config/zookeeper/indexing</tt>.  Within the sensor-specific configuration, you can configure the individual writers.  The writers currently supported are:</p>
 <ul>
-  
+
 <li><tt>elasticsearch</tt></li>
-  
 <li><tt>hdfs</tt></li>
-  
 <li><tt>solr</tt></li>
 </ul>
 <p>Depending on how you start the indexing topology, it will have either elasticsearch or solr and hdfs writers running.</p>
 <p>The configuration for an individual writer-specific configuration is a JSON map with the following fields:</p>
-
 <ul>
-  
+
 <li><tt>index</tt> : The name of the index to write to (defaulted to the name of the sensor).</li>
-  
 <li><tt>batchSize</tt> : The size of the batch that is written to the indices at once. Defaults to <tt>1</tt> (no batching).</li>
-  
-<li><tt>batchTimeout</tt> : The timeout after which a batch will be flushed even if batchSize has not been met. Optional. If unspecified, or set to <tt>0</tt>, it defaults to a system-determined duration which is a fraction of the Storm parameter <tt>topology.message.timeout.secs</tt>. Ignored if batchSize is <tt>1</tt>, since this disables batching.</li>
-  
+<li><tt>batchTimeout</tt> : The timeout after which a batch will be flushed even if batchSize has not been met.  Optional. If unspecified, or set to <tt>0</tt>, it defaults to a system-determined duration which is a fraction of the Storm parameter <tt>topology.message.timeout.secs</tt>.  Ignored if batchSize is <tt>1</tt>, since this disables batching.</li>
 <li><tt>enabled</tt> : Whether the writer is enabled (default <tt>true</tt>).</li>
 </ul>
 <div class="section">
 <h3><a name="Meta_Alerts"></a>Meta Alerts</h3>
-<p>Alerts can be grouped, after appropriate searching, into a set of alerts called a meta alert. A meta alert is useful for maintaining the context of searching and grouping during further investigations. Standard searches can return meta alerts, but grouping and other aggregation or sorting requests will not, because there&#x2019;s not a clear way to aggregate in many cases if there are multiple alerts contained in the meta alert. All meta alerts will have the source type of metaalert, regardless of the contained alert&#x2019;s origins.</p></div>
+<p>Alerts can be grouped, after appropriate searching, into a set of alerts called a meta alert.  A meta alert is useful for maintaining the context of searching and grouping during further investigations. Standard searches can return meta alerts, but grouping and other aggregation or sorting requests will not, because there&#x2019;s not a clear way to aggregate in many cases if there are multiple alerts contained in the meta alert. All meta alerts will have the source type of metaalert, regardless of the contained alert&#x2019;s origins.</p></div>
 <div class="section">
 <h3><a name="Elasticsearch"></a>Elasticsearch</h3>
-<p>Metron comes with built-in templates for the default sensors for Elasticsearch. When adding a new sensor, it will be necessary to add a new template defining the output fields appropriately. In addition, there is a requirement for a field <tt>alert</tt> of type <tt>nested</tt> for Elasticsearch 2.x installs. This is detailed at <a href="../metron-elasticsearch/index.html#Using_Metron_with_Elasticsearch_2.x">Using Metron with Elasticsearch 2.x</a></p></div>
+<p>Metron comes with built-in templates for the default sensors for Elasticsearch. When adding a new sensor, it will be necessary to add a new template defining the output fields appropriately. In addition, there is a requirement for a field <tt>alert</tt> of type <tt>nested</tt> for Elasticsearch 2.x installs.  This is detailed at <a href="../metron-elasticsearch/index.html#Using_Metron_with_Elasticsearch_2.x">Using Metron with Elasticsearch 2.x</a></p></div>
 <div class="section">
 <h3><a name="Indexing_Configuration_Examples"></a>Indexing Configuration Examples</h3>
-<p>For a given sensor, the following scenarios would be indicated by the following cases:</p>
+<p>For a given  sensor, the following scenarios would be indicated by the following cases:</p>
 <div class="section">
 <h4><a name="Base_Case"></a>Base Case</h4>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
 }
 </pre></div></div>
-<p>or no file at all.</p>
 
+<p>or no file at all.</p>
 <ul>
-  
+
 <li>elasticsearch writer
-  
 <ul>
-    
+
 <li>enabled</li>
-    
 <li>batch size of 1</li>
-    
 <li>batch timeout system default</li>
-    
 <li>index name the same as the sensor</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>hdfs writer
-  
 <ul>
-    
+
 <li>enabled</li>
-    
 <li>batch size of 1</li>
-    
 <li>batch timeout system default</li>
-    
 <li>index name the same as the sensor</li>
-  </ul></li>
 </ul>
-<p>If a writer config is unspecified, then a warning is indicated in the Storm console. e.g.: <tt>WARNING: Default and (likely) unoptimized writer config used for hdfs writer and sensor squid</tt></p></div>
+</li>
+</ul>
+<p>If a writer config is unspecified, then a warning is indicated in the Storm console.  e.g.: <tt>WARNING: Default and (likely) unoptimized writer config used for hdfs writer and sensor squid</tt></p></div>
 <div class="section">
 <h4><a name="Fully_specified"></a>Fully specified</h4>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
    &quot;elasticsearch&quot;: {
       &quot;index&quot;: &quot;foo&quot;,
       &quot;batchSize&quot; : 100,
@@ -389,39 +211,32 @@
 </pre></div></div>
 
 <ul>
-  
+
 <li>elasticsearch writer
-  
 <ul>
-    
+
 <li>enabled</li>
-    
 <li>batch size of 100</li>
-    
 <li>batch timeout system default</li>
-    
 <li>index name of &#x201c;foo&#x201d;</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>hdfs writer
-  
 <ul>
-    
+
 <li>enabled</li>
-    
 <li>batch size of 1</li>
-    
 <li>batch timeout system default</li>
-    
 <li>index name of &#x201c;foo&#x201d;</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h4><a name="HDFS_Writer_turned_off"></a>HDFS Writer turned off</h4>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
    &quot;elasticsearch&quot;: {
       &quot;index&quot;: &quot;foo&quot;,
       &quot;enabled&quot; : true 
@@ -436,43 +251,37 @@
 </pre></div></div>
 
 <ul>
-  
+
 <li>elasticsearch writer
-  
 <ul>
-    
+
 <li>enabled</li>
-    
 <li>batch size of 1</li>
-    
 <li>batch timeout system default</li>
-    
 <li>index name of &#x201c;foo&#x201d;</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>hdfs writer
-  
 <ul>
-    
+
 <li>disabled</li>
-  </ul></li>
+</ul>
+</li>
 </ul>
 <p><a name="Updates_to_Indexed_Data"></a></p>
 <h1>Updates to Indexed Data</h1>
 <p>There are clear usecases where we would want to incorporate the capability to update indexed data. Thus far, we have limited capabilities provided to support this use-case:</p>
-
 <ul>
-  
+
 <li>Updates to the random access index (e.g. Elasticsearch and Solr) should be supported</li>
-  
 <li>Updates to the cold storage index (e.g. HDFS) is not supported currently, however to support the batch use-case updated documents will be provided in a NoSQL write-ahead log (e.g. a HBase table) and an Java API will be provided to retrieve those updates scalably (i.e. a scan-free architecture).</li>
 </ul>
 <p>Put simply, the random access index will be always up-to-date, but the HDFS index will need to be joined to the NoSQL write-ahead log to get current updates.</p></div></div></div>
 <div class="section">
 <h2><a name="The_IndexDao_Abstraction"></a>The <tt>IndexDao</tt> Abstraction</h2>
 <p>The indices mentioned above as part of Update should be pluggable by the developer so that new write-ahead logs or real-time indices can be supported by providing an implementation supporting the data access patterns.</p>
-<p>To support a new index, one would need to implement the <tt>org.apache.metron.indexing.dao.IndexDao</tt> abstraction and provide update and search capabilities. IndexDaos may be composed and updates will be performed in parallel. This enables a flexible strategy for specifying your backing store for updates at runtime. For instance, currently the REST API supports the update functionality and may be configured with a list of IndexDao implementations to use to support the updates.</p>
-<p>Updates with the IndexDao.update method replace the current object with the new object. For partial updates, use IndexDao.patch instead.</p>
+<p>To support a new index, one would need to implement the <tt>org.apache.metron.indexing.dao.IndexDao</tt> abstraction and provide update and search capabilities.  IndexDaos may be composed and updates will be performed in parallel.  This enables a flexible strategy for specifying your backing store for updates at runtime. For instance, currently the REST API supports the update functionality and may be configured with a list of IndexDao implementations to use to support the updates.</p>
+<p>Updates with the IndexDao.update method replace the current object with the new object.  For partial updates, use IndexDao.patch instead.</p>
 <div class="section">
 <h3><a name="The_HBaseDao"></a>The <tt>HBaseDao</tt></h3>
 <p>Updates will be written to HBase. The key structure includes the GUID and sensor type and for each new version, a new column is created with value as the message.</p>
@@ -485,71 +294,87 @@
 <p>The HBase column family to use for message updates.</p></div></div>
 <div class="section">
 <h3><a name="The_MetaAlertDao"></a>The <tt>MetaAlertDao</tt></h3>
-<p>The goal of meta alerts is to be able to group together a set of alerts while being able to transparently perform actions like searches, as if meta alerts were normal alerts. <tt>org.apache.metron.indexing.dao.MetaAlertDao</tt> extends <tt>IndexDao</tt> and enables several features: </p>
-
+<p>The goal of meta alerts is to be able to group together a set of alerts while being able to transparently perform actions like searches, as if meta alerts were normal alerts.  <tt>org.apache.metron.indexing.dao.MetaAlertDao</tt> extends <tt>IndexDao</tt> and enables several features:</p>
 <ul>
-  
+
 <li>the ability to get all meta alerts associated with an alert</li>
-  
 <li>creation of a meta alert</li>
-  
 <li>adding alerts to a meta alert</li>
-  
 <li>removing alerts from a meta alert</li>
-  
 <li>changing a meta alert&#x2019;s status</li>
 </ul>
-<p>The implementation of this is to denormalize the relationship between alerts and meta alerts, and store alerts as a nested field within a meta alert. The use of nested fields is to avoid the limitations of parent-child relationships (one-to-many) and merely linking by IDs (which causes issues with pagination as a result of being unable to join indices). A list of containing meta alerts is stored on an alert for the purpose of keeping source alerts and alerts contained in meta alerts in sync.</p>
+<p>The implementation of this is to denormalize the relationship between alerts and meta alerts, and store alerts as a nested field within a meta alert. The use of nested fields is to avoid the limitations of parent-child relationships (one-to-many) and merely linking by IDs (which causes issues with pagination as a result of being unable to join indices).  A list of containing meta alerts is stored on an alert for the purpose of keeping source alerts and alerts contained in meta alerts in sync.</p>
 <p>The search functionality of <tt>IndexDao</tt> is wrapped by the <tt>MetaAlertDao</tt> in order to provide both regular and meta alerts side-by-side with sorting. The updating capabilities are similarly wrapped, in order to ensure updates are carried through both the alerts and associated meta alerts. Both of these functions are handled under the hood.</p>
 <p>In addition, API endpoints have been added to expose the features listed above. The denormalization handles the case of going from meta alert to alert automatically.</p>
 <p><a name="Notes_on_Performance_Tuning"></a></p>
 <h1>Notes on Performance Tuning</h1>
-<p>Default installed Metron is untuned for production deployment. By far and wide, the most likely piece to require TLC from a performance perspective is the indexing layer. An index that does not keep up will back up and you will see errors in the kafka bolt. There are a few knobs to tune to get the most out of your system.</p></div></div>
+<p>Default installed Metron is untuned for production deployment.  By far and wide, the most likely piece to require TLC from a performance perspective is the indexing layer.  An index that does not keep up will back up and you will see errors in the kafka bolt.  There are a few knobs to tune to get the most out of your system.</p></div></div>
 <div class="section">
 <h2><a name="Kafka_Queue"></a>Kafka Queue</h2>
-<p>The <tt>indexing</tt> kafka queue is a collection point from the enrichment topology. As such, make sure that the number of partitions in the kafka topic is sufficient to handle the throughput that you expect.</p></div>
+<p>The <tt>indexing</tt> kafka queue is a collection point from the enrichment topology.  As such, make sure that the number of partitions in the kafka topic is sufficient to handle the throughput that you expect.</p></div>
 <div class="section">
 <h2><a name="Indexing_Topology"></a>Indexing Topology</h2>
-<p>The <tt>indexing</tt> topology as started by the <tt>$METRON_HOME/bin/start_elasticsearch_topology.sh</tt> or <tt>$METRON_HOME/bin/start_solr_topology.sh</tt> script uses a default of one executor per bolt. In a real production system, this should be customized by modifying the flux file in <tt>$METRON_HOME/flux/indexing/remote.yaml</tt>. </p>
-
+<p>The <tt>indexing</tt> topology as started by the <tt>$METRON_HOME/bin/start_elasticsearch_topology.sh</tt> or <tt>$METRON_HOME/bin/start_solr_topology.sh</tt> script uses a default of one executor per bolt.  In a real production system, this should be customized by modifying the flux file in <tt>$METRON_HOME/flux/indexing/remote.yaml</tt>.</p>
 <ul>
-  
-<li>Add a <tt>parallelism</tt> field to the bolts to give Storm a parallelism  hint for the various components. Give bolts which appear to be bottlenecks (e.g. the indexing bolt) a larger hint.</li>
-  
+
+<li>Add a <tt>parallelism</tt> field to the bolts to give Storm a parallelism hint for the various components.  Give bolts which appear to be bottlenecks (e.g. the indexing bolt) a larger hint.</li>
 <li>Add a <tt>parallelism</tt> field to the kafka spout which matches the number of partitions for the enrichment kafka queue.</li>
-  
-<li>Adjust the number of workers for the topology by adjusting the  <tt>topology.workers</tt> field for the topology.</li>
+<li>Adjust the number of workers for the topology by adjusting the <tt>topology.workers</tt> field for the topology.</li>
 </ul>
 <p>Finally, if workers and executors are new to you or you don&#x2019;t know where to modify the flux file, the following might be of use to you:</p>
-
 <ul>
-  
+
 <li><a class="externalLink" href="http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/">Understanding the Parallelism of a Storm Topology</a></li>
-  
 <li><a class="externalLink" href="http://storm.apache.org/releases/current/flux.html">Flux Docs</a></li>
-</ul></div>
+</ul>
+<div class="section">
+<h3><a name="Rest_endpoints"></a>Rest endpoints</h3>
+<p>There are rest endpoints available to perform operations like start, stop, activate, deactivate on the <tt>indexing</tt> topologies.</p>
+<table border="0" class="table table-striped">
+<thead>
+
+<tr class="a">
+<th>            </th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td> <a href="../../metron-interface/metron-rest/index.html#GET_apiv1stormindexingbatch"> <tt>GET /api/v1/storm/indexing/batch</tt></a></td></tr>
+<tr class="a">
+<td> <a href="../../metron-interface/metron-rest/index.html#GET_apiv1stormindexingbatchactivate"> <tt>GET /api/v1/storm/indexing/batch/activate</tt></a></td></tr>
+<tr class="b">
+<td> <a href="../../metron-interface/metron-rest/index.html#GET_apiv1stormindexingbatchdeactivate"> <tt>GET /api/v1/storm/indexing/batch/deactivate</tt></a></td></tr>
+<tr class="a">
+<td> <a href="../../metron-interface/metron-rest/index.html#GET_apiv1stormindexingbatchstart"> <tt>GET /api/v1/storm/indexing/batch/start</tt></a></td></tr>
+<tr class="b">
+<td> <a href="../../metron-interface/metron-rest/index.html#GET_apiv1stormindexingbatchstop"> <tt>GET /api/v1/storm/indexing/batch/stop</tt></a></td></tr>
+<tr class="a">
+<td> <a href="../../metron-interface/metron-rest/index.html#GET_apiv1stormindexingrandomaccess"> <tt>GET /api/v1/storm/indexing/randomaccess</tt></a></td></tr>
+<tr class="b">
+<td> <a href="../../metron-interface/metron-rest/index.html#GET_apiv1stormindexingrandomaccessactivate"> <tt>GET /api/v1/storm/indexing/randomaccess/activate</tt></a></td></tr>
+<tr class="a">
+<td> <a href="../../metron-interface/metron-rest/index.html#GET_apiv1stormindexingrandomaccessdeactivate"> <tt>GET /api/v1/storm/indexing/randomaccess/deactivate</tt></a></td></tr>
+<tr class="b">
+<td> <a href="../../metron-interface/metron-rest/index.html#GET_apiv1stormindexingrandomaccessstart"> <tt>GET /api/v1/storm/indexing/randomaccess/start</tt></a></td></tr>
+<tr class="a">
+<td> <a href="../../metron-interface/metron-rest/index.html#GET_apiv1stormindexingrandomaccessstop"> <tt>GET /api/v1/storm/indexing/randomaccess/stop</tt></a></td></tr>
+</tbody>
+</table></div></div>
 <div class="section">
 <h2><a name="Zeppelin_Notebooks"></a>Zeppelin Notebooks</h2>
-<p>Zeppelin notebooks can be added to <tt>/src/main/config/zeppelin/</tt> (and subdirectories can be created for organization). The placed files must be .json files and be named appropriately. These files must be added to the metron.spec file and the RPMs rebuilt to be available to be loaded into Ambari.</p>
+<p>Zeppelin notebooks can be added to <tt>/src/main/config/zeppelin/</tt> (and subdirectories can be created for organization).  The placed files must be .json files and be named appropriately. These files must be added to the metron.spec file and the RPMs rebuilt to be available to be loaded into Ambari.</p>
 <p>The notebook files will be found on the server in <tt>$METRON_HOME/config/zeppelin</tt></p>
 <p>The Ambari Management Pack has a custom action to load these templates, ZEPPELIN_DASHBOARD_INSTALL, that will import them into Zeppelin.</p></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[28/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html b/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
index eeadfe6..5cf9775 100644
--- a/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
+++ b/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
@@ -1,387 +1,230 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; </title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../../css/site.css" />
     <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class=""></li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active "></li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                                  
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-down"></i>
-        Other-examples</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Manual_Install_CentOS6</a>
-          </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-down"></span>Other-examples</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>Manual_Install_CentOS6</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <div class="section">
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<div class="section">
 <h2><a name="Metron_0.4.0_with_HDP_2.5_bare-metal_install_on_Centos_6_with_MariaDB_for_Metron_REST:"></a>Metron 0.4.0 with HDP 2.5 bare-metal install on Centos 6 with MariaDB for Metron REST:</h2>
 <div class="section">
 <h3><a name="Introduction"></a>Introduction</h3>
 <p>We will be installing Metron 0.4.0 with HDP 2.5 on CentOS 6. We will also install MariaDB as a database for Metron REST. Additionally, we&#x2019;ll also install Apache NiFi. I installed Metron in a test environment with 3 VMs to try it out as well as a single node. I&#x2019;ll try to write this guide so that the necessary steps can easily be adapted for other environments.</p></div>
 <div class="section">
 <h3><a name="Environment"></a>Environment</h3>
-
 <ul>
-  
+
 <li>
-<p>Single node: 4 CPUs, 16 GB RAM.</p></li>
-  
+
+<p>Single node: 4 CPUs, 16 GB RAM.</p>
+</li>
 <li>
+
 <p>Multiple nodes:</p>
-  
 <ul>
-    
+
 <li>3 VMs, 2 CPUs per VM and 8 GB RAM per VM.</li>
-    
 <li>Hosts: 10.10.10.1 node1 10.10.10.2 node2 10.10.10.3 node3</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h3><a name="Prerequisites:"></a>Prerequisites:</h3>
-
 <ul>
-  
+
 <li>
-<p>CentOS 6</p></li>
-  
+
+<p>CentOS 6</p>
+</li>
 <li>
-<p>Add the epel repository and install tmux, vim &amp; htop. Installing these utilities is not strictly necessary, but I install these by default for potential troubleshooting &amp; editing of files locally):</p></li>
+
+<p>Add the epel repository and install tmux, vim &amp; htop. Installing these utilities is not strictly necessary, but I install these by default for potential troubleshooting &amp; editing of files locally):</p>
+</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># yum install epel-release -y
+<div>
+<div>
+<pre class="source"># yum install epel-release -y
 # yum update -y
 # yum install vim tmux htop -y
 </pre></div></div>
 
 <ul>
-  
+
 <li>Set up passwordless SSH between our nodes: If passwordless ssh has not yet been set up within the cluster, then in main node generate key:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># cat /dev/zero | ssh-keygen -q -N &quot;&quot; 2&gt;/dev/null
+<div>
+<div>
+<pre class="source"># cat /dev/zero | ssh-keygen -q -N &quot;&quot; 2&gt;/dev/null
 # cd ~/.ssh
 # cat id_rsa.pub &gt;&gt; authorized_keys
 </pre></div></div>
 
 <ul>
-  
+
 <li>If you&#x2019;re not installing on a single node, add this newly generated key to all the slave nodes:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>ssh-copy-id -i ~/.ssh/id_rsa.pub &lt;replace_with_node_ip&gt;
+<div>
+<div>
+<pre class="source">ssh-copy-id -i ~/.ssh/id_rsa.pub &lt;replace_with_node_ip&gt;
 </pre></div></div>
-<p><b>Side note:</b> You might have to adapt your sshd_config file and add &#x201c;PermitRootLogin yes&#x201d; amongst other parameters if you want passwordless root access, but that&#x2019;s outside the scope of this document.</p>
 
+<p><b>Side note:</b> You might have to adapt your sshd_config file and add &#x201c;PermitRootLogin yes&#x201d; amongst other parameters if you want passwordless root access, but that&#x2019;s outside the scope of this document.</p>
 <ul>
-  
+
 <li>Increase limits for ElasticSearch and Storm on nodes where you will be installing them (if you don&#x2019;t know, increase it everywhere):</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># echo -e &quot;elasticsearch - memlock unlimited\nstorm - nproc 257597&quot; &gt;&gt; /etc/security/limits.conf
+<div>
+<div>
+<pre class="source"># echo -e &quot;elasticsearch - memlock unlimited\nstorm - nproc 257597&quot; &gt;&gt; /etc/security/limits.conf
 </pre></div></div>
 
 <ul>
-  
+
 <li>Adjust limits to secure level (<a class="externalLink" href="https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_installing_manually_book/content/ref-729d1fb0-6d1b-459f-a18a-b5eba4540ab5.1.html">link</a>):</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># ulimit -n 32768
+<div>
+<div>
+<pre class="source"># ulimit -n 32768
 # ulimit -u 65536
 # echo -e &quot;* - nofile 32768\n* - nproc 65536&quot; &gt;&gt; /etc/security/limits.conf
 </pre></div></div>
 
 <ul>
-  
+
 <li>Disable IPv6, leaving it enabled may force service to bind to IPv6 addresses only and thus resulting in inability to connect to it (<a class="externalLink" href="https://wiki.centos.org/FAQ/CentOS6#head-d47139912868bcb9d754441ecb6a8a10d41781df">source link</a>): Disable for the running system:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># sysctl -w net.ipv6.conf.all.disable_ipv6=1
+<div>
+<div>
+<pre class="source"># sysctl -w net.ipv6.conf.all.disable_ipv6=1
 # sysctl -w net.ipv6.conf.default.disable_ipv6=1
 or
 # echo 1 &gt; /proc/sys/net/ipv6/conf/all/disable_ipv6
 # echo 1 &gt; /proc/sys/net/ipv6/conf/default/disable_ipv6
 </pre></div></div>
+
 <p>To survive a reboot: Add: net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 To: /etc/sysctl.conf</p>
 
-<div class="source">
-<div class="source">
-<pre># echo -e &quot;\n# Disable IPv6\nnet.ipv6.conf.all.disable_ipv6 = 1\nnet.ipv6.conf.default.disable_ipv6 = 1&quot; &gt;&gt; /etc/sysctl.conf
+<div>
+<div>
+<pre class="source"># echo -e &quot;\n# Disable IPv6\nnet.ipv6.conf.all.disable_ipv6 = 1\nnet.ipv6.conf.default.disable_ipv6 = 1&quot; &gt;&gt; /etc/sysctl.conf
 </pre></div></div>
 
 <ul>
-  
+
 <li>Disable Transparent Hugepage. Add &#x201c;transparent_hugepage=never&#x201d; to the end of the kernel line in /boot/grub/grub.conf and reboot. (Ambari demands it, do we need to comply?):</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>Add &quot;transparent_hugepage=never&quot; in the kernel line after &quot;quiet:
+<div>
+<div>
+<pre class="source">Add &quot;transparent_hugepage=never&quot; in the kernel line after &quot;quiet:
 &quot;kernel /vmlinuz-2.6.32-696.3.1.el6.x86_64 ro root=/dev/mapper/vg_centos6-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_centos6/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=128M rd_LVM_LV=vg_centos6/lv_root  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet&quot;
 becomes:
 &quot;kernel /vmlinuz-2.6.32-696.3.1.el6.x86_64 ro root=/dev/mapper/vg_centos6-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_centos6/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=128M rd_LVM_LV=vg_centos6/lv_root  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet transparent_hugepage=never&quot;
@@ -391,13 +234,13 @@ Afterwards, run:
 </pre></div></div>
 
 <ul>
-  
+
 <li>If you do not want to mess with grub/kernel parameters, add the following to /etc/rc.local:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>vim /etc/rc.local:
+<div>
+<div>
+<pre class="source">vim /etc/rc.local:
 # Disable THP at boot time
 if test -f /sys/kernel/mm/redhat_transparent_hugepage/enabled; then
   echo never &gt; /sys/kernel/mm/redhat_transparent_hugepage/enabled
@@ -407,87 +250,90 @@ if test -f /sys/kernel/mm/redhat_transparent_hugepage/defrag; then
   echo never &gt; /sys/kernel/mm/redhat_transparent_hugepage/defrag
 fi
 </pre></div></div>
+
 <p>After reboot check that changes were applied (make sure that word &#x201c;never&#x201d; is selected in square-brackets):</p>
 
-<div class="source">
-<div class="source">
-<pre># cat /sys/kernel/mm/transparent_hugepage/enabled
+<div>
+<div>
+<pre class="source"># cat /sys/kernel/mm/transparent_hugepage/enabled
 always madvise [never]
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Metron_install_pre-preparation:"></a>Metron install pre-preparation:</h3>
-
 <ul>
-  
+
 <li>On all nodes Install pre-requisites for Ambari:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># yum install git wget curl rpm tar unzip bzip2 wget createrepo yum-utils ntp python-pip psutils python-psutil ntp libffi-devel gcc openssl-devel npm -y
+<div>
+<div>
+<pre class="source"># yum install git wget curl rpm tar unzip bzip2 wget createrepo yum-utils ntp python-pip psutils python-psutil ntp libffi-devel gcc openssl-devel npm -y
 # pip install --upgrade pip
 # pip install requests urllib
 # pip install --upgrade setuptools
 </pre></div></div>
 
 <ul>
-  
+
 <li>Install Maven 3.3.9 on main node and on Metron node install java 1.8 (if you don&#x2019;t know which it is, install it everywhere):</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel -y
+<div>
+<div>
+<pre class="source"># yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel -y
 </pre></div></div>
 
 <ul>
-  
+
 <li>Set path to Java 8 if it does not exist:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># export JAVA_HOME=$(readlink -f /usr/bin/java | sed &quot;s_/jre/bin/java__&quot;)
+<div>
+<div>
+<pre class="source"># export JAVA_HOME=$(readlink -f /usr/bin/java | sed &quot;s_/jre/bin/java__&quot;)
 </pre></div></div>
 
 <ul>
-  
+
 <li>Save export for future reboots:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># echo 'export JAVA_HOME=$(readlink -f /usr/bin/java | sed &quot;s_/jre/bin/java__&quot;)' &gt; /etc/profile.d/java_18.sh
+<div>
+<div>
+<pre class="source"># echo 'export JAVA_HOME=$(readlink -f /usr/bin/java | sed &quot;s_/jre/bin/java__&quot;)' &gt; /etc/profile.d/java_18.sh
 # chmod +x /etc/profile.d/java_18.sh
 # source /etc/profile.d/java_18.sh
 </pre></div></div>
 
 <ul>
-  
+
 <li>Download and install Maven 3.3.9:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># wget http://apache.volia.net/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
+<div>
+<div>
+<pre class="source"># wget http://apache.volia.net/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
 # tar -zxf apache-maven-3.3.9-bin.tar.gz
 # mv apache-maven-3.3.9 /opt
 # PATH=/opt/apache-maven-3.3.9/bin:$PATH
 # echo 'export PATH=/opt/apache-maven-3.3.9/bin:$PATH' &gt; /etc/profile.d/maven.sh
 # chmod +x /etc/profile.d/maven.sh
 </pre></div></div>
+
 <p>Check whether Maven works:</p>
 
-<div class="source">
-<div class="source">
-<pre># source /etc/profile.d/maven.sh
+<div>
+<div>
+<pre class="source"># source /etc/profile.d/maven.sh
 # mvn -V
 </pre></div></div>
+
 <p>You should see something similar to:</p>
 
-<div class="source">
-<div class="source">
-<pre>[root@base1 ~]# mvn -V
+<div>
+<div>
+<pre class="source">[root@base1 ~]# mvn -V
 Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T08:41:47-08:00)
 Maven home: /opt/apache-maven-3.3.9
 Java version: 1.8.0_131, vendor: Oracle Corporation
@@ -513,54 +359,59 @@ OS name: &quot;linux&quot;, version: &quot;3.10.0-514.16.1.el7.x86_64&quot;, arc
 </pre></div></div>
 
 <ul>
-  
+
 <li>On Ambari node install and enable docker (we will need it to build Metron mpack for Ambari):</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># yum install docker-io -y
+<div>
+<div>
+<pre class="source"># yum install docker-io -y
 # service docker start
 </pre></div></div>
 
 <ul>
-  
+
 <li>Also on your build box, install npm. This is needed to build metron-config, part of the UI.</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># yum install npm -y
+<div>
+<div>
+<pre class="source"># yum install npm -y
 </pre></div></div>
 
 <ul>
-  
+
 <li>
-<p>Remove ipv4 &#x2018;localhost.localdomain&#x2019; from /etc/hosts</p></li>
-  
+
+<p>Remove ipv4 &#x2018;localhost.localdomain&#x2019; from /etc/hosts</p>
+</li>
 <li>
-<p>Remove ipv6 &#x2018;localhost.localdomain&#x2019; from /etc/hosts</p></li>
-  
+
+<p>Remove ipv6 &#x2018;localhost.localdomain&#x2019; from /etc/hosts</p>
+</li>
 <li>
-<p>Add &#x201c;127.0.0.1 localhost&#x201d; to /etc/hosts</p></li>
-  
+
+<p>Add &#x201c;127.0.0.1    localhost&#x201d; to /etc/hosts</p>
+</li>
 <li>
-<p>Install the database we will use for Metron REST:</p></li>
+
+<p>Install the database we will use for Metron REST:</p>
+</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># yum install mariadb-server mysql-connector-java -y
+<div>
+<div>
+<pre class="source"># yum install mariadb-server mysql-connector-java -y
 </pre></div></div>
 
 <ul>
-  
+
 <li>Configure a user and a database for Metron REST: If you haven&#x2019;t run <tt>mysql_secure_installation</tt> after the database installation, do that first:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># service mysqld start
+<div>
+<div>
+<pre class="source"># service mysqld start
 # /sbin/chkconfig --add mysqld
 # /sbin/chkconfig --list mysqld
 # /sbin/chkconfig mysqld on
@@ -628,101 +479,103 @@ Thanks for using MySQL!
 
 Cleaning up...
 #
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Build_Metron_code"></a>Build Metron code</h3>
 <p>Now we are going to start to building Metron. At the time of writing, Metron 0.4.0 was in the final stages of being released.</p>
-
 <ul>
-  
+
 <li>On the main node, clone the Metron repository:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># git clone https://github.com/apache/metron
+<div>
+<div>
+<pre class="source"># git clone https://github.com/apache/metron
 </pre></div></div>
+
 <p>If you want to make sure you&#x2019;re on the 0.4.0 release branch, do:</p>
 
-<div class="source">
-<div class="source">
-<pre># git clone https://github.com/apache/metron
+<div>
+<div>
+<pre class="source"># git clone https://github.com/apache/metron
 # cd metron
 # git checkout Metron_0.4.0
 </pre></div></div>
 
 <ul>
-  
+
 <li>Build Metron with HDP 2.5 profile:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># cd metron
+<div>
+<div>
+<pre class="source"># cd metron
 # mvn clean package -DskipTests -T 2C -P HDP-2.5.0.0,mpack
 # cd metron-deployment/packaging/docker/rpm-docker
 # mvn clean install -DskipTests -PHDP-2.5.0.0
 </pre></div></div>
-<p>If for some reason, the rpm-docker fails with the message &#x201c;/bin/bash: ./build.sh: Permission denied&#x201d;, try disabling selinux (&#x201c;setenforce 0&#x201d;) and run &#x201c;mvn clean install -DskipTests -PHDP-2.5.0.0&#x201d; again.</p>
 
+<p>If for some reason, the rpm-docker fails with the message &#x201c;/bin/bash: ./build.sh: Permission denied&#x201d;, try disabling selinux (&#x201c;setenforce 0&#x201d;) and run &#x201c;mvn clean install -DskipTests -PHDP-2.5.0.0&#x201d; again.</p>
 <ul>
-  
+
 <li>On all nodes, create a localrepo directory and copy the RPMs from Ambari node there:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># mkdir /localrepo
+<div>
+<div>
+<pre class="source"># mkdir /localrepo
 # cp -rp /root/metron/metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/* /localrepo/
 # createrepo /localrepo
 </pre></div></div>
+
 <p>If you&#x2019;re doing a multi node install, also create localrepo on the nodes and copy the packages to the other nodes:</p>
 
-<div class="source">
-<div class="source">
-<pre># ssh root@node2 mkdir /localrepo
+<div>
+<div>
+<pre class="source"># ssh root@node2 mkdir /localrepo
 # scp /localrepo/*\.rpm root@node2:/localrepo/.
 # ssh root@node2 yum install createrepo -y
 # ssh root@node2 createrepo /localrepo
 </pre></div></div>
 
 <ul>
-  
+
 <li>Make sure to do the above on each node.</li>
 </ul>
 <p>Fetch &amp; create logrotate script for Hadoop Services:</p>
 
-<div class="source">
-<div class="source">
-<pre># wget -O /etc/logrotate.d/metron-ambari https://raw.githubusercontent.com/apache/metron/master/metron-deployment/roles/ambari_common/templates/metron-hadoop-logrotate.yml
+<div>
+<div>
+<pre class="source"># wget -O /etc/logrotate.d/metron-ambari https://raw.githubusercontent.com/apache/metron/master/metron-deployment/roles/ambari_common/templates/metron-hadoop-logrotate.yml
 # sed -i 's/^  {{ hadoop_logrotate_frequency }}.*$/  daily/' /etc/logrotate.d/metron-ambari
 # sed -i 's/^  rotate {{ hadoop_logrotate_retention }}.*$/  rotate 30/' /etc/logrotate.d/metron-ambari
 # chmod 0644 /etc/logrotate.d/metron-ambari
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Ambari_2.4_with_HDP_2.5_install"></a>Ambari 2.4 with HDP 2.5 install</h3>
 <p>Inspired by: [http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.1.0/bk_ambari-installation/content/ch_Getting_Ready.html]</p>
-
 <ul>
-  
+
 <li>Adjust limits to secure level (inspired by <a class="externalLink" href="https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_installing_manually_book/content/ref-729d1fb0-6d1b-459f-a18a-b5eba4540ab5.1.html">link</a>):</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># ulimit -n 32768
+<div>
+<div>
+<pre class="source"># ulimit -n 32768
 # ulimit -u 65536
 # echo -e &quot;* - nofile 32768\n* - nproc 65536&quot; &gt;&gt; /etc/security/limits.conf
 </pre></div></div>
 
 <ul>
-  
+
 <li>Enable time sync, disable firewall and SElinux on every node:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># yum install ntp -y
+<div>
+<div>
+<pre class="source"># yum install ntp -y
 # service ntpd start
 # /sbin/chkconfig --add ntpd
 # /sbin/chkconfig --list ntpd
@@ -731,100 +584,99 @@ Cleaning up...
 </pre></div></div>
 
 <ul>
-  
+
 <li>Disable firewall on every node:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># service iptables save
+<div>
+<div>
+<pre class="source"># service iptables save
 # service iptables stop
 # chkconfig iptables off
 </pre></div></div>
 
 <ul>
-  
+
 <li>Disable IPv6 firewall on every node:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># service ip6tables save
+<div>
+<div>
+<pre class="source"># service ip6tables save
 # service ip6tables stop
 # chkconfig ip6tables off
 </pre></div></div>
 
 <ul>
-  
-<li>Disable SElinux on every node:</li>
+
+<li>Disable SElinux  on every node:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># setenforce 0 (=&gt; I know, but for the sake of simplicity, quickness &amp; testing, I've disabled selinux.)
+<div>
+<div>
+<pre class="source"># setenforce 0 (=&gt; I know, but for the sake of simplicity, quickness &amp; testing, I've disabled selinux.)
 </pre></div></div>
 
 <ul>
-  
+
 <li>Make sure each node can resolve every other node&#x2019;s hostname or add hostname of each node to <tt>/etc/hosts</tt> on every node. For example add following lines in /etc/hosts of each node: 10.10.10.1 node1 10.10.10.2 node2 10.10.10.3 node3</li>
 </ul>
 <p>Where 10.10.10.1, 10.10.10.2 and 10.10.10.3 are the IPs of your nodes and node1, node2 and node3 are hostnames.</p>
-
 <ul>
-  
+
 <li>On main node download and setup Ambari repo (you may replace the &#x201c;2.4.2.0&#x201d; with a newer Ambari version number):</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.2.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
+<div>
+<div>
+<pre class="source"># wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.2.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
 # yum update -y
 </pre></div></div>
 
 <ul>
-  
+
 <li>Check that it was added:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># yum repolist | grep ambari
+<div>
+<div>
+<pre class="source"># yum repolist | grep ambari
 Updates-ambari-2.4.2.0   ambari-2.4.2.0 - Updates                             12
 </pre></div></div>
 
 <ul>
-  
+
 <li>Install and setup Ambari server:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># yum install ambari-server -y
+<div>
+<div>
+<pre class="source"># yum install ambari-server -y
 # ambari-server setup -s &amp;&amp; touch /etc/ambari-server/configured
 </pre></div></div>
 
 <ul>
-  
+
 <li>Add Metron service to Ambari by running mpack command (make sure to specify correct path to mpack in &#x2013;mpack=):</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># ambari-server install-mpack --mpack=/root/metron/metron-deployment/packaging/ambari/metron-mpack/target/metron_mpack-0.4.0.0.tar.gz --verbose
+<div>
+<div>
+<pre class="source"># ambari-server install-mpack --mpack=/root/metron/metron-deployment/packaging/ambari/metron-mpack/target/metron_mpack-0.4.0.0.tar.gz --verbose
 </pre></div></div>
 
 <ul>
-  
+
 <li>Start Ambari:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># ambari-server start
+<div>
+<div>
+<pre class="source"># ambari-server start
 </pre></div></div>
 
 <ul>
-  
+
 <li>Access the Ambari UI by going to the following URL in a web browser: <tt>http://&lt;replace_with_master_node_ip&gt;:8080/</tt>. You can use admin/admin as username/password. Start the Install Wizard.</li>
 </ul>
 <p><b>Get Started page:</b> Enter any desired cluster name.</p>
@@ -834,349 +686,206 @@ Updates-ambari-2.4.2.0   ambari-2.4.2.0 - Updates                             12
 <p><b>Assign Masters:</b> Assign &#x201c;Kafka Broker&#x201d; on all nodes. Make sure move following components on one common node (Taken from previous guide, is this still necessary?): Storm UI Server Metron Indexing MySQL Server Kibana Server Elasticsearch Master Metron Parsers Metron Enrichment</p>
 <p><b>Assign Slaves and Clients:</b> select All for: DataNode NodeManager RegionServer Supervisor Client</p>
 <p><b>Customize Services:</b> Following is a list of services that need to be configured:</p>
-
 <ul>
-  
+
 <li>
-<p>Set the &#x201c;NameNode Java heap size&#x201d; (namenode_heapsize) from the default 1024 MB to at least 4096 MB under HDFS -&gt; Configs.</p></li>
-  
+
+<p>Set the &#x201c;NameNode Java heap size&#x201d; (namenode_heapsize) from the default 1024 MB to at least 4096 MB under HDFS -&gt; Configs.</p>
+</li>
 <li>
+
 <p>For ElasticSearch:</p>
-  
 <ul>
-    
+
 <li>Set &#x201c;zen_discovery_ping_unicast_hosts&#x201d; to the IP of the node where you assigned ElasticSearch Master on the Assign Master tab.</li>
-    
 <li>Under &#x201c;Advanced elastic-site&#x201d;: Change &#x201c;network_host&#x201d; to &#x201c;0.0.0.0&#x201d;. Do not do this if your Metron is exposed to the public internet! Is &#x201c;[ <i>local</i>, <i>site</i> ]&#x201d; now.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>
+
 <p>Kibana:</p>
-  
 <ul>
-    
+
 <li>Set &#x201c;kibana_es_url&#x201d; to <tt>http://&lt;replace_with_elasticsearch_master_hostname&gt;:9200</tt>. &#x201c;replace_with_elasticsearch_master_hostname&#x201d; is the IP of the node where you assigned ElasticSearch Master on the Assign Master tab.</li>
-    
-<li>Change kibana_default_application to &#x201c;dashboard/Metron-Dashboard&#x201d;</li>
-  </ul></li>
-  
+<li>Change kibana_default_application to &#x201c;dashboard/AV-YpDmwdXwc6Ua9Muh9&#x201d;</li>
+</ul>
+</li>
 <li>
-<p>Metron: Set &#x201c;Elasticsearch Hosts&#x201d; to the IP of the node where you assigned ElasticSearch Master on the Assign Master tab.</p></li>
-  
+
+<p>Metron: Set &#x201c;Elasticsearch Hosts&#x201d; to the IP of the node where you assigned ElasticSearch Master on the Assign Master tab.</p>
+</li>
 <li>
-<p>Storm: You might have to increase the number of supervisor.slots.ports from the default &#x201c;[6700, 6701]&#x201d; to [6700, 6701, 6702, 6703, 6704] if you&#x2019;re only installing a single node.</p></li>
-  
+
+<p>Storm: You might have to increase the number of supervisor.slots.ports from the default &#x201c;[6700, 6701]&#x201d; to [6700, 6701, 6702, 6703, 6704] if you&#x2019;re only installing a single node.</p>
+</li>
 <li>
+
 <p>For metron REST use:</p>
-  
-<div class="source">
-<div class="source">
-<pre>Metron JDBC client path: /usr/share/java/mysql-connector-java.jar
+
+<div>
+<div>
+<pre class="source">Metron JDBC client path: /usr/share/java/mysql-connector-java.jar
 Metron JDBC Driver: com.mysql.jdbc.Driver
 Metron JDBC password: &lt;DB PASSWORD&gt;
 Metron JDBC platform: mysql
 Metron JDBC URL: jdbc:mysql://127.0.0.1:3306/&lt;DB NAME&gt;
 Metron JDBC username: &lt;DB USERNAME&gt;
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Set rest of the configuration values to recommended by Ambari or the ones you desire (like DB passwords) and perform install. In a 3 node cluster, I ended up with:</p></li>
-</ul>
 
+<p>Set rest of the configuration values to recommended by Ambari or the ones you desire (like DB passwords) and perform install. In a 3 node cluster, I ended up with:</p>
+</li>
+</ul>
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
 <th>node1 </th>
-      
-<th>node2 </th>
-      
-<th>node3</th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> node2 </th>
+<th> node3</th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
 <td>DataNode </td>
-      
-<td>App Timeline Server </td>
-      
-<td>DataNode</td>
-    </tr>
-    
+<td> App Timeline Server </td>
+<td> DataNode</td></tr>
 <tr class="a">
-      
 <td>Elasticsearch Master </td>
-      
-<td>DataNode </td>
-      
-<td>Elasticsearch Data Node</td>
-    </tr>
-    
+<td> DataNode </td>
+<td> Elasticsearch Data Node</td></tr>
 <tr class="b">
-      
 <td>HBase Client </td>
-      
-<td>DRPC Server </td>
-      
-<td>Flume</td>
-    </tr>
-    
+<td> DRPC Server </td>
+<td> Flume</td></tr>
 <tr class="a">
-      
 <td>HBase Master </td>
-      
-<td>HBase Client </td>
-      
-<td>HBase Client</td>
-    </tr>
-    
+<td> HBase Client </td>
+<td> HBase Client</td></tr>
 <tr class="b">
-      
 <td>RegionServer </td>
-      
-<td>RegionServer </td>
-      
-<td>RegionServer</td>
-    </tr>
-    
+<td> RegionServer </td>
+<td> RegionServer</td></tr>
 <tr class="a">
-      
-<td>HCat Client </td>
-      
 <td>HCat Client </td>
-      
-<td>HCat Client</td>
-    </tr>
-    
+<td> HCat Client </td>
+<td> HCat Client</td></tr>
 <tr class="b">
-      
 <td>HDFS Client </td>
-      
-<td>HDFS Client </td>
-      
-<td>HDFS Client</td>
-    </tr>
-    
+<td> HDFS Client </td>
+<td> HDFS Client</td></tr>
 <tr class="a">
-      
 <td>Hive Client </td>
-      
-<td>History Server </td>
-      
-<td>Hive Client</td>
-    </tr>
-    
+<td> History Server </td>
+<td> Hive Client</td></tr>
 <tr class="b">
-      
 <td>Kafka Broker </td>
-      
-<td>Hive Client </td>
-      
-<td>Kafka Broker</td>
-    </tr>
-    
+<td> Hive Client </td>
+<td> Kafka Broker</td></tr>
 <tr class="a">
-      
 <td>Kibana Server </td>
-      
-<td>Hive Metastore </td>
-      
-<td>MapReduce2 Client</td>
-    </tr>
-    
+<td> Hive Metastore </td>
+<td> MapReduce2 Client</td></tr>
 <tr class="b">
-      
 <td>MapReduce2 Client </td>
-      
-<td>HiveServer2 </td>
-      
-<td>Metrics Collector</td>
-    </tr>
-    
+<td> HiveServer2 </td>
+<td> Metrics Collector</td></tr>
 <tr class="a">
-      
 <td>Grafana </td>
-      
-<td>Kafka Broker </td>
-      
-<td>Metrics Monitor</td>
-    </tr>
-    
+<td> Kafka Broker </td>
+<td> Metrics Monitor</td></tr>
 <tr class="b">
-      
 <td>Metrics Monitor </td>
-      
-<td>MapReduce2 Client </td>
-      
-<td>Metron Client</td>
-    </tr>
-    
+<td> MapReduce2 Client </td>
+<td> Metron Client</td></tr>
 <tr class="a">
-      
 <td>Metron Client </td>
-      
-<td>Metrics Monitor </td>
-      
-<td>NodeManager</td>
-    </tr>
-    
+<td> Metrics Monitor </td>
+<td> NodeManager</td></tr>
 <tr class="b">
-      
 <td>Metron Enrichment </td>
-      
-<td>Metron Client </td>
-      
-<td>Pig Client</td>
-    </tr>
-    
+<td> Metron Client </td>
+<td> Pig Client</td></tr>
 <tr class="a">
-      
 <td>Metron Indexing </td>
-      
-<td>MySQL Server </td>
-      
-<td>Slider Client</td>
-    </tr>
-    
+<td> MySQL Server </td>
+<td> Slider Client</td></tr>
 <tr class="b">
-      
 <td>Metron Parsers </td>
-      
-<td>Nimbus </td>
-      
-<td>Spark Client</td>
-    </tr>
-    
+<td> Nimbus </td>
+<td> Spark Client</td></tr>
 <tr class="a">
-      
 <td>Metron REST </td>
-      
-<td>NodeManager </td>
-      
-<td>Supervisor</td>
-    </tr>
-    
+<td> NodeManager </td>
+<td> Supervisor</td></tr>
 <tr class="b">
-      
 <td>NameNode </td>
-      
-<td>Pig Client </td>
-      
-<td>Tez Client</td>
-    </tr>
-    
+<td> Pig Client </td>
+<td> Tez Client</td></tr>
 <tr class="a">
-      
 <td>NodeManager </td>
-      
-<td>ResourceManager </td>
-      
-<td>YARN Client</td>
-    </tr>
-    
+<td> ResourceManager </td>
+<td> YARN Client</td></tr>
 <tr class="b">
-      
 <td>Pig Client </td>
-      
-<td>SNameNode </td>
-      
-<td>ZooKeeper Client</td>
-    </tr>
-    
+<td> SNameNode </td>
+<td> ZooKeeper Client</td></tr>
 <tr class="a">
-      
 <td>Slider Client </td>
-      
-<td>Slider Client </td>
-      
-<td>ZooKeeper Server</td>
-    </tr>
-    
+<td> Slider Client </td>
+<td> ZooKeeper Server</td></tr>
 <tr class="b">
-      
-<td>Spark Client </td>
-      
 <td>Spark Client </td>
-    </tr>
-    
+<td> Spark Client </td></tr>
 <tr class="a">
-      
 <td>Spark History Server </td>
-      
-<td>Supervisor </td>
-    </tr>
-    
+<td> Supervisor </td></tr>
 <tr class="b">
-      
 <td>Storm UI Server </td>
-      
-<td>Tez Client</td>
-    </tr>
-    
+<td> Tez Client</td></tr>
 <tr class="a">
-      
 <td>Supervisor </td>
-      
-<td>WebHCat Server</td>
-    </tr>
-    
+<td> WebHCat Server</td></tr>
 <tr class="b">
-      
 <td>Tez Client </td>
-      
-<td>YARN Client</td>
-    </tr>
-    
+<td> YARN Client</td></tr>
 <tr class="a">
-      
 <td>YARN Client </td>
-      
-<td>ZooKeeper Client</td>
-    </tr>
-    
+<td> ZooKeeper Client</td></tr>
 <tr class="b">
-      
 <td>Zeppelin Notebook </td>
-      
-<td>ZooKeeper Server</td>
-    </tr>
-    
+<td> ZooKeeper Server</td></tr>
 <tr class="a">
-      
-<td>ZooKeeper Client </td>
-    </tr>
-    
+<td>ZooKeeper Client </td></tr>
 <tr class="b">
-      
-<td>ZooKeeper Server </td>
-    </tr>
-  </tbody>
+<td>ZooKeeper Server </td></tr>
+</tbody>
 </table>
-
 <ul>
-  
+
 <li>
-<p>Install everything. Metron REST will probably not work as we still need to add a user and the database to MySQL. At this point, make sure that all the services are up. You might have to manually start a few.</p></li>
-  
+
+<p>Install everything. Metron REST will probably not work as we still need to add a user and the database to MySQL. At this point, make sure that all the services are up. You might have to manually start a few.</p>
+</li>
 <li>
-<p>Configure a user for Metron REST in MySQL. On the node where you installed the Metron REST UI, do:</p></li>
+
+<p>Configure a user for Metron REST in MySQL. On the node where you installed the Metron REST UI, do:</p>
+</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># mysql -u root -p
+<div>
+<div>
+<pre class="source"># mysql -u root -p
 CREATE USER '&lt;DB USERNAME&gt;'@'localhost' IDENTIFIED BY '&lt;DB PASSWORD&gt;';
 CREATE DATABASE IF NOT EXISTS &lt;DB NAME&gt;;
 GRANT ALL PRIVILEGES ON &lt;DB NAME&gt;.* TO '&lt;DB USERNAME&gt;'@'localhost';
 </pre></div></div>
+
 <p>For example:</p>
 
-<div class="source">
-<div class="source">
-<pre># mysql -u root -p
+<div>
+<div>
+<pre class="source"># mysql -u root -p
 &gt; CREATE USER 'metron'@'localhost' IDENTIFIED BY 'metron';
 &gt; CREATE DATABASE IF NOT EXISTS metronrest;
 &gt; GRANT ALL PRIVILEGES ON metronrest.* TO 'metron'@'localhost';
@@ -1184,11 +893,12 @@ GRANT ALL PRIVILEGES ON &lt;DB NAME&gt;.* TO '&lt;DB USERNAME&gt;'@'localhost';
 Bye
 # 
 </pre></div></div>
+
 <p>Add the Metron REST username and password to the metronrest database:</p>
 
-<div class="source">
-<div class="source">
-<pre># mysql -u &lt;DB USERNAME&gt; -p
+<div>
+<div>
+<pre class="source"># mysql -u &lt;DB USERNAME&gt; -p
 &gt; use &lt;DB NAME&gt;;
 &gt; insert into users (username, password, enabled) values ('&lt;USERNAME&gt;','&lt;PASSWORD&gt;',1);
 &gt; insert into authorities (username, authority) values ('&lt;USERNAME&gt;', 'ROLE_USER');
@@ -1196,11 +906,12 @@ Bye
 Bye
 #
 </pre></div></div>
+
 <p>For example, to use the username &#x2018;metron&#x2019; with password &#x2018;metron&#x2019;, do the following:</p>
 
-<div class="source">
-<div class="source">
-<pre># mysql -u metron -p
+<div>
+<div>
+<pre class="source"># mysql -u metron -p
 &gt; use metronrest;
 &gt; insert into users (username, password, enabled) values ('metron','metron',1);
 &gt; insert into authorities (username, authority) values ('metron', 'ROLE_USER');
@@ -1208,12 +919,13 @@ Bye
 Bye
 #
 </pre></div></div>
+
 <p>Make sure that all the services are up.</p>
 <p>Install metron_pcapservice:</p>
 
-<div class="source">
-<div class="source">
-<pre># cp /root/metron/metron-platform/metron-api/target/metron-api-0.4.0.jar /usr/metron/0.4.0/lib/
+<div>
+<div>
+<pre class="source"># cp /root/metron/metron-platform/metron-api/target/metron-api-0.4.0.jar /usr/metron/0.4.0/lib/
 # wget -O /etc/init.d/pcapservice https://raw.githubusercontent.com/apache/metron/master/metron-deployment/roles/metron_pcapservice/templates/pcapservice
 # sed -i 's/{{ pcapservice_jar_dst }}/\/usr\/metron\/0.4.0\/lib\/metron-api-0.4.0.jar/' /etc/init.d/pcapservice
 # sed -i 's/{{ pcapservice_port }}/8081/' /etc/init.d/pcapservice
@@ -1225,25 +937,28 @@ Bye
 # sed -i 's/^  rotate {{ metron_pcapservice_logrotate_retention }}.*$/  rotate 30/' /etc/logrotate.d/metron-pcapservice
 # chmod 644 /etc/logrotate.d/metron-pcapservice
 </pre></div></div>
+
 <p>Install tap interface:</p>
 
-<div class="source">
-<div class="source">
-<pre># yum install tunctl -y
+<div>
+<div>
+<pre class="source"># yum install tunctl -y
 # tunctl -p
 </pre></div></div>
+
 <p>Bring up tap0 on 10.0.0.100:</p>
 
-<div class="source">
-<div class="source">
-<pre># ifconfig tap0 10.0.0.100 up
+<div>
+<div>
+<pre class="source"># ifconfig tap0 10.0.0.100 up
 # ip link set tap0 promisc on
 </pre></div></div>
+
 <p>Install librdkafka:</p>
 
-<div class="source">
-<div class="source">
-<pre># yum install cmake make gcc gcc-c++ flex bison libpcap libpcap-devel openssl-devel python-devel swig zlib-devel perlcyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi -y
+<div>
+<div>
+<pre class="source"># yum install cmake make gcc gcc-c++ flex bison libpcap libpcap-devel openssl-devel python-devel swig zlib-devel perlcyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi -y
 # cd /tmp
 # wget -O /tmp/librdkafka-0.9.4.tar.gz https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz
 # /bin/gtar --extract -C /tmp -z -f /tmp/librdkafka-0.9.4.tar.gz
@@ -1252,11 +967,12 @@ Bye
 # make
 # make install
 </pre></div></div>
+
 <p>Install pycapa</p>
 
-<div class="source">
-<div class="source">
-<pre># yum install centos-release-scl -y
+<div>
+<div>
+<pre class="source"># yum install centos-release-scl -y
 # yum update -y
 # yum install python27 -y
 # scl enable python27 bash
@@ -1277,11 +993,12 @@ Bye
 # ln -s /usr/local/lib/librdkafka.so.1 /opt/rh/python27/root/usr/lib64
 # deactivate
 </pre></div></div>
+
 <p>Log out and log in to make sure Python is back to version 2.6 instead of 2.7.</p>
 
-<div class="source">
-<div class="source">
-<pre># wget -O /etc/init.d/pycapa https://raw.githubusercontent.com/apache/metron/master/metron-deployment/roles/pycapa/templates/pycapa
+<div>
+<div>
+<pre class="source"># wget -O /etc/init.d/pycapa https://raw.githubusercontent.com/apache/metron/master/metron-deployment/roles/pycapa/templates/pycapa
 # sed -i 's/{{ pycapa_log }}/\/var\/log\/pycapa.log/' /etc/init.d/pycapa
 # sed -i 's/{{ pycapa_home }}/\/usr\/local\/pycapa/' /etc/init.d/pycapa
 # sed -i 's/{{ python27_home }}/\/opt\/rh\/python27\/root/' /etc/init.d/pycapa
@@ -1293,47 +1010,52 @@ Bye
 # yum install @Development libdnet-devel rpm-build libpcap libpcap-devel pcre pcre-devel zlib zlib-devel glib2-devel -y
 # yum install kafka -y
 </pre></div></div>
+
 <p>Install bro:</p>
 
-<div class="source">
-<div class="source">
-<pre># wget -O /tmp/bro-2.4.2.tar.gz https://www.bro.org/downloads/bro-2.4.2.tar.gz
+<div>
+<div>
+<pre class="source"># wget -O /tmp/bro-2.4.2.tar.gz https://www.bro.org/downloads/bro-2.4.2.tar.gz
 # /bin/gtar --extract -C /tmp -z -f /tmp/bro-2.4.2.tar.gz
 # cd /tmp/bro-2.4.2
 # ./configure --prefix=/usr/local/bro
 # make -j4
 # make install
 </pre></div></div>
+
 <p>Configure bro:</p>
 
-<div class="source">
-<div class="source">
-<pre># sed -i 's/interface=eth0/interface=tap0/' /usr/local/bro/etc/node.cfg
+<div>
+<div>
+<pre class="source"># sed -i 's/interface=eth0/interface=tap0/' /usr/local/bro/etc/node.cfg
 # /usr/local/bro/bin/broctl install
 </pre></div></div>
+
 <p>Edit crontab with <tt># crontab -e</tt> and add:</p>
 
-<div class="source">
-<div class="source">
-<pre>0-59/5  *   *   *   *   /usr/local/bro/bin/broctl cron
+<div>
+<div>
+<pre class="source">0-59/5  *   *   *   *   /usr/local/bro/bin/broctl cron
 0-59/5  *   *   *   *   rm -rf /usr/local/bro/spool/tmp/*
 </pre></div></div>
+
 <p>bro-kafka:</p>
 
-<div class="source">
-<div class="source">
-<pre># git clone https://github.com/apache/metron-bro-plugin-kafka /tmp/metron-bro-plugin-kafka
+<div>
+<div>
+<pre class="source"># git clone https://github.com/apache/metron-bro-plugin-kafka /tmp/metron-bro-plugin-kafka
 # cd /tmp/metron-bro-plugin-kafka
 # rm -rf build/
 # ./configure --bro-dist=/tmp/bro-2.4.2 --install-root=/usr/local/bro/lib/bro/plugins/ --with-librdkafka=/usr/local
 # make -j4
 # make install
 </pre></div></div>
+
 <p>Configure metron-bro-plugin-kafka:</p>
 
-<div class="source">
-<div class="source">
-<pre># cat &lt;&lt; EOF &gt;&gt; /usr/local/bro/share/bro/site/local.bro
+<div>
+<div>
+<pre class="source"># cat &lt;&lt; EOF &gt;&gt; /usr/local/bro/share/bro/site/local.bro
 @load Bro/Kafka/logs-to-kafka.bro
 redef Kafka::logs_to_send = set(HTTP::LOG, DNS::LOG);
 redef Kafka::topic_name = &quot;bro&quot;;
@@ -1343,33 +1065,37 @@ EOF
 # /usr/local/bro/bin/broctl deploy
 # ip link set tap0 promisc on
 </pre></div></div>
+
 <p>Install daq:</p>
 
-<div class="source">
-<div class="source">
-<pre># wget -O /tmp/daq-2.0.6-1.src.rpm https://snort.org/downloads/snort/daq-2.0.6-1.src.rpm
+<div>
+<div>
+<pre class="source"># wget -O /tmp/daq-2.0.6-1.src.rpm https://snort.org/downloads/snort/daq-2.0.6-1.src.rpm
 # cd /tmp
 # rpmbuild --rebuild daq-2.0.6-1.src.rpm
 </pre></div></div>
+
 <p>This last command creates the files /root/rpmbuild/RPMS/x86_64/daq-2.0.6-1.x86_64.rpm &amp; /root/rpmbuild/RPMS/x86_64/daq-debuginfo-2.0.6-1.x86_64.rpm. We only need to install the first rpm.</p>
 
-<div class="source">
-<div class="source">
-<pre># yum install /root/rpmbuild/RPMS/x86_64/daq-2.0.6-1.x86_64.rpm -y
+<div>
+<div>
+<pre class="source"># yum install /root/rpmbuild/RPMS/x86_64/daq-2.0.6-1.x86_64.rpm -y
 </pre></div></div>
+
 <p>Install snort:</p>
 
-<div class="source">
-<div class="source">
-<pre># wget -O /tmp/snort-2.9.8.0-1.src.rpm https://snort.org/downloads/archive/snort/snort-2.9.8.0-1.src.rpm
+<div>
+<div>
+<pre class="source"># wget -O /tmp/snort-2.9.8.0-1.src.rpm https://snort.org/downloads/archive/snort/snort-2.9.8.0-1.src.rpm
 # cd /tmp
 # rpmbuild --rebuild snort-2.9.8.0-1.src.rpm
 </pre></div></div>
+
 <p>This last command creates the files /root/rpmbuild/RPMS/x86_64/snort-2.9.8.0-1.x86_64.rpm &amp; /root/rpmbuild/RPMS/x86_64/snort-debuginfo-2.9.8.0-1.x86_64.rpm. We only need to install the first rpm.</p>
 
-<div class="source">
-<div class="source">
-<pre># yum install /root/rpmbuild/RPMS/x86_64/snort-2.9.8.0-1.x86_64.rpm -y
+<div>
+<div>
+<pre class="source"># yum install /root/rpmbuild/RPMS/x86_64/snort-2.9.8.0-1.x86_64.rpm -y
 # wget -O /tmp/community-rules.tar.gz https://www.snort.org/downloads/community/community-rules.tar.gz
 # /bin/gtar --extract -C /tmp -z -f /tmp/community-rules.tar.gz
 # cp -r community-rules/community.rules /etc/snort/rules
@@ -1389,9 +1115,9 @@ EOF
 # chmod 755 /opt/snort-producer
 </pre></div></div>
 
-<div class="source">
-<div class="source">
-<pre># wget -O /opt/snort-producer/start-snort-producer.sh https://github.com/apache/metron/raw/master/metron-deployment/roles/snort/templates/start-snort-producer.sh
+<div>
+<div>
+<pre class="source"># wget -O /opt/snort-producer/start-snort-producer.sh https://github.com/apache/metron/raw/master/metron-deployment/roles/snort/templates/start-snort-producer.sh
 # sed -i 's/{{ snort_alert_csv_path }}/\/var\/log\/snort\/alert.csv/' /opt/snort-producer/start-snort-producer.sh
 # sed -i 's/{{ kafka_prod }}/\/usr\/hdp\/current\/kafka-broker\/bin\/kafka-console-producer.sh/' /opt/snort-producer/start-snort-producer.sh
 # sed -i 's/{{ kafka_broker_url }}/&lt;KAFKA_BROKER_IP&gt;:6667/' /opt/snort-producer/start-snort-producer.sh
@@ -1399,18 +1125,19 @@ EOF
 # chmod 755 /opt/snort-producer/start-snort-producer.sh
 </pre></div></div>
 
-<div class="source">
-<div class="source">
-<pre># wget -O /etc/init.d/snort-producer https://github.com/apache/metron/raw/master/metron-deployment/roles/snort/templates/snort-producer
+<div>
+<div>
+<pre class="source"># wget -O /etc/init.d/snort-producer https://github.com/apache/metron/raw/master/metron-deployment/roles/snort/templates/snort-producer
 # sed -i 's/{{ snort_producer_home }}/\/opt\/snort-producer/' /etc/init.d/snort-producer
 # sed -i 's/{{ snort_producer_start }}/\/opt\/snort-producer\/start-snort-producer.sh/' /etc/init.d/snort-producer
 # chmod 755 /etc/init.d/snort-producer
 </pre></div></div>
+
 <p>Install yaf:</p>
 
-<div class="source">
-<div class="source">
-<pre># wget -O /tmp/libfixbuf-1.7.1.tar.gz http://tools.netsa.cert.org/releases/libfixbuf-1.7.1.tar.gz
+<div>
+<div>
+<pre class="source"># wget -O /tmp/libfixbuf-1.7.1.tar.gz http://tools.netsa.cert.org/releases/libfixbuf-1.7.1.tar.gz
 # /bin/gtar --extract -C /tmp -z -f /tmp/libfixbuf-1.7.1.tar.gz
 # cd /tmp/libfixbuf-1.7.1
 # ./configure
@@ -1438,11 +1165,12 @@ EOF
 # sed -i 's/^DAEMONOPTS=\&quot;${@:2}\&quot;$/DAEMONOPTS=\&quot;${@:2} --idle-timeout 0\&quot;/' /etc/init.d/yaf
 # chmod 755 /etc/init.d/yaf
 </pre></div></div>
+
 <p>Install tcpreplay:</p>
 
-<div class="source">
-<div class="source">
-<pre># wget -O /tmp/tcpreplay-4.1.1.tar.gz https://github.com/appneta/tcpreplay/releases/download/v4.1.1/tcpreplay-4.1.1.tar.gz
+<div>
+<div>
+<pre class="source"># wget -O /tmp/tcpreplay-4.1.1.tar.gz https://github.com/appneta/tcpreplay/releases/download/v4.1.1/tcpreplay-4.1.1.tar.gz
 # /bin/gtar --extract -C /opt -z  -f /tmp/tcpreplay-4.1.1.tar.gz
 # cd /opt/tcpreplay-4.1.1/
 # ./configure --prefix=/opt
@@ -1461,11 +1189,12 @@ EOF
 # sed -i 's/{{ tcpreplay_prefix }}/\/opt/' /etc/init.d/pcap-replay
 # chmod 755 /etc/init.d/pcap-replay
 </pre></div></div>
+
 <p>Install monit</p>
 
-<div class="source">
-<div class="source">
-<pre># yum install monit -y
+<div>
+<div>
+<pre class="source"># yum install monit -y
 # wget -O /etc/monit.conf https://github.com/apache/metron/raw/master/metron-deployment/roles/monit/templates/monit/monit.conf
 
 # sed -i 's/{{ inventory_hostname }}/&lt;IP ADDRESS&gt;/' /etc/monit.conf
@@ -1500,184 +1229,177 @@ EOF
 # monit stop all
 # monit start all
 # monit summary | tail -n +3 | awk -F&quot;'&quot; '{print $2}'
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Miscellaneous_Issues"></a>Miscellaneous Issues</h3>
-
 <ul>
-  
+
 <li>
-<p>There&#x2019;s currently a bug in Metron 0.4.0 where Metron REST doesn&#x2019;t start when restarting the Metron services. This bug was fixed in METRON-990 (<a class="externalLink" href="https://github.com/apache/metron/pull/613/commits/1a9b19a0101ada58cb671ab224934f304df6fff8">https://github.com/apache/metron/pull/613/commits/1a9b19a0101ada58cb671ab224934f304df6fff8</a>) but unfortunately, this fix didn&#x2019;t make the 0.4.0 release. In order to fix this, edit the file &#x201c;/etc/rc.d/init.d/metron-rest&#x201d; and on line 148, change &#x201c;$0 start&#x201d; to &#x201c;$0 start $2&#x201d;.</p></li>
-  
+
+<p>There&#x2019;s currently a bug in Metron 0.4.0 where Metron REST doesn&#x2019;t start when restarting the Metron services. This bug was fixed in METRON-990 (<a class="externalLink" href="https://github.com/apache/metron/pull/613/commits/1a9b19a0101ada58cb671ab224934f304df6fff8">https://github.com/apache/metron/pull/613/commits/1a9b19a0101ada58cb671ab224934f304df6fff8</a>) but unfortunately, this fix didn&#x2019;t make the 0.4.0 release. In order to fix this, edit the file &#x201c;/etc/rc.d/init.d/metron-rest&#x201d; and on line 148, change &#x201c;$0 start&#x201d; to &#x201c;$0 start $2&#x201d;.</p>
+</li>
 <li>
-<p>I had a problem with Zeppelin after rebooting this machine and had to manually create the Zeppelin run directory:</p></li>
+
+<p>I had a problem with Zeppelin after rebooting this machine and had to manually create the Zeppelin run directory:</p>
+</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># mkdir /var/run/zeppelin
+<div>
+<div>
+<pre class="source"># mkdir /var/run/zeppelin
 # chown zeppelin.hadoop zeppelin/
 </pre></div></div>
 
 <ul>
-  
+
 <li>Additionally, while working with Metron, I&#x2019;ve noticed that at some point Zeppelin Notebook started, but immediately stopped again. In the logs, I could see &#x201c;Address already in use&#x201d; messages. It turns out that there was still a lingering Zeppelin process on the host. To fix it, stop Zeppelin Notebook in Ambari and then kill the latent process:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># ps aux | grep zeppelin
+<div>
+<div>
+<pre class="source"># ps aux | grep zeppelin
 # kill &lt;zeppelin_java_pid&gt;
 </pre></div></div>
-<p>Afterwards, restart Zeppelin Notebook via Ambari.</p>
 
+<p>Afterwards, restart Zeppelin Notebook via Ambari.</p>
 <ul>
-  
+
 <li>I had a couple of issues with Elasticsearch where it wouldn&#x2019;t find a master. This was fixed by doing the following. In Ambari, set the following items: &#x201c;masters_also_are_datanodes&#x201d; to &#x201c;true&#x201d; &#x201c;expected_data_nodes&#x201d; = &#x201c;0&#x201d; &#x201c;gateway_recover_after_data_nodes&#x201d; = &#x201c;1&#x201d; Restart all services. At this point, I noticed the following in :/etc/elasticsearch/elasticsearch.yml&quot;:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>node:
+<div>
+<div>
+<pre class="source">node:
   data: true
   master: true
   name: metron1.local
 </pre></div></div>
+
 <p>After changing this to :</p>
 
-<div class="source">
-<div class="source">
-<pre>node:
+<div>
+<div>
+<pre class="source">node:
   data: true
   master: true
   name: metron
 </pre></div></div>
-<p>and restarting elasticsearch with &#x201c;service elasticsearch restart&#x201d;, elasticsearch started indexing.</p>
 
+<p>and restarting elasticsearch with &#x201c;service elasticsearch restart&#x201d;, elasticsearch started indexing.</p>
 <ul>
-  
+
 <li>Another with Elasticsearch was that I saw the following error message in Kibana:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>plugin:elasticsearch     Elasticsearch is still initializing the kibana index.
+<div>
+<div>
+<pre class="source">plugin:elasticsearch     Elasticsearch is still initializing the kibana index.
 </pre></div></div>
+
 <p>This was fixed by deleting the Kibana index &#x201c;.kibana&#x201d;: <tt>curl -XDELETE http://localhost:9200/.kibana</tt></p></div>
 <div class="section">
 <h3><a name="Miscellaneous_Services"></a>Miscellaneous Services</h3>
-
 <ul>
-  
+
 <li>Load the correct Elasticsearch template with:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>curl -s -w &quot;%{http_code}&quot; -u &lt;USERNAME&gt;:&lt;PASSWORD&gt; -H &quot;X-Requested-By: ambari&quot; -X POST -d '{ &quot;RequestInfo&quot;: { &quot;context&quot;: &quot;Install ES Template from REST&quot;, &quot;command&quot;: &quot;ELASTICSEARCH_TEMPLATE_INSTALL&quot;},&quot;Requests/resource_filters&quot;: [{&quot;service_name&quot;: &quot;METRON&quot;,&quot;component_name&quot;: &quot;METRON_INDEXING&quot;,&quot;hosts&quot; : &quot;&lt;HOSTNAME&gt;&quot;}]}' http://&lt;AMBARI HOST&gt;:8080/api/v1/clusters/&lt;CLUSTERNAME&gt;/requests
+<div>
+<div>
+<pre class="source">curl -s -w &quot;%{http_code}&quot; -u &lt;USERNAME&gt;:&lt;PASSWORD&gt; -H &quot;X-Requested-By: ambari&quot; -X POST -d '{ &quot;RequestInfo&quot;: { &quot;context&quot;: &quot;Install ES Template from REST&quot;, &quot;command&quot;: &quot;ELASTICSEARCH_TEMPLATE_INSTALL&quot;},&quot;Requests/resource_filters&quot;: [{&quot;service_name&quot;: &quot;METRON&quot;,&quot;component_name&quot;: &quot;METRON_INDEXING&quot;,&quot;hosts&quot; : &quot;&lt;HOSTNAME&gt;&quot;}]}' http://&lt;AMBARI HOST&gt;:8080/api/v1/clusters/&lt;CLUSTERNAME&gt;/requests
 </pre></div></div>
+
 <p>For example:</p>
 
-<div class="source">
-<div class="source">
-<pre>curl -s -w &quot;%{http_code}&quot; -u admin:admin -H &quot;X-Requested-By: ambari&quot; -X POST -d '{ &quot;RequestInfo&quot;: { &quot;context&quot;: &quot;Install ES Template from REST&quot;, &quot;command&quot;: &quot;ELASTICSEARCH_TEMPLATE_INSTALL&quot;},&quot;Requests/resource_filters&quot;: [{&quot;service_name&quot;: &quot;METRON&quot;,&quot;component_name&quot;: &quot;METRON_INDEXING&quot;,&quot;hosts&quot; : &quot;metron&quot;}]}' http://192.168.10.10:8080/api/v1/clusters/metron/requests
+<div>
+<div>
+<pre class="source">curl -s -w &quot;%{http_code}&quot; -u admin:admin -H &quot;X-Requested-By: ambari&quot; -X POST -d '{ &quot;RequestInfo&quot;: { &quot;context&quot;: &quot;Install ES Template from REST&quot;, &quot;command&quot;: &quot;ELASTICSEARCH_TEMPLATE_INSTALL&quot;},&quot;Requests/resource_filters&quot;: [{&quot;service_name&quot;: &quot;METRON&quot;,&quot;component_name&quot;: &quot;METRON_INDEXING&quot;,&quot;hosts&quot; : &quot;metron&quot;}]}' http://192.168.10.10:8080/api/v1/clusters/metron/requests
 </pre></div></div>
 
 <ul>
-  
+
 <li>Load Kibana Dashboard with:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>curl -s -w &quot;%{http_code}&quot; -u &lt;USERNAME&gt;:&lt;PASSWORD&gt; -H &quot;X-Requested-By: ambari&quot; -X POST -d '{ &quot;RequestInfo&quot;: { &quot;context&quot;: &quot;Install Kibana Dashboard from REST&quot;, &quot;command&quot;: &quot;LOAD_TEMPLATE&quot;},&quot;Requests/resource_filters&quot;: [{&quot;service_name&quot;: &quot;KIBANA&quot;,&quot;component_name&quot;: &quot;KIBANA_MASTER&quot;,&quot;hosts&quot; : &quot;&lt;HOSTNAME&gt;&quot;}]}' http://&lt;AMBARI HOST&gt;:8080/api/v1/clusters/&lt;CLUSTERNAME&gt;/requests
+<div>
+<div>
+<pre class="source">curl -s -w &quot;%{http_code}&quot; -u &lt;USERNAME&gt;:&lt;PASSWORD&gt; -H &quot;X-Requested-By: ambari&quot; -X POST -d '{ &quot;RequestInfo&quot;: { &quot;context&quot;: &quot;Install Kibana Dashboard from REST&quot;, &quot;command&quot;: &quot;KIBANA_DASHBOARD_INSTALL&quot;},&quot;Requests/resource_filters&quot;: [{&quot;service_name&quot;: &quot;METRON&quot;,&quot;component_name&quot;: &quot;METRON_INDEXING&quot;,&quot;hosts&quot; : &quot;&lt;HOSTNAME&gt;&quot;}]}' http://&lt;AMBARI HOST&gt;:8080/api/v1/clusters/&lt;CLUSTERNAME&gt;/requests
 </pre></div></div>
+
 <p>For example:</p>
 
-<div class="source">
-<div class="source">
-<pre>curl -s -w &quot;%{http_code}&quot; -u admin:admin -H &quot;X-Requested-By: ambari&quot; -X POST -d '{ &quot;RequestInfo&quot;: { &quot;context&quot;: &quot;Install Kibana Dashboard from REST&quot;, &quot;command&quot;: &quot;LOAD_TEMPLATE&quot;},&quot;Requests/resource_filters&quot;: [{&quot;service_name&quot;: &quot;KIBANA&quot;,&quot;component_name&quot;: &quot;KIBANA_MASTER&quot;,&quot;hosts&quot; : &quot;metron&quot;}]}' http://192.168.10.10:8080/api/v1/clusters/metron/requests
+<div>
+<div>
+<pre class="source">curl -s -w &quot;%{http_code}&quot; -u admin:admin -H &quot;X-Requested-By: ambari&quot; -X POST -d '{ &quot;RequestInfo&quot;: { &quot;context&quot;: &quot;Install Kibana Dashboard from REST&quot;, &quot;command&quot;: &quot;KIBANA_DASHBOARD_INSTALL&quot;},&quot;Requests/resource_filters&quot;: [{&quot;service_name&quot;: &quot;METRON&quot;,&quot;component_name&quot;: &quot;METRON_INDEXING&quot;,&quot;hosts&quot; : &quot;metron&quot;}]}' http://192.168.10.10:8080/api/v1/clusters/metron/requests
 </pre></div></div>
 
 <ul>
-  
+
 <li>If you installed Metron on a single node, you might have to increase the number of Storm supervisor slots from the default 2 to 5 or more. This can be done by editing the &#x201c;supervisor.slots.ports&#x201d; under Storm in the Ambari UI. Change:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>supervisor.slots.ports: [6700, 6701]
+<div>
+<div>
+<pre class="source">supervisor.slots.ports: [6700, 6701]
 </pre></div></div>
+
 <p>To:</p>
 
-<div class="source">
-<div class="source">
-<pre>supervisor.slots.ports: [6700, 6701, 6702, 6703, 6704, 6705]
+<div>
+<div>
+<pre class="source">supervisor.slots.ports: [6700, 6701, 6702, 6703, 6704, 6705]
 </pre></div></div>
 
 <ul>
-  
+
 <li>Install Apache NiFi in /root (You can pretty much use any directory you want). Download nifi-1.2.0-bin.tar.gz from <a class="externalLink" href="https://nifi.apache.org/download.html">https://nifi.apache.org/download.html</a></li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre># cd /root
+<div>
+<div>
+<pre class="source"># cd /root
 # wget http://apache.mirror.iweb.ca/nifi/1.2.0/nifi-1.2.0-bin.tar.gz
 # tar xf nifi-1.2.0-bin.tar.gz
 </pre></div></div>
+
 <p>Before we run NiFi, we need to change the port as the default port collides with the Ambari port. To do this, we need to change the value &#x201c;nifi.web.http.port=8080&#x201d; to &#x201c;nifi.web.http.port=8089&#x201d; in the file &#x201c;nifi-1.1.2/conf/nifi.properties&#x201d;. Install and start NiFi afterwards:</p>
 
-<div class="source">
-<div class="source">
-<pre># nifi-1.2.0/bin/nifi.sh install
+<div>
+<div>
+<pre class="source"># nifi-1.2.0/bin/nifi.sh install
 # nifi-1.2.0/bin/nifi.sh start
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Exposed_Interfaces"></a>Exposed Interfaces</h3>
 <p>In the end, you&#x2019;ll end up with a bunch of exposed UIs:</p>
-
 <ul>
-  
+
 <li>Ambari: <a class="externalLink" href="http://node1:8080/">http://node1:8080/</a></li>
-  
 <li>Kibana: <a class="externalLink" href="http://node1:5000/">http://node1:5000/</a></li>
-  
 <li>Sensor Status (monit): <a class="externalLink" href="http://node1:2812">http://node1:2812</a></li>
-  
 <li>Elasticsearch: <a class="externalLink" href="http://node1:9200/">http://node1:9200/</a></li>
-  
 <li>Storm UI: <a class="externalLink" href="http://node1:8744/">http://node1:8744/</a></li>
-  
 <li>Metron REST interface: <a class="externalLink" href="http://node1:8082/swagger-ui.html#/">http://node1:8082/swagger-ui.html#/</a></li>
-  
 <li>Management UI: <a class="externalLink" href="http://node1:4200/">http://node1:4200/</a> (user/password)</li>
-  
 <li>Apache Nifi: <a class="externalLink" href="http://node1:8089/nifi/">http://node1:8089/nifi/</a></li>
-  
 <li>Zookeeper: <a class="externalLink" href="http://node1:2181">http://node1:2181</a></li>
-  
 <li>Kafka: <a class="externalLink" href="http://node1:6667">http://node1:6667</a></li>
 </ul></div>
 <div class="section">
 <h3><a name="TROUBLESHOOTING"></a>TROUBLESHOOTING</h3></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/packaging/ambari/elasticsearch-mpack/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/packaging/ambari/elasticsearch-mpack/index.html b/site/current-book/metron-deployment/packaging/ambari/elasticsearch-mpack/index.html
new file mode 100644
index 0000000..499e89b
--- /dev/null
+++ b/site/current-book/metron-deployment/packaging/ambari/elasticsearch-mpack/index.html
@@ -0,0 +1,190 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/ambari/elasticsearch-mpack/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; </title>
+    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../../css/site.css" />
+    <link rel="stylesheet" href="../../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active "></li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-down"></span>Ambari</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>Elasticsearch-mpack</a></li>
+    <li><a href="../../../../metron-deployment/packaging/ambari/metron-mpack/index.html" title="Mpack"><span class="none"></span>Mpack</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<p>This provides a Management Pack (MPack) extension for <a class="externalLink" href="https://ambari.apache.org/">Apache Ambari</a> that simplifies the provisioning, management and monitoring of Elasticsearch and Kibana on clusters of any size.</p>
+<p>This allows you to easily install Elasticsearch and Kibana using a simple, guided process.  This also allows you to monitor cluster health.</p>
+<div class="section">
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<ul>
+
+<li>Ambari 2.4.2+</li>
+</ul></div>
+<div class="section">
+<h3><a name="Quick_Start"></a>Quick Start</h3>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Build the Elasticsearch MPack. Execute the following command from the project&#x2019;s root directory.</p>
+
+<div>
+<div>
+<pre class="source">mvn clean package -Pmpack -DskipTests
+</pre></div></div>
+</li>
+<li>
+
+<p>This results in the Mpack being produced at the following location.</p>
+
+<div>
+<div>
+<pre class="source">metron-deployment/packaging/ambari/elasticsearch-mpack/target/elasticsearch_mpack-x.y.z.0.tar.gz
+</pre></div></div>
+</li>
+<li>
+
+<p>Copy the tarball to the host where Ambari Server is installed.</p>
+</li>
+<li>
+
+<p>Ensure that Ambari Server is stopped.</p>
+</li>
+<li>
+
+<p>Install the MPack.</p>
+
+<div>
+<div>
+<pre class="source">ambari-server install-mpack --mpack=elasticsearch_mpack-x.y.z.0.tar.gz --verbose
+</pre></div></div>
+</li>
+<li>
+
+<p>Elasticsearch and Kibana will now be available as an installable service within Ambari.</p>
+</li>
+</ol></div>
+<div class="section">
+<h3><a name="Installation_Notes"></a>Installation Notes</h3>
+<p>The MPack will make all Elasticsearch services available in Ambari in the same manner as any other services in a stack.  These can be installed using Ambari&#x2019;s user interface using &#x201c;Add Services&#x201d; or during an initial cluster install.</p>
+<div class="section">
+<h4><a name="Kerberization"></a>Kerberization</h4>
+<p>Elasticsearch does not provide free native Kerberos support.</p></div>
+<div class="section">
+<h4><a name="Limitations"></a>Limitations</h4>
+<p>There are a few limitations that should be addressed to improve the Elasticsearch MPack installation.</p>
+<ul>
+
+<li>The MPack does not support upgrades.</li>
+</ul></div></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>


[49/50] [abbrv] metron git commit: METRON-1608 Add configuration for threat.triage.field name (merrimanr) closes apache/metron#1055

Posted by mm...@apache.org.
METRON-1608 Add configuration for threat.triage.field name (merrimanr) closes apache/metron#1055


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/32d132b3
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/32d132b3
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/32d132b3

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: 32d132b3f8ac7d915180882934d38504263d86ae
Parents: 8202cd2
Author: merrimanr <me...@gmail.com>
Authored: Mon Jun 11 08:31:51 2018 -0500
Committer: merrimanr <me...@apache.org>
Committed: Mon Jun 11 08:31:51 2018 -0500

----------------------------------------------------------------------
 .../rest/service/impl/SearchServiceImpl.java    | 10 +--
 .../service/impl/SearchServiceImplTest.java     |  3 +-
 .../org/apache/metron/common/Constants.java     |  1 +
 .../configuration/ConfigurationsUtils.java      |  7 ++
 .../elasticsearch/dao/ElasticsearchDao.java     |  8 ++
 .../dao/ElasticsearchMetaAlertDao.java          | 78 ++++++++++----------
 .../dao/ElasticsearchMetaAlertDaoTest.java      | 33 ++++++++-
 .../ElasticsearchMetaAlertIntegrationTest.java  | 64 ++++++++++++++--
 .../metron/indexing/dao/MetaAlertDao.java       |  3 +-
 9 files changed, 150 insertions(+), 57 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/32d132b3/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/SearchServiceImpl.java
----------------------------------------------------------------------
diff --git a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/SearchServiceImpl.java b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/SearchServiceImpl.java
index 21d158f..1c92fcb 100644
--- a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/SearchServiceImpl.java
+++ b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/SearchServiceImpl.java
@@ -18,6 +18,7 @@
 package org.apache.metron.rest.service.impl;
 
 import static org.apache.metron.common.Constants.ERROR_TYPE;
+import static org.apache.metron.common.Constants.SENSOR_TYPE_FIELD_PROPERTY;
 import static org.apache.metron.indexing.dao.MetaAlertDao.METAALERT_TYPE;
 import static org.apache.metron.rest.MetronRestConstants.INDEX_WRITER_NAME;
 import static org.apache.metron.rest.MetronRestConstants.SEARCH_FACET_FIELDS_SPRING_PROPERTY;
@@ -31,6 +32,7 @@ import java.util.List;
 import java.util.Map;
 import java.util.Optional;
 import org.apache.metron.common.Constants;
+import org.apache.metron.common.configuration.ConfigurationsUtils;
 import org.apache.metron.indexing.dao.IndexDao;
 import org.apache.metron.indexing.dao.search.FieldType;
 import org.apache.metron.indexing.dao.search.GetRequest;
@@ -146,13 +148,9 @@ public class SearchServiceImpl implements SearchService {
     if (!alertUserSettings.isPresent() || alertUserSettings.get().getFacetFields() == null) {
       String facetFieldsProperty = environment
           .getProperty(SEARCH_FACET_FIELDS_SPRING_PROPERTY, String.class, "");
-
-      Map<String, Object> globalConfig = globalConfigService.get();
-      String sourceTypeField = Constants.SENSOR_TYPE.replace('.', ':');
+      String sourceTypeField = ConfigurationsUtils.getFieldName(globalConfigService.get(), SENSOR_TYPE_FIELD_PROPERTY,
+              Constants.SENSOR_TYPE.replace('.', ':'));
       List<String> facetFields = new ArrayList<>();
-      if (globalConfig != null) {
-        sourceTypeField = (String) globalConfig.getOrDefault("source.type.field", sourceTypeField);
-      }
       facetFields.add(sourceTypeField);
       if (facetFieldsProperty != null) {
         facetFields.addAll(Arrays.asList(facetFieldsProperty.split(",")));

http://git-wip-us.apache.org/repos/asf/metron/blob/32d132b3/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/service/impl/SearchServiceImplTest.java
----------------------------------------------------------------------
diff --git a/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/service/impl/SearchServiceImplTest.java b/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/service/impl/SearchServiceImplTest.java
index 4c63c6d..82e7221 100644
--- a/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/service/impl/SearchServiceImplTest.java
+++ b/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/service/impl/SearchServiceImplTest.java
@@ -17,6 +17,7 @@
  */
 package org.apache.metron.rest.service.impl;
 
+import static org.apache.metron.common.Constants.SENSOR_TYPE_FIELD_PROPERTY;
 import static org.apache.metron.rest.MetronRestConstants.INDEX_WRITER_NAME;
 import static org.apache.metron.rest.MetronRestConstants.SEARCH_FACET_FIELDS_SPRING_PROPERTY;
 import static org.junit.Assert.assertEquals;
@@ -183,7 +184,7 @@ public class SearchServiceImplTest {
     when(environment.getProperty(SEARCH_FACET_FIELDS_SPRING_PROPERTY, String.class, ""))
         .thenReturn("ip_src_addr");
     Map<String, Object> globalConfig = new HashMap<>();
-    globalConfig.put("source.type.field", "source.type");
+    globalConfig.put(SENSOR_TYPE_FIELD_PROPERTY, "source.type");
     when(globalConfigService.get()).thenReturn(globalConfig);
     when(alertsUIService.getAlertsUIUserSettings()).thenReturn(Optional.empty());
     List<String> defaultFields = searchService.getDefaultFacetFields();

http://git-wip-us.apache.org/repos/asf/metron/blob/32d132b3/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
index 12b541c..f74660c 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
@@ -27,6 +27,7 @@ public class Constants {
   public static final String ZOOKEEPER_TOPOLOGY_ROOT = ZOOKEEPER_ROOT + "/topology";
   public static final long DEFAULT_CONFIGURED_BOLT_TIMEOUT = 5000;
   public static final String SENSOR_TYPE = "source.type";
+  public static final String SENSOR_TYPE_FIELD_PROPERTY = "source.type.field";
   public static final String ENRICHMENT_TOPIC = "enrichments";
   public static final String INDEXING_TOPIC = "indexing";
   public static final String ERROR_STREAM = "error";

http://git-wip-us.apache.org/repos/asf/metron/blob/32d132b3/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
index c7b39f0..4550a41 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
@@ -799,5 +799,12 @@ public class ConfigurationsUtils {
       out.println(type + " Config: " + name + System.lineSeparator() + data);
     }, configType, configName);
   }
+
+  public static String getFieldName(Map<String, Object> globalConfig, String globalConfigKey, String defaultFieldName) {
+    if (globalConfig == null) {
+      return defaultFieldName;
+    }
+    return (String) globalConfig.getOrDefault(globalConfigKey, defaultFieldName);
+  }
 }
 

http://git-wip-us.apache.org/repos/asf/metron/blob/32d132b3/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
index cb5bb58..181cb87 100644
--- a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
+++ b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
@@ -125,6 +125,14 @@ public class ElasticsearchDao implements IndexDao {
     //uninitialized.
   }
 
+  public AccessConfig getAccessConfig() {
+    return accessConfig;
+  }
+
+  public void setAccessConfig(AccessConfig accessConfig) {
+    this.accessConfig = accessConfig;
+  }
+
   private static Map<String, FieldType> elasticsearchSearchTypeMap;
 
   static {

http://git-wip-us.apache.org/repos/asf/metron/blob/32d132b3/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
index d12e40c..ee3ca89 100644
--- a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
+++ b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
@@ -18,21 +18,9 @@
 
 package org.apache.metron.elasticsearch.dao;
 
-import static org.apache.metron.common.Constants.GUID;
-import static org.elasticsearch.index.query.QueryBuilders.boolQuery;
-import static org.elasticsearch.index.query.QueryBuilders.constantScoreQuery;
-import static org.elasticsearch.index.query.QueryBuilders.existsQuery;
-import static org.elasticsearch.index.query.QueryBuilders.nestedQuery;
-import static org.elasticsearch.index.query.QueryBuilders.termQuery;
-
-import com.fasterxml.jackson.databind.JsonNode;
-import java.io.IOException;
-import java.util.*;
-import java.util.Map.Entry;
-import java.util.stream.Collectors;
-import org.apache.commons.collections4.SetUtils;
 import org.apache.lucene.search.join.ScoreMode;
 import org.apache.metron.common.Constants;
+import org.apache.metron.common.configuration.ConfigurationsUtils;
 import org.apache.metron.indexing.dao.AccessConfig;
 import org.apache.metron.indexing.dao.IndexDao;
 import org.apache.metron.indexing.dao.MetaAlertDao;
@@ -51,40 +39,48 @@ import org.apache.metron.indexing.dao.search.SearchRequest;
 import org.apache.metron.indexing.dao.search.SearchResponse;
 import org.apache.metron.indexing.dao.search.SearchResult;
 import org.apache.metron.indexing.dao.update.Document;
-import org.elasticsearch.action.get.GetResponse;
-import org.elasticsearch.action.get.MultiGetItemResponse;
-import org.elasticsearch.action.get.MultiGetRequest.Item;
-import org.elasticsearch.action.get.MultiGetRequestBuilder;
-import org.elasticsearch.action.get.MultiGetResponse;
-import org.elasticsearch.action.index.IndexRequest;
-import org.elasticsearch.action.search.SearchRequestBuilder;
-import org.elasticsearch.action.support.replication.ReplicationResponse.ShardInfo;
-import org.elasticsearch.action.update.UpdateRequest;
-import org.elasticsearch.action.update.UpdateResponse;
-import org.elasticsearch.common.xcontent.XContentBuilder;
-import org.elasticsearch.index.query.InnerHitBuilder;
-import org.elasticsearch.index.query.QueryBuilder;
-import org.elasticsearch.index.query.QueryBuilders;
-import org.elasticsearch.index.query.QueryStringQueryBuilder;
-import org.elasticsearch.search.SearchHit;
 import org.apache.metron.indexing.dao.update.OriginalNotFoundException;
 import org.apache.metron.indexing.dao.update.PatchRequest;
 import org.apache.metron.stellar.common.utils.ConversionUtils;
 import org.elasticsearch.action.search.SearchRequestBuilder;
+import org.elasticsearch.index.query.InnerHitBuilder;
 import org.elasticsearch.index.query.QueryBuilder;
 import org.elasticsearch.index.query.QueryBuilders;
 import org.elasticsearch.index.query.QueryStringQueryBuilder;
+import org.elasticsearch.search.SearchHit;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+import static org.apache.metron.common.Constants.GUID;
+import static org.apache.metron.common.Constants.SENSOR_TYPE_FIELD_PROPERTY;
+import static org.elasticsearch.index.query.QueryBuilders.boolQuery;
+import static org.elasticsearch.index.query.QueryBuilders.constantScoreQuery;
+import static org.elasticsearch.index.query.QueryBuilders.existsQuery;
+import static org.elasticsearch.index.query.QueryBuilders.nestedQuery;
+import static org.elasticsearch.index.query.QueryBuilders.termQuery;
 
 public class ElasticsearchMetaAlertDao implements MetaAlertDao {
 
   public static final String SOURCE_TYPE = Constants.SENSOR_TYPE.replace('.', ':');
+  public static final String THREAT_TRIAGE_FIELD = THREAT_FIELD_DEFAULT.replace('.', ':');
   private static final String STATUS_PATH = "/status";
   private static final String ALERT_PATH = "/alert";
 
   private IndexDao indexDao;
   private ElasticsearchDao elasticsearchDao;
   private String index = METAALERTS_INDEX;
-  private String threatTriageField = THREAT_FIELD_DEFAULT;
 
   /**
    * Defines which summary aggregation is used to represent the overall threat triage score for
@@ -101,21 +97,19 @@ public class ElasticsearchMetaAlertDao implements MetaAlertDao {
    * @param indexDao The Dao to wrap
    */
   public ElasticsearchMetaAlertDao(IndexDao indexDao) {
-    this(indexDao, METAALERTS_INDEX, THREAT_FIELD_DEFAULT, THREAT_SORT_DEFAULT);
+    this(indexDao, METAALERTS_INDEX, THREAT_SORT_DEFAULT);
   }
 
   /**
    * Wraps an {@link org.apache.metron.indexing.dao.IndexDao} to handle meta alerts.
    * @param indexDao The Dao to wrap
-   * @param triageLevelField The field name to use as the threat scoring field
    * @param threatSort The summary aggregation of all child threat triage scores used
    *                   as the overall threat triage score for the metaalert. This
    *                   can be either max, min, average, count, median, or sum.
    */
-  public ElasticsearchMetaAlertDao(IndexDao indexDao, String index, String triageLevelField, String threatSort) {
+  public ElasticsearchMetaAlertDao(IndexDao indexDao, String index, String threatSort) {
     init(indexDao, Optional.of(threatSort));
     this.index = index;
-    this.threatTriageField = triageLevelField;
   }
 
   public ElasticsearchMetaAlertDao() {
@@ -196,7 +190,7 @@ public class ElasticsearchMetaAlertDao implements MetaAlertDao {
     Document metaAlert = buildCreateDocument(alerts, request.getGroups());
     calculateMetaScores(metaAlert);
     // Add source type to be consistent with other sources and allow filtering
-    metaAlert.getDocument().put(SOURCE_TYPE, MetaAlertDao.METAALERT_TYPE);
+    metaAlert.getDocument().put(getFieldName(SENSOR_TYPE_FIELD_PROPERTY, SOURCE_TYPE), MetaAlertDao.METAALERT_TYPE);
 
     // Start a list of updates / inserts we need to run
     Map<Document, Optional<String>> updates = new HashMap<>();
@@ -353,7 +347,7 @@ public class ElasticsearchMetaAlertDao implements MetaAlertDao {
       List<Map<String, Object>> currentAlerts = (List<Map<String, Object>>) metaAlert.getDocument()
           .get(MetaAlertDao.ALERT_FIELD);
       currentAlerts.stream().forEach(currentAlert -> {
-        getRequests.add(new GetRequest((String) currentAlert.get(GUID), (String) currentAlert.get(SOURCE_TYPE)));
+        getRequests.add(new GetRequest((String) currentAlert.get(GUID), (String) currentAlert.get(getFieldName(SENSOR_TYPE_FIELD_PROPERTY, SOURCE_TYPE))));
       });
       Iterable<Document> alerts = indexDao.getAllLatest(getRequests);
       List<Map<String, Object>> updatedAlerts = new ArrayList<>();
@@ -685,7 +679,7 @@ public class ElasticsearchMetaAlertDao implements MetaAlertDao {
       ArrayList<Double> scores = new ArrayList<>();
       for (Object alertRaw : alertsRaw) {
         Map<String, Object> alert = (Map<String, Object>) alertRaw;
-        Double scoreNum = parseThreatField(alert.get(threatTriageField));
+        Double scoreNum = parseThreatField(alert.get(getFieldName(THREAT_FIELD_PROPERTY, THREAT_TRIAGE_FIELD)));
         if (scoreNum != null) {
           scores.add(scoreNum);
         }
@@ -700,7 +694,7 @@ public class ElasticsearchMetaAlertDao implements MetaAlertDao {
     Object threatScore = metaScores.getMetaScores().get(threatSort);
 
     // add the threat score as a float; type needs to match the threat score field from each of the sensor indices
-    metaAlert.getDocument().put(threatTriageField, ConversionUtils.convert(threatScore, Float.class));
+    metaAlert.getDocument().put(getFieldName(THREAT_FIELD_PROPERTY, THREAT_TRIAGE_FIELD), ConversionUtils.convert(threatScore, Float.class));
   }
 
   private Double parseThreatField(Object threatRaw) {
@@ -720,4 +714,12 @@ public class ElasticsearchMetaAlertDao implements MetaAlertDao {
   public void setPageSize(int pageSize) {
     this.pageSize = pageSize;
   }
+
+  private String getFieldName(String globalConfigKey, String defaultFieldName) {
+    if (this.elasticsearchDao == null || this.elasticsearchDao.getAccessConfig() == null) {
+      return defaultFieldName;
+    }
+    Map<String, Object> globalConfig = this.elasticsearchDao.getAccessConfig().getGlobalConfigSupplier().get();
+    return ConfigurationsUtils.getFieldName(globalConfig, globalConfigKey, defaultFieldName);
+  }
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/32d132b3/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDaoTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDaoTest.java b/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDaoTest.java
index 1bfa9d6..44defb3 100644
--- a/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDaoTest.java
+++ b/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDaoTest.java
@@ -214,10 +214,10 @@ public class ElasticsearchMetaAlertDaoTest {
     List<Map<String, Object>> alertList = new ArrayList<>();
 
     // add an alert with a threat score
-    alertList.add( Collections.singletonMap(MetaAlertDao.THREAT_FIELD_DEFAULT, 10.0f));
+    alertList.add( Collections.singletonMap(ElasticsearchMetaAlertDao.THREAT_TRIAGE_FIELD, 10.0f));
 
     // add a second alert with a threat score
-    alertList.add( Collections.singletonMap(MetaAlertDao.THREAT_FIELD_DEFAULT, 20.0f));
+    alertList.add( Collections.singletonMap(ElasticsearchMetaAlertDao.THREAT_TRIAGE_FIELD, 20.0f));
 
     // add a third alert with NO threat score
     alertList.add( Collections.singletonMap("alert3", "has no threat score"));
@@ -230,7 +230,7 @@ public class ElasticsearchMetaAlertDaoTest {
     // calculate the threat score for the metaalert
     ElasticsearchMetaAlertDao metaAlertDao = new ElasticsearchMetaAlertDao();
     metaAlertDao.calculateMetaScores(metaalert);
-    Object threatScore = metaalert.getDocument().get(ElasticsearchMetaAlertDao.THREAT_FIELD_DEFAULT);
+    Object threatScore = metaalert.getDocument().get(ElasticsearchMetaAlertDao.THREAT_TRIAGE_FIELD);
 
     // the metaalert must contain a summary of all child threat scores
     assertEquals(20D, (Double) metaalert.getDocument().get("max"), delta);
@@ -246,4 +246,31 @@ public class ElasticsearchMetaAlertDaoTest {
     // by default, the overall threat score is the sum of all child threat scores
     assertEquals(30.0F, threatScore);
   }
+
+  @Test
+  public void testCalculateMetaScoresWithDifferentFieldName() {
+    List<Map<String, Object>> alertList = new ArrayList<>();
+
+    // add an alert with a threat score
+    alertList.add( Collections.singletonMap(MetaAlertDao.THREAT_FIELD_DEFAULT, 10.0f));
+
+    // create the metaalert
+    Map<String, Object> docMap = new HashMap<>();
+    docMap.put(MetaAlertDao.ALERT_FIELD, alertList);
+    Document metaalert = new Document(docMap, "guid", MetaAlertDao.METAALERT_TYPE, 0L);
+
+    // Configure a different threat triage score field name
+    AccessConfig accessConfig = new AccessConfig();
+    accessConfig.setGlobalConfigSupplier(() -> new HashMap<String, Object>() {{
+      put(MetaAlertDao.THREAT_FIELD_PROPERTY, MetaAlertDao.THREAT_FIELD_DEFAULT);
+    }});
+    ElasticsearchDao elasticsearchDao = new ElasticsearchDao();
+    elasticsearchDao.setAccessConfig(accessConfig);
+
+    // calculate the threat score for the metaalert
+    ElasticsearchMetaAlertDao metaAlertDao = new ElasticsearchMetaAlertDao();
+    metaAlertDao.init(elasticsearchDao);
+    metaAlertDao.calculateMetaScores(metaalert);
+    assertNotNull(metaalert.getDocument().get(MetaAlertDao.THREAT_FIELD_DEFAULT));
+  }
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/32d132b3/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java b/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
index b001050..adc1a27 100644
--- a/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
+++ b/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
@@ -18,11 +18,14 @@
 
 package org.apache.metron.elasticsearch.integration;
 
+import static org.apache.metron.common.Constants.SENSOR_TYPE;
+import static org.apache.metron.common.Constants.SENSOR_TYPE_FIELD_PROPERTY;
 import static org.apache.metron.indexing.dao.MetaAlertDao.ALERT_FIELD;
 import static org.apache.metron.indexing.dao.MetaAlertDao.METAALERTS_INDEX;
 import static org.apache.metron.indexing.dao.MetaAlertDao.METAALERT_FIELD;
 import static org.apache.metron.indexing.dao.MetaAlertDao.METAALERT_TYPE;
 import static org.apache.metron.indexing.dao.MetaAlertDao.STATUS_FIELD;
+import static org.apache.metron.indexing.dao.MetaAlertDao.THREAT_FIELD_PROPERTY;
 
 import com.fasterxml.jackson.core.JsonProcessingException;
 import com.google.common.base.Joiner;
@@ -85,7 +88,7 @@ public class ElasticsearchMetaAlertIntegrationTest {
   private static final String NEW_FIELD = "new-field";
   private static final String NAME_FIELD = "name";
 
-  private static IndexDao esDao;
+  private static ElasticsearchDao esDao;
   private static MetaAlertDao metaDao;
   private static ElasticSearchComponent es;
 
@@ -193,6 +196,12 @@ public class ElasticsearchMetaAlertIntegrationTest {
         .withIndexDir(new File(INDEX_DIR))
         .build();
     es.start();
+  }
+
+  @Before
+  public void setup() throws IOException {
+    es.createIndexWithMapping(METAALERTS_INDEX, MetaAlertDao.METAALERT_DOC, template.replace("%MAPPING_NAME%", "metaalert"));
+    es.createIndexWithMapping(INDEX, "index_doc", template.replace("%MAPPING_NAME%", "index"));
 
     AccessConfig accessConfig = new AccessConfig();
     Map<String, Object> globalConfig = new HashMap<String, Object>() {
@@ -212,12 +221,6 @@ public class ElasticsearchMetaAlertIntegrationTest {
     metaDao = new ElasticsearchMetaAlertDao(esDao);
   }
 
-  @Before
-  public void setup() throws IOException {
-    es.createIndexWithMapping(METAALERTS_INDEX, MetaAlertDao.METAALERT_DOC, template.replace("%MAPPING_NAME%", "metaalert"));
-    es.createIndexWithMapping(INDEX, "index_doc", template.replace("%MAPPING_NAME%", "index"));
-  }
-
   @AfterClass
   public static void teardown() {
     if (es != null) {
@@ -320,6 +323,12 @@ public class ElasticsearchMetaAlertIntegrationTest {
         findCreatedDoc(metaAlertCreateResponse.getGuid(), MetaAlertDao.METAALERT_TYPE);
       }
       {
+        // Verify metaalert has the default field names
+        Document metaAlert = metaDao.getLatest(metaAlertCreateResponse.getGuid(), MetaAlertDao.METAALERT_TYPE);
+        Assert.assertTrue(metaAlert.getDocument().containsKey(ElasticsearchMetaAlertDao.SOURCE_TYPE));
+        Assert.assertTrue(metaAlert.getDocument().containsKey(ElasticsearchMetaAlertDao.THREAT_TRIAGE_FIELD));
+      }
+      {
         // Verify alert 0 was not updated with metaalert field
         Document alert = metaDao.getLatest("message_0", SENSOR_NAME);
         Assert.assertEquals(4, alert.getDocument().size());
@@ -343,6 +352,45 @@ public class ElasticsearchMetaAlertIntegrationTest {
   }
 
   @Test
+  public void shouldCreateMetaAlertWithConfiguredFieldNames() throws Exception {
+    // Configure field names
+    AccessConfig accessConfig = esDao.getAccessConfig();
+    accessConfig.setGlobalConfigSupplier(() -> new HashMap<String, Object>() {{
+      put("es.date.format", DATE_FORMAT);
+      put(SENSOR_TYPE_FIELD_PROPERTY, SENSOR_TYPE);
+      put(THREAT_FIELD_PROPERTY, MetaAlertDao.THREAT_FIELD_DEFAULT);
+    }});
+
+    // Load alerts
+    List<Map<String, Object>> alerts = buildAlerts(1);
+    elasticsearchAdd(alerts, INDEX, SENSOR_NAME);
+
+    // Verify load was successful
+    findCreatedDocs(Collections.singletonList(
+            new GetRequest("message_0", SENSOR_NAME)));
+
+    {
+      MetaAlertCreateRequest metaAlertCreateRequest = new MetaAlertCreateRequest() {{
+        setAlerts(new ArrayList<GetRequest>() {{
+          add(new GetRequest("message_0", SENSOR_NAME));
+        }});
+        setGroups(Collections.singletonList("group"));
+      }};
+      MetaAlertCreateResponse metaAlertCreateResponse = metaDao.createMetaAlert(metaAlertCreateRequest);
+      {
+        // Verify metaAlert was created
+        findCreatedDoc(metaAlertCreateResponse.getGuid(), MetaAlertDao.METAALERT_TYPE);
+      }
+      {
+        // Verify alert 0 was not updated with metaalert field
+        Document metaAlert = metaDao.getLatest(metaAlertCreateResponse.getGuid(), MetaAlertDao.METAALERT_TYPE);
+        Assert.assertTrue(metaAlert.getDocument().containsKey(SENSOR_TYPE));
+        Assert.assertTrue(metaAlert.getDocument().containsKey(MetaAlertDao.THREAT_FIELD_DEFAULT));
+      }
+    }
+  }
+
+  @Test
   public void shouldAddAlertsToMetaAlert() throws Exception {
     // Load alerts
     List<Map<String, Object>> alerts = buildAlerts(4);
@@ -1173,7 +1221,7 @@ public class ElasticsearchMetaAlertIntegrationTest {
       Map<String, Object> alerts = new HashMap<>();
       alerts.put(Constants.GUID, guid);
       alerts.put("source:type", SENSOR_NAME);
-      alerts.put(MetaAlertDao.THREAT_FIELD_DEFAULT, i);
+      alerts.put(ElasticsearchMetaAlertDao.THREAT_TRIAGE_FIELD, i);
       alerts.put("timestamp", System.currentTimeMillis());
       inputData.add(alerts);
     }

http://git-wip-us.apache.org/repos/asf/metron/blob/32d132b3/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/MetaAlertDao.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/MetaAlertDao.java b/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/MetaAlertDao.java
index 8807849..93c791b 100644
--- a/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/MetaAlertDao.java
+++ b/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/MetaAlertDao.java
@@ -68,7 +68,8 @@ public interface MetaAlertDao extends IndexDao {
   String METAALERT_TYPE = "metaalert";
   String METAALERT_FIELD = "metaalerts";
   String METAALERT_DOC = METAALERT_TYPE + "_doc";
-  String THREAT_FIELD_DEFAULT = "threat:triage:score";
+  String THREAT_FIELD_DEFAULT = "threat.triage.score";
+  String THREAT_FIELD_PROPERTY = "threat.triage.score.field";
   String THREAT_SORT_DEFAULT = "sum";
   String ALERT_FIELD = "metron_alert";
   String STATUS_FIELD = "status";


[29/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/development/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/development/index.html b/site/current-book/metron-deployment/development/index.html
new file mode 100644
index 0000000..ea99fdf
--- /dev/null
+++ b/site/current-book/metron-deployment/development/index.html
@@ -0,0 +1,137 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/development/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Metron Development Environments</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron Development Environments</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Development</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-deployment/development/centos6/index.html" title="Centos6"><span class="none"></span>Centos6</a></li>
+    <li><a href="../../metron-deployment/development/fastcapa/index.html" title="Fastcapa"><span class="none"></span>Fastcapa</a></li>
+    <li><a href="../../metron-deployment/development/ubuntu14/index.html" title="Ubuntu14"><span class="none"></span>Ubuntu14</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron Development Environments</h1>
+<p><a name="Metron_Development_Environments"></a></p>
+<p>This directory contains environments useful for Metron developers.  These environments are not intended for proof-of-concept, testing, or production use.  These are extremely resource constrained and cannot support anything beyond the most basic work loads.</p>
+<ul>
+
+<li>Metron running on CentOS 6</li>
+<li>Metron running on Ubuntu 14</li>
+<li>Fastcapa</li>
+</ul>
+<div class="section">
+<h2><a name="Vagrant_Cachier_recommendations"></a>Vagrant Cachier recommendations</h2>
+<p>The development boxes are designed to be spun up and destroyed on a regular basis as part of the development cycle. In order to avoid the overhead of re-downloading many of the heavy platform dependencies, Vagrant can use the <a class="externalLink" href="http://fgrehm.viewdocs.io/vagrant-cachier/">vagrant-cachier</a> plugin to store package caches between builds. If the plugin has been installed to your vagrant it will be used, and packages will be cached in ~/.vagrant/cache.</p></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/development/ubuntu14/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/development/ubuntu14/index.html b/site/current-book/metron-deployment/development/ubuntu14/index.html
new file mode 100644
index 0000000..16b2d81
--- /dev/null
+++ b/site/current-book/metron-deployment/development/ubuntu14/index.html
@@ -0,0 +1,233 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/development/ubuntu14/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Metron on Ubuntu 14</title>
+    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../css/site.css" />
+    <link rel="stylesheet" href="../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron on Ubuntu 14</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-down"></span>Development</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/development/centos6/index.html" title="Centos6"><span class="none"></span>Centos6</a></li>
+    <li><a href="../../../metron-deployment/development/fastcapa/index.html" title="Fastcapa"><span class="none"></span>Fastcapa</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Ubuntu14</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron on Ubuntu 14</h1>
+<p>This project fully automates the provisioning and deployment of Apache Metron and all necessary prerequisites on a single, virtualized host running Ubuntu 14.</p>
+<p>Metron is composed of many components and installing all of these on a single host, especially a virtualized one, will greatly stress the resources of the host.   The host will require at least 8 GB of RAM and a fair amount of patience.  It is highly recommended that you shut down all unnecessary services.</p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<p>The computer used to deploy Apache Metron will need to have the following components installed.</p>
+<ul>
+
+<li><a class="externalLink" href="https://github.com/ansible/ansible">Ansible</a> (2.0.0.2, 2.2.2.0, or 2.5.0)</li>
+<li><a class="externalLink" href="https://www.docker.com/community-edition">Docker</a></li>
+<li><a class="externalLink" href="https://www.vagrantup.com">Vagrant</a> 2.0+</li>
+<li><a class="externalLink" href="https://github.com/devopsgroup-io/vagrant-hostmanager">Vagrant Hostmanager Plugin</a></li>
+<li><a class="externalLink" href="https://virtualbox.org">Virtualbox</a> 5.0+</li>
+<li>Python 2.7</li>
+<li>Maven 3.3.9</li>
+<li>C++11 compliant compiler, like <a class="externalLink" href="https://gcc.gnu.org/projects/cxx-status.html#cxx11">GCC</a></li>
+</ul>
+<p>Running the following script can help validate whether you have all the prerequisites installed and running correctly.</p>
+
+<div>
+<div>
+<pre class="source">  metron-deployment/scripts/platform-info.sh
+</pre></div></div>
+
+<div class="section">
+<h4><a name="How_do_I_install_these_on_MacOS.3F"></a>How do I install these on MacOS?</h4>
+<p>Any platform that supports these tools is suitable, but the following instructions cover installation on macOS.  The easiest means of installing these tools on a Mac is to use the excellent <a class="externalLink" href="http://brew.sh/">Homebrew</a> project.</p>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Install Homebrew by following the instructions at <a class="externalLink" href="http://brew.sh/">Homebrew</a>.</p>
+</li>
+<li>
+
+<p>Run the following command in a terminal to install all of the required tools.</p>
+
+<div>
+<div>
+<pre class="source">brew cask install vagrant virtualbox docker
+brew cask install caskroom/versions/java8
+brew install maven@3.3 git
+pip install ansible==2.2.2.0
+vagrant plugin install vagrant-hostmanager
+open /Applications/Docker.app
+</pre></div></div>
+</li>
+</ol></div></div>
+<div class="section">
+<h3><a name="Deploy_Metron"></a>Deploy Metron</h3>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Ensure that the Docker service is running.</p>
+</li>
+<li>
+
+<p>Deploy Metron</p>
+
+<div>
+<div>
+<pre class="source">cd metron-deployment/development/ubuntu14
+vagrant up
+</pre></div></div>
+
+<p>Should the process fail before completing the deployment, the following command will continue the deployment process without re-instantiating the host.</p>
+
+<div>
+<div>
+<pre class="source">vagrant provision
+</pre></div></div>
+</li>
+</ol></div>
+<div class="section">
+<h3><a name="Explore_Metron"></a>Explore Metron</h3>
+<p>Navigate to the following resources to explore your newly minted Apache Metron environment.</p>
+<ul>
+
+<li><a class="externalLink" href="http://node1:4201">Metron Alerts</a></li>
+<li><a class="externalLink" href="http://node1:8080">Ambari</a></li>
+</ul>
+<p>Connecting to the host through SSH is as simple as running the following command.</p>
+
+<div>
+<div>
+<pre class="source">vagrant ssh
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="Working_with_Metron"></a>Working with Metron</h3>
+<p>In addition to re-running the entire provisioning play book, you may now re-run an individual Ansible tag or a collection of tags in the following ways.  The following commands will re-run the <tt>sensor-stubs</tt> role on the Vagrant image. This will install and start the sensor stub components.</p>
+
+<div>
+<div>
+<pre class="source">vagrant --ansible-tags=&quot;sensor-stubs&quot; provision
+</pre></div></div>
+
+<p>Tags are listed in the playbooks, some frequently used tags:</p>
+<ul>
+
+<li><tt>hdp-install</tt> - Install HDP</li>
+<li><tt>hdp-deploy</tt> - Deploy and Start HDP Services (will start all Hadoop Services)</li>
+<li><tt>sensors</tt> - Deploy and start the sensors.</li>
+<li><tt>sensor-stubs</tt> - Deploy and start the sensor stubs.</li>
+</ul></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/index.html b/site/current-book/metron-deployment/index.html
index 72723a6..a3ffcbd 100644
--- a/site/current-book/metron-deployment/index.html
+++ b/site/current-book/metron-deployment/index.html
@@ -1,540 +1,285 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Overview</title>
-    <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+    <title>Metron &#x2013; </title>
+    <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../css/site.css" />
     <link rel="stylesheet" href="../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Overview</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active "></li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Deployment</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Overview</h1>
-<p><a name="Overview"></a></p>
-<p>This set of playbooks can be used to deploy an Ambari-managed Hadoop cluster containing Metron services using Ansible. These playbooks target RHEL/CentOS 6.x operating systems.</p>
-<p>Installation consists of -</p>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
 
-<ul>
-  
-<li>Building Metron tarballs, RPMs and the Ambari MPack</li>
-  
-<li>Deploying Ambari</li>
-  
-<li>Leveraging Ambari to install:
-  
-<ul>
-    
-<li>The required Hadoop Components</li>
-    
-<li>Core Metron (Parsing, Enrichment, Indexing)</li>
-    
-<li>Elasticsearch</li>
-    
-<li>Kibana</li>
-  </ul></li>
-  
-<li>Starting All Services</li>
-</ul>
-<div class="section">
-<h2><a name="Prerequisites"></a>Prerequisites</h2>
-<p>The following tools are required to run these scripts:</p>
+http://www.apache.org/licenses/LICENSE-2.0
 
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<p>This project contains tools for building, packaging, and deploying Apache Metron.  Please refer to the following sections for more information on how to get Apache Metron running in your environment.</p>
 <ul>
-  
-<li><a class="externalLink" href="https://maven.apache.org/">Maven</a></li>
-  
-<li><a class="externalLink" href="https://git-scm.com/">Git</a></li>
-  
-<li><a class="externalLink" href="http://www.ansible.com/">Ansible</a> (2.0.0.2 or 2.2.2.0)</li>
-  
-<li><a class="externalLink" href="https://www.docker.com/">Docker</a> (Docker for Mac on OSX)</li>
-</ul>
-<p>These scripts depend on two files for configuration:</p>
 
-<ul>
-  
-<li>hosts - declares which Ansible roles will be run on which hosts</li>
-  
-<li>group_vars/all - various configuration settings needed to install Metron</li>
+<li><a href="#How_do_I_deploy_Metron_with_Ambari">How do I deploy Metron with Ambari?</a></li>
+<li><a href="#How_do_I_deploy_Metron_on_a_single_VM">How do I deploy Metron on a single VM?</a></li>
+<li><a href="#How_do_I_build_RPM_packages">How do I build RPM packages?</a></li>
+<li><a href="#How_do_I_build_DEB_packages">How do I build DEB packages?</a></li>
+<li><a href="#How_do_I_deploy_Metron_within_AWS">How do I deploy Metron within AWS?</a></li>
+<li><a href="#How_do_I_build_Metron_with_Docker">How do I build Metron with Docker?</a></li>
 </ul>
-<p>For production use, it is recommended that Metron be installed on an existing cluster managed by Ambari as described in the Installing Management Pack section below.</p></div>
 <div class="section">
-<h2><a name="Ambari"></a>Ambari</h2>
-<p>The Ambari playbook will install a Hadoop cluster including the Metron Services (Parsing, Enrichment, Indexing). Ambari will also install Elasticsearch and Kibana.</p>
-<p>Currently, the playbooks supports building a local development cluster running on one node or deploying to a 10 node cluster on AWS EC2.</p></div>
+<h2><a name="How_do_I_deploy_Metron_with_Ambari.3F"></a>How do I deploy Metron with Ambari?</h2>
+<p>This provides a Management Pack (MPack) extension for <a class="externalLink" href="https://ambari.apache.org/">Apache Ambari</a> that simplifies the provisioning, management and monitoring of Metron on clusters of any size.</p>
+<p>This allows you to easily install Metron using a simple, guided process.  This also allows you to monitor cluster health and even secure your cluster with kerberos.</p>
 <div class="section">
-<h2><a name="Vagrant"></a>Vagrant</h2>
-<p>There is a development environment based on Vagrant that is referred to as &#x201c;Full Dev&#x201d;. This installs the entire Ambari/Metron stack. This is useful in testing out changes to the installation procedure.</p>
 <div class="section">
-<h3><a name="Prerequsities"></a>Prerequsities</h3>
-
+<h4><a name="What_is_this_good_for.3F"></a>What is this good for?</h4>
 <ul>
-  
-<li>Install <a class="externalLink" href="https://www.vagrantup.com/">Vagrant</a> (5.0.16+)</li>
-  
-<li>Install the Hostmanager plugin for vagrant - Run <tt>vagrant plugin install vagrant-hostmanager</tt> on the machine where Vagrant is installed</li>
+
+<li>
+
+<p>If you want to see how Metron can really scale by deploying it on your own hardware, or even in the cloud, this is the best option for you.</p>
+</li>
+<li>
+
+<p>If you want to run a proof-of-concept to see how Apache Metron can benefit your organization, then this is the way to do it.</p>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="Full-Dev"></a>Full-Dev</h3>
-<p>Navigate to <tt>metron/metron-deployment/vagrant/full-dev-platform</tt> and run <tt>vagrant up</tt>.</p></div></div>
+<h4><a name="How.3F"></a>How?</h4>
+<p>To deploy Apache Metron using Ambari, follow the instructions at <a href="packaging/ambari/metron-mpack/index.html">packaging/ambari/metron-mpack</a>.</p></div></div></div>
 <div class="section">
-<h2><a name="Ambari_Management_Pack"></a>Ambari Management Pack</h2>
-<p>An Ambari Management Pack can be built in order to make the Metron service available on top of an existing stack, rather than needing a direct stack update.</p>
-<p>This will set up</p>
-
-<ul>
-  
-<li>Metron Parsers</li>
-  
-<li>Enrichment</li>
-  
-<li>Indexing</li>
-  
-<li>GeoIP data</li>
-  
-<li>Optional Elasticsearch</li>
-  
-<li>Optional Kibana</li>
-</ul>
+<h2><a name="How_do_I_deploy_Metron_on_a_single_VM.3F"></a>How do I deploy Metron on a single VM?</h2>
+<p>This will deploy Metron and all of its dependencies on a virtual machine running on your computer.</p>
 <div class="section">
-<h3><a name="Prerequisites"></a>Prerequisites</h3>
-
+<div class="section">
+<h4><a name="What_is_this_good_for.3F"></a>What is this good for?</h4>
 <ul>
-  
-<li>A cluster managed by Ambari 2.4.2+</li>
-  
-<li>Metron RPMs available on the cluster in the /localrepo directory. See <a href="#RPMs">RPMs</a> for further information.</li>
-  
-<li><a class="externalLink" href="https://nodejs.org/en/download/package-manager/">Node.js</a> repository installed on the Management UI host</li>
+
+<li>
+
+<p>If you are new to Metron and want to explore the functionality that it offers, this is good place to start.</p>
+</li>
+<li>
+
+<p>If you are a developer contributing to the Apache Metron project, this is also a great way to test your changes.</p>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="Building_Management_Pack"></a>Building Management Pack</h3>
-<p>From <tt>metron-deployment</tt> run</p>
+<h4><a name="What_is_this_not_good_for.3F"></a>What is this <b>not</b> good for?</h4>
+<ul>
 
-<div class="source">
-<div class="source">
-<pre>mvn clean package
-</pre></div></div>
-<p>A tar.gz that can be used with Ambari can be found at <tt>metron-deployment/packaging/ambari/metron-mpack/target/</tt></p></div>
-<div class="section">
-<h3><a name="Installing_Management_Pack"></a>Installing Management Pack</h3>
-<p>Before installing the mpack, update Storm&#x2019;s topology.classpath in Ambari to include &#x2018;/etc/hbase/conf:/etc/hadoop/conf&#x2019;. Restart Storm service.</p>
-<p>Place the mpack&#x2019;s tar.gz onto the node running Ambari Server. From the command line on this node, run</p>
+<li>
 
-<div class="source">
-<div class="source">
-<pre>ambari-server install-mpack --mpack=&lt;mpack_location&gt; --verbose
-</pre></div></div>
-<p>This will make the services available in Ambari in the same manner as any services in a stack, e.g. through Add Services or during cluster install. The Indexing / Parsers/ Enrichment masters should be colocated with a Kafka Broker (to create topics) and HBase client (to create the enrichment and theatintel tables). This colocation is currently not enforced by Ambari, and should be managed by either a Service or Stack advisor as an enhancement.</p>
-<p>Several configuration parameters will need to be filled in, and should be pretty self explanatory (primarily a couple of Elasticsearch configs, and the Storm REST URL). Examples are provided in the descriptions on Ambari. Notably, the URL for the GeoIP database that is preloaded (and is prefilled by default) can be set to use a <tt>file:///</tt> location</p>
-<p>After installation, a custom action is available in Ambari (where stop / start services are) to install Elasticsearch templates. Similar to this, a custom Kibana action to Load Template is available.</p>
-<p>Another custom action is available in Ambari to import Zeppelin dashboards. See the <a href="../metron-platform/metron-indexing/index.html">metron-indexing documentation</a></p>
-<div class="section">
-<h4><a name="Offline_installation"></a>Offline installation</h4>
-<p>Currently there is only one point that would reach out to the internet during an install. This is the URL for the GeoIP database information.</p>
-<p>The RPMs DO NOT reach out to the internet (because there is currently no hosting for them). They look on the local filesystem in <tt>/localrepo</tt>.</p></div></div>
-<div class="section">
-<h3><a name="Current_Limitations"></a>Current Limitations</h3>
-<p>There are a set of limitations that should be addressed based to improve the current state of the mpacks.</p>
+<p>This VM is <b>not</b> intended for processing anything beyond the most basic, low volume work loads.</p>
+</li>
+<li>
 
-<ul>
-  
-<li>There is currently no hosting for RPMs remotely. They will have to be built locally.</li>
-  
-<li>Colocation of appropriate services should be enforced by Ambari. See [#Installing Management Pack] for more details.</li>
-  
-<li>Storm&#x2019;s topology.classpath is not updated with the Metron service install and needs to be updated separately.</li>
-  
-<li>Several configuration parameters used when installing the Metron service could (and should) be grabbed from Ambari. Install will require them to be manually entered.</li>
-  
-<li>Need to handle upgrading Metron</li>
-</ul></div></div>
-<div class="section">
-<h2><a name="RPMs"></a>RPMs</h2>
-<p>RPMs can be built to install the components in metron-platform. These RPMs are built in a Docker container and placed into <tt>target</tt>.</p>
-<p>Components in the RPMs:</p>
+<p>Additional services should <b>not</b> be installed along side Metron in this VM.</p>
+</li>
+<li>
 
-<ul>
-  
-<li>metron-common</li>
-  
-<li>metron-data-management</li>
-  
-<li>metron-elasticsearch</li>
-  
-<li>metron-enrichment</li>
-  
-<li>metron-parsers</li>
-  
-<li>metron-pcap</li>
-  
-<li>metron-solr</li>
-  
-<li>stellar-common</li>
+<p>This VM should <b>not</b> be used to run a proof-of-concept for Apache Metron within your organization.</p>
+</li>
 </ul>
+<p>Running Metron within the resource constraints of a single VM is incredibly challenging. Failing to respect this warning, will cause various services to fail mysteriously as the system runs into memory and processing limits.</p></div>
 <div class="section">
-<h3><a name="Prerequisites"></a>Prerequisites</h3>
-
+<h4><a name="How.3F"></a>How?</h4>
+<p>To deploy Metron in a VM running on your computer, follow the instructions at <a href="development/centos6/index.html">development/centos6</a>.</p></div></div></div>
+<div class="section">
+<h2><a name="How_do_I_build_RPM_packages.3F"></a>How do I build RPM packages?</h2>
+<p>This provides RPM packages that allow you to install Metron on an RPM-based operating system like CentOS.</p>
+<div class="section">
+<div class="section">
+<h4><a name="What_is_this_good_for.3F"></a>What is this good for?</h4>
 <ul>
-  
-<li>Docker. The image detailed in: <tt>metron-deployment/packaging/docker/rpm-docker/README.md</tt> will automatically be built (or rebuilt if necessary).</li>
-  
-<li>Artifacts for metron-platform have been produced. E.g. <tt>mvn clean package -DskipTests</tt> in <tt>metron-platform</tt></li>
-</ul>
-<p>The artifacts are required because there is a dependency on modules not expressed via Maven (we grab the resulting assemblies, but don&#x2019;t need the jars). These are</p>
 
+<li>
+
+<p>If you want to manually install Apache Metron on an RPM-based system like CentOS, installation can be simplified by using these packages.</p>
+</li>
+<li>
+
+<p>If you want a guided installation process using Ambari on an RPM-based system, then these RPMs are a necessary prerequisite.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="What_is_this_not_good_for.3F"></a>What is this <b>not</b> good for?</h4>
 <ul>
-  
-<li>metron-common</li>
-  
-<li>metron-data-management</li>
-  
-<li>metron-elasticsearch</li>
-  
-<li>metron-enrichment</li>
-  
-<li>metron-indexing</li>
-  
-<li>metron-parsers</li>
-  
-<li>metron-pcap-backend</li>
-  
-<li>metron-solr</li>
-  
-<li>metron-profiler</li>
-  
-<li>metron-config</li>
+
+<li>If you want a complete, guided installation process, use Ambari rather than just these packages.  Installing Metron using <b>only</b> these RPMs still leaves a considerable amount of configuration necessary to get Metron running.  Installing with Ambari automates these additional steps.</li>
 </ul></div>
 <div class="section">
-<h3><a name="Building_RPMs"></a>Building RPMs</h3>
+<h4><a name="How.3F"></a>How?</h4>
+<p>To build the RPM packages, follow the instructions at <a href="packaging/docker/rpm-docker/index.html">packaging/docker/rpm-docker</a>.</p></div></div></div>
+<div class="section">
+<h2><a name="How_do_I_build_DEB_packages.3F"></a>How do I build DEB packages?</h2>
+<p>This builds installable DEB packages that allow you to install Metron on an APT-based operating system like Ubuntu.</p>
+<div class="section">
+<div class="section">
+<h4><a name="What_is_this_good_for.3F"></a>What is this good for?</h4>
+<ul>
+
+<li>
 
-<div class="source">
-<div class="source">
-<pre>cd metron-deployment
-mvn clean package -Pbuild-rpms
-</pre></div></div>
-<p>The output RPM files will land in <tt>target/RPMS/noarch</tt>. They can be installed with the standard</p>
+<p>If you want to manually install Metron on a APT-based system like Ubuntu, installation can be simplified by using these packages.</p>
+</li>
+<li>
 
-<div class="source">
-<div class="source">
-<pre>rpm -i &lt;package&gt;
-</pre></div></div></div></div>
+<p>If you want a guided installation process using Ambari on an APT-based system, then these DEBs are a necessary prerequisite.</p>
+</li>
+</ul></div>
 <div class="section">
-<h2><a name="Kibana_Dashboards"></a>Kibana Dashboards</h2>
-<p>The dashboards installed by the Kibana custom action are managed by the dashboard.p file. This file is created by exporting existing dashboards from a running Kibana instance.</p>
-<p>To create a new version of the file, make any necessary changes to Kibana (e.g. on full-dev), and export with the appropriate script.</p>
+<h4><a name="What_is_this_not_good_for.3F"></a>What is this <b>not</b> good for?</h4>
+<ul>
 
-<div class="source">
-<div class="source">
-<pre>python packaging/ambari/metron-mpack/src/main/resources/common-services/KIBANA/4.5.1/package/scripts/dashboard/dashboardindex.py \
-$ES_HOST 9200 \
-packaging/ambari/metron-mpack/src/main/resources/common-services/KIBANA/4.5.1/package/scripts/dashboard/dashboard.p -s
-</pre></div></div>
-<p>Build the Ambari Mpack to get the dashboard updated appropriately.</p>
-<p>Once the MPack is installed, run the Kibana service&#x2019;s action &#x201c;Load Template&#x201d; to install dashboards. This will completely overwrite the .kibana in Elasticsearch, so use with caution.</p></div>
+<li>If you want a complete, guided installation process, use Ambari rather than just these packages.  Installing Metron using <b>only</b> these RPMs still leaves a considerable amount of configuration necessary to get Metron running.  Installing with Ambari automates these additional steps.</li>
+</ul></div>
 <div class="section">
-<h2><a name="Kerberos"></a>Kerberos</h2>
-<p>The MPack can allow Metron to be installed and then Kerberized, or installed on top of an already Kerberized cluster. This is done through Ambari&#x2019;s standard Kerberization setup.</p>
+<h4><a name="How.3F"></a>How?</h4>
+<p>To build the DEB packages, follow the instructions at <a href="packaging/docker/deb-docker/index.html">packaging/docker/deb-docker</a>.</p></div></div></div>
 <div class="section">
-<h3><a name="Caveats"></a>Caveats</h3>
-
+<h2><a name="How_do_I_deploy_Metron_within_AWS.3F"></a>How do I deploy Metron within AWS?</h2>
+<p>This deploys Apache Metron on an automatically provisioned 10-node cluster running in Amazon Web Service&#x2019;s EC2 platform.</p>
+<p>This installs real sources of telemetry like Bro, Snort, and YAF, but feeds those sensors with canned pcap data.</p>
+<div class="section">
+<div class="section">
+<h4><a name="What_is_this_good_for.3F"></a>What is this good for?</h4>
 <ul>
-  
-<li>For nodes using a Metron client and a local repo, the repo must exist on all nodes (e.g via createrepo). This repo can be empty; only the main Metron services need the RPMs.</li>
-  
-<li>A Metron client must be installed on each supervisor node in a secured cluster. This is to ensure that the Metron keytab and client_jaas.conf get distributed in order to allow reading and writing from Kafka.
-  
+
+<li>If you are a Metron developer wanting to test at-scale on a multi-node cluster, then this is the right option for you.</li>
+</ul></div>
+<div class="section">
+<h4><a name="What_is_this_not_good_for.3F"></a>What is this <b>not</b> good for?</h4>
 <ul>
-    
-<li>When Metron is already installed on the cluster, this should be done before Kerberizing.</li>
-    
-<li>When addding Metron to an already Kerberized cluster, ensure that all supervisor nodes receive a Metron client.</li>
-  </ul></li>
-  
-<li>Storm (and Metron) must be restarted after Metron is installed on an already Kerberized cluster. Several Storm configs get updated, and Metron will be unable to write to Kafka without a restart.
-  
+
+<li>
+
+<p>If you want to run Metron in AWS with real data for either testing or production, then this is NOT the right option for you.</p>
+</li>
+<li>
+
+<p><b>WARNING</b> This is only intended for creating an ephemeral cluster for brief periods of testing.  This deployment method has the following severe limitations.</p>
 <ul>
-    
-<li>Kerberizing a cluster with an existing Metron already has restarts of all services during Kerberization, so it&#x2019;s unneeded.</li>
-  </ul></li>
+
+<li>The cluster is not secured in any way. It is up to you to manually secure it.</li>
+<li>The cluster will not survive a reboot.</li>
 </ul>
-<p>Instructions for setup on Full Dev can be found at <a href="Kerberos-ambari-setup.html">Kerberos-ambari-setup.md</a>. These instructions reference the manual install instructions.</p></div>
+</li>
+</ul></div>
 <div class="section">
-<h3><a name="Kerberos_Without_an_MPack"></a>Kerberos Without an MPack</h3>
-<p>Using the MPack is preferred, but instructions for Kerberizing manually can be found at <a href="Kerberos-manual-setup.html">Kerberos-manual-setup.md</a>. These instructions are reference by the Ambari Kerberos install instructions and include commands for setting up a KDC.</p></div></div>
+<h4><a name="How.3F"></a>How?</h4>
+<p>Follow the instructions available at <a href="amazon-ec2/index.html">amazon-ec2</a>.</p></div></div></div>
 <div class="section">
-<h2><a name="TODO"></a>TODO</h2>
-
+<h2><a name="How_do_I_build_Metron_with_Docker.3F"></a>How do I build Metron with Docker?</h2>
+<p>This provides a Docker containing all of the prerequisites required to build Metron.  This allows you to easily build Metron without installing all of the build dependencies manually.</p>
+<div class="section">
+<div class="section">
+<h4><a name="What_is_this_good_for.3F"></a>What is this good for?</h4>
 <ul>
-  
-<li>Support Ubuntu deployments</li>
-</ul></div>
-                  </div>
-            </div>
-          </div>
 
+<li>If you want to build Metron, but do not want to manually install all of the build dependencies, then this is a good option.</li>
+</ul></div>
+<div class="section">
+<h4><a name="How.3F"></a>How?</h4>
+<p>Follow the instructions available at <a href="packaging/docker/ansible-docker/index.html">packaging/docker/ansible-docker</a>.</p></div></div></div>
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/other-examples/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/other-examples/index.html b/site/current-book/metron-deployment/other-examples/index.html
index 3cf67e2..3a89cb0 100644
--- a/site/current-book/metron-deployment/other-examples/index.html
+++ b/site/current-book/metron-deployment/other-examples/index.html
@@ -1,297 +1,130 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/other-examples/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Other Example Deployments</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Other Example Deployments</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Other Example Deployments</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                            
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Other-examples</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html" title="Manual_Install_CentOS6">
-          <i class="none"></i>
-        Manual_Install_CentOS6</a>
-            </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Other-examples</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html" title="Manual_Install_CentOS6"><span class="none"></span>Manual_Install_CentOS6</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Other Example Deployments</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Other Example Deployments</h1>
 <p><a name="Other_Example_Deployments"></a></p>
 <p>This folder contains other example deployments which are not covered under the automatic installs. These examples have been vetted and tested by the Metron community at the time of entry. However, development moves fast and things may change. If there are problems with these steps, please check the mailing lists first or log a Jira ticket.</p>
-
 <ul>
-  
+
 <li><a href="manual-install/Manual_Install_CentOS6.html">Metron 0.4.0 on CentOS 6 Manual Install</a> (July 2017)</li>
 </ul>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[10/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-sensors/pycapa/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-sensors/pycapa/index.html b/site/current-book/metron-sensors/pycapa/index.html
index 8c46464..f81aee9 100644
--- a/site/current-book/metron-sensors/pycapa/index.html
+++ b/site/current-book/metron-sensors/pycapa/index.html
@@ -1,313 +1,199 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-sensors/pycapa/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Pycapa</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Pycapa</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Pycapa</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                                    
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-down"></i>
-        Sensors</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-sensors/fastcapa/index.html" title="Fastcapa">
-          <i class="none"></i>
-        Fastcapa</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Pycapa</a>
-          </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-down"></span>Sensors</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-sensors/fastcapa/index.html" title="Fastcapa"><span class="none"></span>Fastcapa</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Pycapa</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <div class="section">
-<h2><a name="Pycapa"></a>Pycapa</h2>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
 
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<div class="section">
+<h2><a name="Pycapa"></a>Pycapa</h2>
 <ul>
-  
+
 <li><a href="#Overview">Overview</a></li>
-  
 <li><a href="#Installation">Installation</a></li>
-  
 <li><a href="#Usage">Usage</a>
-  
 <ul>
-    
+
 <li><a href="#Parameters">Parameters</a></li>
-    
 <li><a href="#Examples">Examples</a></li>
-    
 <li><a href="#Kerberos">Kerberos</a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><a href="#FAQs">FAQs</a></li>
 </ul>
 <h1>Overview</h1>
-<p>Pycapa performs network packet capture, both off-the-wire and from a Kafka topic, which is useful for the testing and development of <a class="externalLink" href="https://github.com/apache/metron">Apache Metron</a>. It is not intended for production use. The tool will capture packets from a specified interface and push them into a Kafka Topic. The tool can also do the reverse. It can consume packets from Kafka and reconstruct each network packet. This can then be used to create a <a class="externalLink" href="https://wiki.wireshark.org/Development/LibpcapFileFormat">libpcap-compliant file</a> or even to feed directly into a tool like Wireshark to monitor ongoing activity.</p>
+<p>Pycapa performs network packet capture, both off-the-wire and from a Kafka topic, which is useful for the testing and development of <a class="externalLink" href="https://github.com/apache/metron">Apache Metron</a>.  It is not intended for production use. The tool will capture packets from a specified interface and push them into a Kafka Topic.  The tool can also do the reverse.  It can consume packets from Kafka and reconstruct each network packet.  This can then be used to create a <a class="externalLink" href="https://wiki.wireshark.org/Development/LibpcapFileFormat">libpcap-compliant file</a> or even to feed directly into a tool like Wireshark to monitor ongoing activity.</p>
 <h1>Installation</h1>
 <p>General notes on the installation of Pycapa.</p>
-
 <ul>
-  
+
 <li>Python 2.7 is required.</li>
-  
 <li>The following package dependencies are required and can be installed automatically with <tt>pip</tt>.
-  
 <ul>
-    
+
 <li><a class="externalLink" href="https://github.com/confluentinc/confluent-kafka-python">confluent-kafka-python</a></li>
-    
 <li><a class="externalLink" href="https://github.com/CoreSecurity/pcapy">pcapy</a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>These instructions can be used directly on CentOS 7+.</li>
-  
 <li>Other Linux distributions that come with Python 2.7 can use these instructions with some minor modifications.</li>
-  
 <li>Older distributions, like CentOS 6, that come with Python 2.6 installed, should install Python 2.7 within a virtual environment and then run Pycapa from within the virtual environment.</li>
 </ul>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Install system dependencies including the core development tools, Python libraries and header files, and Libpcap libraries and header files. On CentOS 7+, you can install these requirements with the following command.</p>
-  
-<div class="source">
-<div class="source">
-<pre>   yum -y install &quot;@Development tools&quot; python-devel libpcap-devel
-</pre></div></div></li>
-  
+
+<p>Install system dependencies including the core development tools, Python libraries and header files, and Libpcap libraries and header files.  On CentOS 7+, you can install these requirements with the following command.</p>
+
+<div>
+<div>
+<pre class="source">yum -y install &quot;@Development tools&quot; python-devel libpcap-devel
+</pre></div></div>
+</li>
 <li>
+
 <p>Install Librdkafka at your chosen $PREFIX.</p>
-  
-<div class="source">
-<div class="source">
-<pre>   export PREFIX=/usr
-   wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz  -O - | tar -xz
-   cd librdkafka-0.9.4/
-   ./configure --prefix=$PREFIX
-   make
-   make install
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">export PREFIX=/usr
+wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz   -O - | tar -xz
+cd librdkafka-0.9.4/
+./configure --prefix=$PREFIX
+make
+make install
+</pre></div></div>
+</li>
 <li>
+
 <p>Add Librdkafka to the dynamic library load path.</p>
-  
-<div class="source">
-<div class="source">
-<pre>echo &quot;$PREFIX/lib&quot; &gt;&gt; /etc/ld.so.conf.d/pycapa.conf
+
+<div>
+<div>
+<pre class="source">echo &quot;$PREFIX/lib&quot; &gt;&gt; /etc/ld.so.conf.d/pycapa.conf
 ldconfig -v
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Install Pycapa. This assumes that you already have the Metron source code on the host.</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron/metron-sensors/pycapa
+
+<p>Install Pycapa.  This assumes that you already have the Metron source code on the host.</p>
+
+<div>
+<div>
+<pre class="source">cd metron/metron-sensors/pycapa
 pip install -r requirements.txt
 python setup.py install
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol>
 <h1>Usage</h1>
 <p>Pycapa has two primary runtime modes.</p>
-
 <ul>
-  
+
 <li>
-<p><b>Producer Mode</b>: Pycapa can capture packets from a network interface and forward those packets to a Kafka topic. Pycapa embeds the raw network packet data in the Kafka message body. The message key contains the timestamp indicating when the packet was captured in microseconds from the epoch, in network byte order.</p></li>
-  
+
+<p><b>Producer Mode</b>: Pycapa can capture packets from a network interface and forward those packets to a Kafka topic.  Pycapa embeds the raw network packet data in the Kafka message body.  The message key contains the timestamp indicating when the packet was captured in microseconds from the epoch, in network byte order.</p>
+</li>
 <li>
-<p><b>Consumer Mode</b>: Pycapa can also perform the reverse operation. It can consume packets from Kafka and reconstruct each network packet. This can then be used to create a <a class="externalLink" href="https://wiki.wireshark.org/Development/LibpcapFileFormat">libpcap-compliant file</a> or even to feed directly into a tool like Wireshark to monitor activity.</p></li>
+
+<p><b>Consumer Mode</b>: Pycapa can also perform the reverse operation.  It can consume packets from Kafka and reconstruct each network packet.  This can then be used to create a <a class="externalLink" href="https://wiki.wireshark.org/Development/LibpcapFileFormat">libpcap-compliant file</a> or even to feed directly into a tool like Wireshark to monitor activity.</p>
+</li>
 </ul>
 <div class="section">
 <h3><a name="Parameters"></a>Parameters</h3>
 
-<div class="source">
-<div class="source">
-<pre>$ pycapa --help
+<div>
+<div>
+<pre class="source">$ pycapa --help
 usage: pycapa [-h] [-p] [-c] [-k KAFKA_BROKERS] [-t KAFKA_TOPIC]
               [-o {begin,end,stored}] [-i NETWORK_IFACE] [-m MAX_PACKETS]
               [-pp PRETTY_PRINT] [-ll LOG_LEVEL] [-X KAFKA_CONFIGS]
@@ -335,16 +221,17 @@ optional arguments:
   -s SNAPLEN, --snaplen SNAPLEN
                         capture only the first X bytes of each packet;
                         default=65535
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Examples"></a>Examples</h3>
 <div class="section">
 <h4><a name="Example_1"></a>Example 1</h4>
-<p>Capture 10 packets from the <tt>eth0</tt> network interface and forward those to a Kafka topic called <tt>pcap</tt> running on <tt>localhost:9092</tt>. The process will not terminate until all messages have been delivered to Kafka.</p>
+<p>Capture 10 packets from the <tt>eth0</tt> network interface and forward those to a Kafka topic called <tt>pcap</tt> running on <tt>localhost:9092</tt>.  The process will not terminate until all messages have been delivered to Kafka.</p>
 
-<div class="source">
-<div class="source">
-<pre>$ pycapa --producer \
+<div>
+<div>
+<pre class="source">$ pycapa --producer \
     --interface eth0 \
     --kafka-broker localhost:9092 \
     --kafka-topic pcap \
@@ -353,14 +240,15 @@ INFO:root:Connecting to Kafka; {'bootstrap.servers': 'localhost:9092', 'group.id
 INFO:root:Starting packet capture
 INFO:root:Waiting for '6' message(s) to flush
 INFO:root:'10' packet(s) in, '10' packet(s) out
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Example_2"></a>Example 2</h4>
-<p>Capture packets until SIGINT is received (the interrupt signal sent when entering CTRL-C in the console.) In this example, nothing will be reported as packets are captured and delivered to Kafka. Simply wait a few seconds, then type CTRL-C and the number of packets will be reported.</p>
+<p>Capture packets until SIGINT is received (the interrupt signal sent when entering CTRL-C in the console.)  In this example, nothing will be reported as packets are captured and delivered to Kafka.  Simply wait a few seconds, then type CTRL-C and the number of packets will be reported.</p>
 
-<div class="source">
-<div class="source">
-<pre>$ pycapa --producer \
+<div>
+<div>
+<pre class="source">$ pycapa --producer \
     --interface en0 \
     --kafka-broker localhost:9092 \
     --kafka-topic pcap
@@ -370,14 +258,15 @@ INFO:root:Starting packet capture
 INFO:root:Clean shutdown process started
 INFO:root:Waiting for '2' message(s) to flush
 INFO:root:'21' packet(s) in, '21' packet(s) out
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Example_3"></a>Example 3</h4>
-<p>While capturing packets, output diagnostic information every 5 packets. Diagnostics will report when packets have been received from the network interface and when they have been successfully delivered to Kafka.</p>
+<p>While capturing packets, output diagnostic information every 5 packets.  Diagnostics will report when packets have been received from the network interface and when they have been successfully delivered to Kafka.</p>
 
-<div class="source">
-<div class="source">
-<pre>$ pycapa --producer \
+<div>
+<div>
+<pre class="source">$ pycapa --producer \
     --interface eth0 \
     --kafka-broker localhost:9092 \
     --kafka-topic pcap \
@@ -399,37 +288,39 @@ INFO:root:'21' packet(s) in, '21' packet(s) out
   INFO:root:Waiting for '1' message(s) to flush
   INFO:root:'27' packet(s) in, '27' packet(s) out
 
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Example_4"></a>Example 4</h4>
 <p>Consume 10 packets and create a libpcap-compliant pcap file.</p>
 
-<div class="source">
-<div class="source">
-<pre>  $ pycapa --consumer \
-      --kafka-broker localhost:9092 \
-      --kafka-topic pcap \
-      --max-packets 10 \
-      &gt; out.pcap
-  $ tshark -r out.pcap
-      1   0.000000 199.193.204.147 &#x2192; 192.168.0.3  TLSv1.2 151 Application Data
-      2   0.000005 199.193.204.147 &#x2192; 192.168.0.3  TLSv1.2 1191 Application Data
-      3   0.000088  192.168.0.3 &#x2192; 199.193.204.147 TCP 66 54788 &#x2192; 443 [ACK] Seq=1 Ack=86 Win=4093 Len=0 TSval=961284465 TSecr=943744612
-      4   0.000089  192.168.0.3 &#x2192; 199.193.204.147 TCP 66 54788 &#x2192; 443 [ACK] Seq=1 Ack=1211 Win=4058 Len=0 TSval=961284465 TSecr=943744612
-      5   0.948788  192.168.0.3 &#x2192; 192.30.253.125 TCP 54 54671 &#x2192; 443 [ACK] Seq=1 Ack=1 Win=4096 Len=0
-      6   1.005175 192.30.253.125 &#x2192; 192.168.0.3  TCP 66 [TCP ACKed unseen segment] 443 &#x2192; 54671 [ACK] Seq=1 Ack=2 Win=31 Len=0 TSval=2658544467 TSecr=961240339
-      7   1.636312 fe80::1286:8cff:fe0e:65df &#x2192; ff02::1      ICMPv6 134 Router Advertisement from 10:86:8c:0e:65:df
-      8   2.253052 192.175.27.112 &#x2192; 192.168.0.3  TLSv1.2 928 Application Data
-      9   2.253140  192.168.0.3 &#x2192; 192.175.27.112 TCP 66 55078 &#x2192; 443 [ACK] Seq=1 Ack=863 Win=4069 Len=0 TSval=961286699 TSecr=967172238
-     10   2.494769  192.168.0.3 &#x2192; 224.0.0.251  MDNS 82 Standard query 0x0000 PTR _googlecast._tcp.local, &quot;QM&quot; question
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">    $ pycapa --consumer \
+        --kafka-broker localhost:9092 \
+        --kafka-topic pcap \
+        --max-packets 10 \
+        &gt; out.pcap
+    $ tshark -r out.pcap
+        1   0.000000 199.193.204.147 &#x2192; 192.168.0.3  TLSv1.2 151   Application Data
+        2   0.000005 199.193.204.147 &#x2192; 192.168.0.3  TLSv1.2 1191   Application Data
+        3   0.000088  192.168.0.3 &#x2192; 199.193.204.147 TCP 66 54788 &#x2192; 443   [ACK] Seq=1 Ack=86 Win=4093 Len=0 TSval=961284465 TSecr=943744612
+        4   0.000089  192.168.0.3 &#x2192; 199.193.204.147 TCP 66 54788 &#x2192; 443   [ACK] Seq=1 Ack=1211 Win=4058 Len=0 TSval=961284465 TSecr=943744612
+        5   0.948788  192.168.0.3 &#x2192; 192.30.253.125 TCP 54 54671 &#x2192; 443   [ACK] Seq=1 Ack=1 Win=4096 Len=0
+        6   1.005175 192.30.253.125 &#x2192; 192.168.0.3  TCP 66 [TCP ACKed unseen segment] 443 &#x2192; 54671 [ACK] Seq=1 Ack=2 Win=31 Len=0 TSval=2658544467 TSecr=961240339
+        7   1.636312 fe80::1286:8cff:fe0e:65df &#x2192; ff02::1      ICMPv6 134 Router Advertisement from 10:86:8c:0e:65:df
+        8   2.253052 192.175.27.112 &#x2192; 192.168.0.3  TLSv1.2 928 Application Data
+        9   2.253140  192.168.0.3 &#x2192; 192.175.27.112 TCP 66 55078 &#x2192; 443 [ACK] Seq=1 Ack=863 Win=4069 Len=0 TSval=961286699 TSecr=967172238
+       10   2.494769  192.168.0.3 &#x2192; 224.0.0.251  MDNS 82 Standard query 0x0000 PTR _googlecast._tcp.local, &quot;QM&quot; question
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Example_5"></a>Example 5</h4>
 <p>Consume 10 packets from the Kafka topic <tt>pcap</tt> running on <tt>localhost:9092</tt>, then pipe those into Wireshark for DPI.</p>
 
-<div class="source">
-<div class="source">
-<pre>$ pycapa --consumer \
+<div>
+<div>
+<pre class="source">$ pycapa --consumer \
     --kafka-broker localhost:9092 \
     --kafka-topic pcap \
     --max-packets 10 \
@@ -446,59 +337,58 @@ Capturing on 'Standard input'
     9   2.083872 192.30.253.125 &#x2192; 192.168.0.3  TCP 66 443 &#x2192; 54671 [ACK] Seq=32 Ack=36 Win=31 Len=0 TSval=2658503087 TSecr=961120495
    10   3.173189 fe80::1286:8cff:fe0e:65df &#x2192; ff02::1      ICMPv6 134 Router Advertisement from 10:86:8c:0e:65:df
 10 packets captured
-</pre></div></div></div></div>
+</pre></div></div>
+</div></div>
 <div class="section">
 <h3><a name="Kerberos"></a>Kerberos</h3>
-<p>The probe can be used in a Kerberized environment. Follow these additional steps to use Pycapa with Kerberos. The following assumptions have been made. These may need altered to fit your environment.</p>
-
+<p>The probe can be used in a Kerberized environment.  Follow these additional steps to use Pycapa with Kerberos.  The following assumptions have been made.  These may need altered to fit your environment.</p>
 <ul>
-  
+
 <li>The Kafka broker is at <tt>kafka1:6667</tt></li>
-  
 <li>Zookeeper is at <tt>zookeeper1:2181</tt></li>
-  
 <li>The Kafka security protocol is <tt>SASL_PLAINTEXT</tt></li>
-  
 <li>The keytab used is located at <tt>/etc/security/keytabs/metron.headless.keytab</tt></li>
-  
 <li>The service principal is <tt>metron@EXAMPLE.COM</tt></li>
 </ul>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Build Librdkafka with SASL support (<tt>--enable-sasl</tt>) and install at your chosen $PREFIX.</p>
-  
-<div class="source">
-<div class="source">
-<pre>wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz  -O - | tar -xz
+
+<div>
+<div>
+<pre class="source">wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz  -O - | tar -xz
 cd librdkafka-0.9.4/
 ./configure --prefix=$PREFIX --enable-sasl
 make
 make install
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Validate Librdkafka does indeed support SASL. Run the following command and ensure that <tt>sasl</tt> is returned as a built-in feature.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ examples/rdkafka_example -X builtin.features
+
+<p>Validate Librdkafka does indeed support SASL.  Run the following command and ensure that <tt>sasl</tt> is returned as a built-in feature.</p>
+
+<div>
+<div>
+<pre class="source">$ examples/rdkafka_example -X builtin.features
 builtin.features = gzip,snappy,ssl,sasl,regex
 </pre></div></div>
-<p>If it is not, ensure that you have <tt>libsasl</tt> or <tt>libsasl2</tt> installed. On CentOS, this can be installed with the following command.</p>
-  
-<div class="source">
-<div class="source">
-<pre>yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi
-</pre></div></div></li>
-  
+
+<p>If it is not, ensure that you have <tt>libsasl</tt> or <tt>libsasl2</tt> installed.  On CentOS, this can be installed with the following command.</p>
+
+<div>
+<div>
+<pre class="source">yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi
+</pre></div></div>
+</li>
 <li>
-<p>Grant access to your Kafka topic. In this example the topic is simply named <tt>pcap</tt>.</p>
-  
-<div class="source">
-<div class="source">
-<pre>${KAFKA_HOME}/bin/kafka-acls.sh \
+
+<p>Grant access to your Kafka topic.  In this example the topic is simply named <tt>pcap</tt>.</p>
+
+<div>
+<div>
+<pre class="source">${KAFKA_HOME}/bin/kafka-acls.sh \
   --authorizer kafka.security.auth.SimpleAclAuthorizer \
   --authorizer-properties zookeeper.connect=zookeeper1:2181 \
   --add \
@@ -510,65 +400,60 @@ ${KAFKA_HOME}/bin/kafka-acls.sh \
   --add \
   --allow-principal User:metron \
   --group pycapa
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Use Pycapa as you normally would, but append the following three additional parameters</p>
-  
 <ul>
-    
+
 <li><tt>security.protocol</tt></li>
-    
 <li><tt>sasl.kerberos.keytab</tt></li>
-    
-<li><tt>sasl.kerberos.principal</tt></li>
-  </ul>
-  
-<div class="source">
-<div class="source">
-<pre>  $ pycapa --producer \
-  --interface eth0 \
-  --kafka-broker kafka1:6667 \
-  --kafka-topic pcap --max-packets 10 \
-  -X security.protocol=SASL_PLAINTEXT \
-  -X sasl.kerberos.keytab=/etc/security/keytabs/metron.headless.keytab \
-  -X sasl.kerberos.principal=metron-metron@METRONEXAMPLE.COM
-  INFO:root:Connecting to Kafka; {'sasl.kerberos.principal': 'metron-metron@METRONEXAMPLE.COM', 'group.id': 'ORNLVWJZZUAA', 'security.protocol': 'SASL_PLAINTEXT', 'sasl.kerberos.keytab': '/etc/security/keytabs/metron.headless.keytab', 'bootstrap.servers': 'kafka1:6667'}
-  INFO:root:Starting packet capture
-  INFO:root:Waiting for '1' message(s) to flush
-  INFO:root:'10' packet(s) in, '10' packet(s) out
-</pre></div></div></li>
+<li><tt>sasl.kerberos.principal</tt>
+
+<div>
+<div>
+<pre class="source">$ pycapa --producer \
+    --interface eth0 \
+    --kafka-broker kafka1:6667 \
+    --kafka-topic pcap --max-packets 10 \
+    -X security.protocol=SASL_PLAINTEXT \
+    -X sasl.kerberos.keytab=/etc/security/keytabs/metron.headless  .keytab \
+    -X sasl.kerberos.principal=metron-metron@METRONEXAMPLE.COM
+INFO:root:Connecting to Kafka; {'sasl.kerberos.principal':   'metron-metron@METRONEXAMPLE.COM', 'group.id': 'ORNLVWJZZUAA',   'security.protocol': 'SASL_PLAINTEXT', 'sasl.kerberos.keytab':   '/etc/security/keytabs/metron.headless.keytab', 'bootstrap.servers': 'kafka1:6667'}
+INFO:root:Starting packet capture
+INFO:root:Waiting for '1' message(s) to flush
+INFO:root:'10' packet(s) in, '10' packet(s) out
+</pre></div></div>
+</li>
+</ul>
+</li>
 </ol>
 <h1>FAQs</h1></div>
 <div class="section">
-<h3><a name="How_do_I_get_more_logs"></a>How do I get more logs?</h3>
+<h3><a name="How_do_I_get_more_logs.3F"></a>How do I get more logs?</h3>
 <p>Use the following two command-line arguments to get detailed logging.</p>
 
-<div class="source">
-<div class="source">
-<pre>-X debug=all --log-level DEBUG
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">-X debug=all --log-level DEBUG
+</pre></div></div>
+</div>
 <div class="section">
-<h3><a name="When_I_run_Pycapa_against_a_Kafka_broker_with_Kerberos_enabled_why_do_I_get_an_error_like_No_such_configuration_property:_sasl.kerberos.principal"></a>When I run Pycapa against a Kafka broker with Kerberos enabled, why do I get an error like &#x201c;No such configuration property: &#x2018;sasl.kerberos.principal&#x2019;&#x201d;?</h3>
-<p>This can be a confusing error message because <tt>sasl.kerberos.principal</tt> is indeed a valid property for librdkafka as defined <a class="externalLink" href="https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md">here</a>. This is most likely because Pycapa is running against a version of Librdkafka without SASL support enabled. This might happen if you have accidentally installed multiple versions of Librdkafka and Pycapa is unexpectedly using the version without SASL support enabled.</p>
+<h3><a name="When_I_run_Pycapa_against_a_Kafka_broker_with_Kerberos_enabled.2C_why_do_I_get_an_error_like_.E2.80.9CNo_such_configuration_property:_.E2.80.98sasl.kerberos.principal.E2.80.99.E2.80.9D.3F"></a>When I run Pycapa against a Kafka broker with Kerberos enabled, why do I get an error like &#x201c;No such configuration property: &#x2018;sasl.kerberos.principal&#x2019;&#x201d;?</h3>
+<p>This can be a confusing error message because <tt>sasl.kerberos.principal</tt> is indeed a valid property for librdkafka as defined <a class="externalLink" href="https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md">here</a>.  This is most likely because Pycapa is running against a version of Librdkafka without SASL support enabled.  This might happen if you have accidentally installed multiple versions of Librdkafka and Pycapa is unexpectedly using the version without SASL support enabled.</p>
 <p>Bottom Line: Make sure that Pycapa is running against a version of Librdkafka with SASL support enabled.</p></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-stellar/stellar-3rd-party-example/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-stellar/stellar-3rd-party-example/index.html b/site/current-book/metron-stellar/stellar-3rd-party-example/index.html
index d60a128..4e19e16 100644
--- a/site/current-book/metron-stellar/stellar-3rd-party-example/index.html
+++ b/site/current-book/metron-stellar/stellar-3rd-party-example/index.html
@@ -1,211 +1,108 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-stellar/stellar-3rd-party-example/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Introduction</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Introduction</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Introduction</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Stellar-3rd-party-example</a>
-          </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Introduction</h1>
-<p><a name="Introduction"></a></p>
-<p>This is just a sample 3rd party stellar function (which is used in our unit testing). If Stellar changes its API (e.g. the annotation changes classes), then this will need to be built and the output jar of this (in <tt>target</tt>) will need to be placed in <tt>../stellar-common/src/test/classpath-resources/</tt>.</p>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
             </div>
           </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
 
-    <hr/>
+http://www.apache.org/licenses/LICENSE-2.0
 
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Introduction</h1>
+<p><a name="Introduction"></a></p>
+<p>This is just a sample 3rd party stellar function (which is used in our unit testing).  If Stellar changes its API (e.g. the annotation changes classes), then this will need to be built and the output jar of this (in <tt>target</tt>) will need to be placed in <tt>../stellar-common/src/test/classpath-resources/</tt>.</p>
+        </div>
+      </div>
+    </div>
+    <hr/>
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-stellar/stellar-common/3rdPartyStellar.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-stellar/stellar-common/3rdPartyStellar.html b/site/current-book/metron-stellar/stellar-common/3rdPartyStellar.html
index eb9e054..88d7ac6 100644
--- a/site/current-book/metron-stellar/stellar-common/3rdPartyStellar.html
+++ b/site/current-book/metron-stellar/stellar-common/3rdPartyStellar.html
@@ -1,216 +1,114 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-stellar/stellar-common/3rdPartyStellar.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Custom Stellar Functions</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Custom Stellar Functions</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Custom Stellar Functions</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                                  
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-down"></i>
-        Stellar-common</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>3rdPartyStellar</a>
-          </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-down"></span>Stellar-common</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>3rdPartyStellar</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Custom Stellar Functions</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Custom Stellar Functions</h1>
 <p><a name="Custom_Stellar_Functions"></a></p>
-<p>Metron is fundamentally a programmable, extensible system and Stellar is the extension language. We have some great Stellar functions available out of the box and we&#x2019;ll be adding more over time, but they may not quite scratch quite your particular itch. </p>
-<p>Of course, we&#x2019;d love to have your contribution inside of Metron if you think it general purpose enough, but not every function is general-purpose or it may rely on libraries those licenses aren&#x2019;t acceptable for an Apache project. In that case, then you will be wondering how to add your custom function to a running instance of Metron.</p>
+<p>Metron is fundamentally a programmable, extensible system and Stellar is the extension language.  We have some great Stellar functions available out of the box and we&#x2019;ll be adding more over time, but they may not quite scratch quite your particular itch.</p>
+<p>Of course, we&#x2019;d love to have your contribution inside of Metron if you think it general purpose enough, but not every function is general-purpose or it may rely on libraries those licenses aren&#x2019;t acceptable for an Apache project.  In that case, then you will be wondering how to add your custom function to a running instance of Metron.</p>
 <div class="section">
 <h2><a name="Building_Your_Own_Function"></a>Building Your Own Function</h2>
-<p>Let&#x2019;s say that I need a function that returns the current time in milliseconds since the epoch. I notice that there&#x2019;s nothing like that currently in Metron, so I embark on the adventure of adding it for my cluster.</p>
-<p>I will presume that you have an installed Metron into your local maven repo via <tt>mvn install</tt> . In the future, when we publish to a maven repo, you will not need this. I will depend on 0.4.2 for the purpose of this demonstration</p>
+<p>Let&#x2019;s say that I need a function that returns the current time in milliseconds since the epoch.  I notice that there&#x2019;s nothing like that currently in Metron, so I embark on the adventure of adding it for my cluster.</p>
+<p>I will presume that you have an installed Metron into your local maven repo via <tt>mvn install</tt> .  In the future, when we publish to a maven repo, you will not need this.  I will depend on 0.4.2 for the purpose of this demonstration</p>
 <div class="section">
-<h3><a name="Hack_Hack_Hack"></a>Hack, Hack, Hack</h3>
-<p>I like to use Maven, so we&#x2019;ll use that for this demonstration, but you can use whatever build system that you like. Here&#x2019;s my favorite way to build a project with groupId <tt>com.mycompany.stellar</tt> and artifactId of <tt>tempus</tt> <tt>mvn archetype:create -DgroupId=com.mycompany.stellar -DartifactId=tempus -DarchetypeArtifactId=maven-archetype-quickstart</tt></p>
+<h3><a name="Hack.2C_Hack.2C_Hack"></a>Hack, Hack, Hack</h3>
+<p>I like to use Maven, so we&#x2019;ll use that for this demonstration, but you can use whatever build system that you like.  Here&#x2019;s my favorite way to build a project with groupId <tt>com.mycompany.stellar</tt> and artifactId of <tt>tempus</tt> <tt>mvn archetype:create -DgroupId=com.mycompany.stellar -DartifactId=tempus -DarchetypeArtifactId=maven-archetype-quickstart</tt></p>
 <p>First, we should depend on <tt>metron-common</tt> and we can do that by adjusting the <tt>pom.xml</tt> just created:</p>
 
-<div class="source">
-<div class="source">
-<pre>&lt;project xmlns=&quot;http://maven.apache.org/POM/4.0.0&quot; xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot;
+<div>
+<div>
+<pre class="source">&lt;project xmlns=&quot;http://maven.apache.org/POM/4.0.0&quot; xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot;
          xsi:schemaLocation=&quot;http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd&quot;&gt;
   &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
   
@@ -243,11 +141,12 @@
   &lt;/dependencies&gt;
 &lt;/project&gt;
 </pre></div></div>
+
 <p>Let&#x2019;s add our implementation in <tt>src/main/java/com/mycompany/stellar/TimeFunctions.java</tt> with the following content:</p>
 
-<div class="source">
-<div class="source">
-<pre>package com.notmetron.stellar;
+<div>
+<div>
+<pre class="source">package com.notmetron.stellar;
     
 import org.apache.metron.stellar.dsl.Context;
 import org.apache.metron.stellar.dsl.ParseException;
@@ -276,6 +175,7 @@ public class TimeFunction {
   }
 }
 </pre></div></div>
+
 <p>Now we can build the project via <tt>mvn package</tt> which will create a <tt>target/tempus-1.0-SNAPSHOT.jar</tt> file.</p></div></div>
 <div class="section">
 <h2><a name="Install_the_Function"></a>Install the Function</h2>
@@ -283,26 +183,22 @@ public class TimeFunction {
 <div class="section">
 <h3><a name="Deploy_the_Jar"></a>Deploy the Jar</h3>
 <p>First you need to place the jar in HDFS, if we have it on an access node, one way to do that is:</p>
-
 <ul>
-  
+
 <li><tt>hadoop fs -put tempus-1.0-SNAPSHOT.jar /apps/metron/stellar</tt> This presumes that:</li>
-  
 <li>you&#x2019;ve standardized on <tt>/apps/metron/stellar</tt> as the location for custom jars</li>
-  
 <li>you are running the command from an access node with the <tt>hadoop</tt> command installed</li>
-  
 <li>you are running from a user that has write access to <tt>/apps/metron/stellar</tt></li>
 </ul></div>
 <div class="section">
 <h3><a name="Set_Global_Config"></a>Set Global Config</h3>
 <p>You may not need this if your Metron administrator already has this setup.</p>
-<p>With that dispensed with, we need to ensure that Metron knows to look at that location. We need to ensure that the <tt>stellar.function.paths</tt> property in the <tt>global.json</tt> is in place that makes Metron aware to look for Stellar functions in <tt>/apps/metron/stellar</tt> on HDFS. </p>
+<p>With that dispensed with, we need to ensure that Metron knows to look at that location. We need to ensure that the <tt>stellar.function.paths</tt> property in the <tt>global.json</tt> is in place that makes Metron aware to look for Stellar functions in <tt>/apps/metron/stellar</tt> on HDFS.</p>
 <p>This property looks like, the following for a vagrant install</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;es.clustername&quot;: &quot;metron&quot;,
   &quot;es.ip&quot;: &quot;node1&quot;,
   &quot;es.port&quot;: &quot;9300&quot;,
@@ -310,21 +206,23 @@ public class TimeFunction {
   &quot;stellar.function.paths&quot; : &quot;hdfs://node1:8020/apps/metron/stellar/.*.jar&quot;,
 }
 </pre></div></div>
+
 <p>The <tt>stellar.function.paths</tt> property takes a comma separated list of URIs or URIs with regex expressions at the end. Also, note path is prefaced by the HDFS default name, which, if you do not know, can be found by executing, <tt>hdfs getconf -confKey fs.default.name</tt>, such as</p>
 
-<div class="source">
-<div class="source">
-<pre>[root@node1 ~]# hdfs getconf -confKey fs.default.name
+<div>
+<div>
+<pre class="source">[root@node1 ~]# hdfs getconf -confKey fs.default.name
 hdfs://node1:8020
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Use_the_Function"></a>Use the Function</h3>
 <p>Now that we have deployed the function, if we want to use it, any running topologies that use Stellar will need to be restarted.</p>
 <p>Beyond that, let&#x2019;s take a look at it in the REPL:</p>
 
-<div class="source">
-<div class="source">
-<pre>Stellar, Go!
+<div>
+<div>
+<pre class="source">Stellar, Go!
 Please note that functions are loading lazily in the background and will be unavailable until loaded fully.
 {es.clustername=metron, es.ip=node1, es.port=9300, es.date.format=yyyy.MM.dd.HH, stellar.function.paths=hdfs://node1:8020/apps/metron/stellar/.*.jar, profiler.client.period.duration=1, profiler.client.period.duration.units=MINUTES}
 [Stellar]&gt;&gt;&gt; # Get the help for NOW
@@ -339,23 +237,17 @@ Returns: Timestamp
 1488400515655
 [Stellar]&gt;&gt;&gt; # Looks like I got a timestamp, success!
 </pre></div></div></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[37/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-analytics/metron-profiler-client/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/metron-profiler-client/index.html b/site/current-book/metron-analytics/metron-profiler-client/index.html
index 4fc31d6..d05835a 100644
--- a/site/current-book/metron-analytics/metron-profiler-client/index.html
+++ b/site/current-book/metron-analytics/metron-profiler-client/index.html
@@ -1,233 +1,113 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-analytics/metron-profiler-client/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Metron Profiler Client</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Metron Profiler Client</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron Profiler Client</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                                
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-down"></i>
-        Analytics</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service">
-          <i class="none"></i>
-        Maas-service</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-profiler/index.html" title="Profiler">
-          <i class="none"></i>
-        Profiler</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Profiler-client</a>
-          </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-analytics/metron-statistics/index.html" title="Statistics">
-          <i class="icon-chevron-right"></i>
-        Statistics</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-down"></span>Analytics</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service"><span class="none"></span>Maas-service</a></li>
+    <li><a href="../../metron-analytics/metron-profiler/index.html" title="Profiler"><span class="none"></span>Profiler</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Profiler-client</a></li>
+    <li><a href="../../metron-analytics/metron-statistics/index.html" title="Statistics"><span class="icon-chevron-right"></span>Statistics</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Metron Profiler Client</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron Profiler Client</h1>
 <p><a name="Metron_Profiler_Client"></a></p>
-<p>This project provides a client API for accessing the profiles generated by the <a href="../metron-profiler/index.html">Metron Profiler</a>. This includes both a Java API and Stellar API for accessing the profile data. The primary use case is to extract profile data for use during model scoring.</p>
+<p>This project provides a client API for accessing the profiles generated by the <a href="../metron-profiler/index.html">Metron Profiler</a>.  This includes both a Java API and Stellar API for accessing the profile data.  The primary use case is to extract profile data for use during model scoring.</p>
 <div class="section">
 <h2><a name="Stellar_Client_API"></a>Stellar Client API</h2>
 <div class="section">
 <h3><a name="PROFILE_GET"></a><tt>PROFILE_GET</tt></h3>
-<p>The <tt>PROFILE_GET</tt> command allows you to select all of the profile measurements written. This command takes the following arguments:</p>
+<p>The <tt>PROFILE_GET</tt> command allows you to select all of the profile measurements written.  This command takes the following arguments:</p>
 
-<div class="source">
-<div class="source">
-<pre>REQUIRED:
+<div>
+<div>
+<pre class="source">REQUIRED:
     profile - The name of the profile
     entity - The name of the entity
     periods - The list of profile periods to grab.  These are ProfilePeriod objects.
@@ -238,123 +118,89 @@ OPTIONAL:
     config_overrides - Optional - Map (in curly braces) of name:value pairs, each overriding the global config parameter
             of the same name. Default is the empty Map, meaning no overrides.
 </pre></div></div>
-<p>There is an older calling format where <tt>groups_list</tt> is specified as a sequence of group names, &#x201c;varargs&#x201d; style, instead of a List object. This format is still supported for backward compatibility, but it is deprecated, and it is disallowed if the optional <tt>config_overrides</tt> argument is used.</p>
+
+<p>There is an older calling format where <tt>groups_list</tt> is specified as a sequence of group names, &#x201c;varargs&#x201d; style, instead of a List object.  This format is still supported for backward compatibility, but it is deprecated, and it is disallowed if the optional <tt>config_overrides</tt> argument is used.</p>
 <p>The <tt>periods</tt> field is (likely) the output of another Stellar function which defines the times to include.</p>
 <div class="section">
 <h4><a name="Groups_list_argument"></a>Groups_list argument</h4>
-<p>The <tt>groups_list</tt> argument in the client must exactly correspond to the <a href="../metron-profiler/index.html#groupBy"><tt>groupBy</tt></a> configuration in the profile definition. If <tt>groupBy</tt> was not used in the profile, <tt>groups_list</tt> must be empty in the client. If <tt>groupBy</tt> was used in the profile, then the client <tt>groups_list</tt> is <b>not</b> optional; it must be the same length as the <tt>groupBy</tt> list, and specify exactly one selected group value for each <tt>groupBy</tt> criterion, in the same order. For example:</p>
+<p>The <tt>groups_list</tt> argument in the client must exactly correspond to the <a href="../metron-profiler/index.html#groupBy"><tt>groupBy</tt></a> configuration in the profile definition.  If <tt>groupBy</tt> was not used in the profile, <tt>groups_list</tt> must be empty in the client.  If <tt>groupBy</tt> was used in the profile, then the client <tt>groups_list</tt> is <b>not</b> optional; it must be the same length as the <tt>groupBy</tt> list, and specify exactly one selected group value for each <tt>groupBy</tt> criterion, in the same order.  For example:</p>
 
-<div class="source">
-<div class="source">
-<pre>If in Profile, the groupBy criteria are:  [ &#x201c;DAY_OF_WEEK()&#x201d;, &#x201c;URL_TO_PORT()&#x201d; ]
+<div>
+<div>
+<pre class="source">If in Profile, the groupBy criteria are:  [ &#x201c;DAY_OF_WEEK()&#x201d;, &#x201c;URL_TO_PORT()&#x201d; ]
 Then in PROFILE_GET, an allowed groups value would be:  [ &#x201c;3&#x201d;, &#x201c;8080&#x201d; ]
 which will select only records from Tuesdays with port number 8080.
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Configuration_and_the_config_overrides_argument"></a>Configuration and the config_overrides argument</h4>
-<p>By default, the Profiler creates profiles with a period duration of 15 minutes. This means that data is accumulated, summarized and flushed every 15 minutes. The Client API must also have knowledge of this duration to correctly retrieve the profile data. If the Client is expecting 15 minute periods, it will not be able to read data generated by a Profiler that was configured for 1 hour periods, and will return zero results. </p>
-<p>Similarly, all six Client configuration parameters listed in the table below must match the Profiler configuration parameter settings from the time the profile was created. The period duration and other configuration parameters from the Profiler topology are stored in local filesystem at <tt>$METRON_HOME/config/profiler.properties</tt>. The Stellar Client API can be configured correspondingly by setting the following properties in Metron&#x2019;s global configuration, on local filesystem at <tt>$METRON_HOME/config/zookeeper/global.json</tt>, then uploaded to Zookeeper (at <tt>/metron/topology/global</tt>) by using <tt>zk_load_configs.sh</tt>: </p>
+<p>By default, the Profiler creates profiles with a period duration of 15 minutes. This means that data is accumulated, summarized and flushed every 15 minutes. The Client API must also have knowledge of this duration to correctly retrieve the profile data. If the Client is expecting 15 minute periods, it will not be able to read data generated by a Profiler that was configured for 1 hour periods, and will return zero results.</p>
+<p>Similarly, all six Client configuration parameters listed in the table below must match the Profiler configuration parameter settings from the time the profile was created. The period duration and other configuration parameters from the Profiler topology are stored in local filesystem at <tt>$METRON_HOME/config/profiler.properties</tt>. The Stellar Client API can be configured correspondingly by setting the following properties in Metron&#x2019;s global configuration, on local filesystem at <tt>$METRON_HOME/config/zookeeper/global.json</tt>, then uploaded to Zookeeper (at <tt>/metron/topology/global</tt>) by using <tt>zk_load_configs.sh</tt>:</p>
 
-<div class="source">
-<div class="source">
-<pre>    $ cd $METRON_HOME
+<div>
+<div>
+<pre class="source">    $ cd $METRON_HOME
     $ bin/zk_load_configs.sh -m PUSH -i config/zookeeper/ -z node1:2181
 </pre></div></div>
-<p>Any of these six Client configuration parameters may be overridden at run time using the <tt>config_overrides</tt> Map argument in PROFILE_GET. The primary use case is when historical profiles have been created with a different Profiler configuration than is currently configured, and the analyst needing to access them does not want to change the global Client configuration so as not to disrupt the work of other analysts working with current profiles.</p>
 
+<p>Any of these six Client configuration parameters may be overridden at run time using the <tt>config_overrides</tt> Map argument in PROFILE_GET. The primary use case is when historical profiles have been created with a different Profiler configuration than is currently configured, and the analyst needing to access them does not want to change the global Client configuration so as not to disrupt the work of other analysts working with current profiles.</p>
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Key </th>
-      
-<th>Description </th>
-      
-<th>Required </th>
-      
-<th>Default </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Key                                   </th>
+<th> Description                                                                                                                        </th>
+<th> Required </th>
+<th> Default  </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>profiler.client.period.duration </td>
-      
-<td>The duration of each profile period. This value should be defined along with <tt>profiler.client.period.duration.units</tt>. </td>
-      
-<td>Optional </td>
-      
-<td>15 </td>
-    </tr>
-    
+<td> profiler.client.period.duration       </td>
+<td> The duration of each profile period.  This value should be defined along with <tt>profiler.client.period.duration.units</tt>.             </td>
+<td> Optional </td>
+<td> 15       </td></tr>
 <tr class="a">
-      
-<td>profiler.client.period.duration.units </td>
-      
-<td>The units used to specify the profile period duration. This value should be defined along with <tt>profiler.client.period.duration</tt>. </td>
-      
-<td>Optional </td>
-      
-<td>MINUTES </td>
-    </tr>
-    
+<td> profiler.client.period.duration.units </td>
+<td> The units used to specify the profile period duration.  This value should be defined along with <tt>profiler.client.period.duration</tt>. </td>
+<td> Optional </td>
+<td> MINUTES  </td></tr>
 <tr class="b">
-      
-<td>profiler.client.hbase.table </td>
-      
-<td>The name of the HBase table used to store profile data. </td>
-      
-<td>Optional </td>
-      
-<td>profiler </td>
-    </tr>
-    
+<td> profiler.client.hbase.table           </td>
+<td> The name of the HBase table used to store profile data.                                                                            </td>
+<td> Optional </td>
+<td> profiler </td></tr>
 <tr class="a">
-      
-<td>profiler.client.hbase.column.family </td>
-      
-<td>The name of the HBase column family used to store profile data. </td>
-      
-<td>Optional </td>
-      
-<td>P </td>
-    </tr>
-    
+<td> profiler.client.hbase.column.family   </td>
+<td> The name of the HBase column family used to store profile data.                                                                    </td>
+<td> Optional </td>
+<td> P        </td></tr>
 <tr class="b">
-      
-<td>profiler.client.salt.divisor </td>
-      
-<td>The salt divisor used to store profile data. </td>
-      
-<td>Optional </td>
-      
-<td>1000 </td>
-    </tr>
-    
+<td> profiler.client.salt.divisor          </td>
+<td> The salt divisor used to store profile data.                                                                                       </td>
+<td> Optional </td>
+<td> 1000     </td></tr>
 <tr class="a">
-      
-<td>hbase.provider.impl </td>
-      
-<td>The name of the HBaseTableProvider implementation class. </td>
-      
-<td>Optional </td>
-      
-<td> </td>
-    </tr>
-  </tbody>
+<td> profiler.default.value                </td>
+<td> The default value to be returned if a profile is not written for a given period for a profile and entity.                          </td>
+<td> Optional </td>
+<td> null     </td></tr>
+<tr class="b">
+<td> hbase.provider.impl                   </td>
+<td> The name of the HBaseTableProvider implementation class.                                                                           </td>
+<td> Optional </td>
+<td>          </td></tr>
+</tbody>
 </table></div></div>
 <div class="section">
 <h3><a name="Profile_Selectors"></a>Profile Selectors</h3>
-<p>You will notice that the third argument for <tt>PROFILE_GET</tt> is a list of <tt>ProfilePeriod</tt> objects. This list is expected to be produced by another Stellar function. There are a couple options available.</p>
+<p>You will notice that the third argument for <tt>PROFILE_GET</tt> is a list of <tt>ProfilePeriod</tt> objects.  This list is expected to be produced by another Stellar function.  There are a couple options available.</p>
 <div class="section">
 <h4><a name="PROFILE_FIXED"></a><tt>PROFILE_FIXED</tt></h4>
-<p>The profiler periods associated with a fixed lookback starting from now. These are ProfilePeriod objects.</p>
+<p>The profiler periods associated with a fixed lookback starting from now.  These are ProfilePeriod objects.</p>
 
-<div class="source">
-<div class="source">
-<pre>REQUIRED:
+<div>
+<div>
+<pre class="source">REQUIRED:
     durationAgo - How long ago should values be retrieved from?
     units - The units of 'durationAgo'.
 OPTIONAL:
@@ -363,33 +209,28 @@ OPTIONAL:
 
 e.g. To retrieve all the profiles for the last 5 hours.  PROFILE_GET('profile', 'entity', PROFILE_FIXED(5, 'HOURS'))
 </pre></div></div>
-<p>Note that the <tt>config_overrides</tt> parameter operates exactly as the <tt>config_overrides</tt> argument in <tt>PROFILE_GET</tt>. The only available parameters for override are:</p>
 
+<p>Note that the <tt>config_overrides</tt> parameter operates exactly as the <tt>config_overrides</tt> argument in <tt>PROFILE_GET</tt>. The only available parameters for override are:</p>
 <ul>
-  
+
 <li><tt>profiler.client.period.duration</tt></li>
-  
 <li><tt>profiler.client.period.duration.units</tt></li>
 </ul></div>
 <div class="section">
 <h4><a name="PROFILE_WINDOW"></a><tt>PROFILE_WINDOW</tt></h4>
 <p><tt>PROFILE_WINDOW</tt> is intended to provide a finer-level of control over selecting windows for profiles:</p>
-
 <ul>
-  
+
 <li>Specify windows relative to the data timestamp (see the optional <tt>now</tt> parameter below)</li>
-  
 <li>Specify non-contiguous windows to better handle seasonal data (e.g. the last hour for every day for the last month)</li>
-  
 <li>Specify profile output excluding holidays</li>
-  
 <li>Specify only profile output on a specific day of the week</li>
 </ul>
 <p>It does this by a domain specific language mimicking natural language that defines the windows excluded.</p>
 
-<div class="source">
-<div class="source">
-<pre>REQUIRED:
+<div>
+<div>
+<pre class="source">REQUIRED:
     windowSelector - The statement specifying the window to select.
     now - Optional - The timestamp to use for now.
 OPTIONAL:
@@ -400,273 +241,213 @@ e.g. To retrieve all the measurements written for 'profile' and 'entity' for the
 on the same weekday excluding weekends and US holidays across the last 14 days: 
 PROFILE_GET('profile', 'entity', PROFILE_WINDOW('1 hour window every 24 hours starting from 14 days ago including the current day of the week excluding weekends, holidays:us'))
 </pre></div></div>
-<p>Note that the <tt>config_overrides</tt> parameter operates exactly as the <tt>config_overrides</tt> argument in <tt>PROFILE_GET</tt>. The only available parameters for override are:</p>
 
+<p>Note that the <tt>config_overrides</tt> parameter operates exactly as the <tt>config_overrides</tt> argument in <tt>PROFILE_GET</tt>. The only available parameters for override are:</p>
 <ul>
-  
+
 <li><tt>profiler.client.period.duration</tt></li>
-  
 <li><tt>profiler.client.period.duration.units</tt></li>
 </ul>
 <div class="section">
 <h5><a name="The_Profile_Selector_Language"></a>The Profile Selector Language</h5>
 <p>The domain specific language can be broken into a series of clauses, some optional</p>
-
 <ul>
-  
+
 <li><a href="#Temporal_Window_Width"><span style="color:blue">Total Temporal Duration</span></a> - The total range of time in which windows may be specified</li>
-  
 <li><a href="#InclusionExclusion_specifiers"><span style="color:red">Temporal Window Width</span></a> - How large each temporal window</li>
-  
 <li><a href="#Skip_distance"><span style="color:green">Skip distance</span></a> (optional)- How far to skip between when one window starts and when the next begins</li>
-  
 <li><a href="#InclusionExclusion_specifiers"><span style="color:purple">Inclusion/Exclusion specifiers</span></a> (optional) - The set of specifiers to further filter the window</li>
 </ul>
 <p>One <i>must</i> specify either a total temporal duration or a temporal window width. The remaining clauses are optional. During the course of the following discussion, we will color code the clauses in the examples and link them to the relevant section for more detail.</p>
 <p>From a high level, the language fits the following three forms, which are composed of the clauses above:</p>
-
 <ul>
-  
-<li><a href="#Temporal_Window_Width"><span style="color:red">time_interval WINDOW?</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">(INCLUDING specifier_list)? (EXCLUDING specifier_list)?</span></a></li>
-  
+
+<li><a href="#Temporal_Window_Width"><span style="color:red">time_interval WINDOW?</span></a>  <a href="#InclusionExclusion_specifiers"><span style="color:purple">(INCLUDING specifier_list)? (EXCLUDING specifier_list)?</span></a></li>
 <li><a href="#Temporal_Window_Width"><span style="color:red">time_interval WINDOW?</span></a> <a href="#Skip_distance"><span style="color:green">EVERY time_interval</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">FROM time_interval (TO time_interval)?</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">(INCLUDING specifier_list)? (EXCLUDING specifier_list)?</span></a></li>
-  
 <li><a href="#Total_Temporal_Duration"><span style="color:blue">FROM time_interval (TO time_interval)?</span></a></li>
 </ul>
 <div class="section">
 <h6><a name="Total_Temporal_Duration"></a><span style="color:blue">Total Temporal Duration</span></h6>
-<p>Total temporal duration is specified by a phrase: <tt>FROM time_interval AGO TO time_interval AGO</tt> This indicates the beginning and ending of a time interval. This is an inclusive duration.</p>
-
+<p>Total temporal duration is specified by a phrase: <tt>FROM time_interval AGO TO time_interval AGO</tt> This indicates the beginning and ending of a time interval.  This is an inclusive duration.</p>
 <ul>
-  
+
 <li><tt>FROM</tt> - Can be the words &#x201c;from&#x201d; or &#x201c;starting from&#x201d;</li>
-  
 <li><tt>time_interval</tt> - A time amount followed by a unit (e.g. 1 hour). Fractional amounts are not supported. The unit may be &#x201c;minute&#x201d;, &#x201c;day&#x201d;, &#x201c;hour&#x201d; with any pluralization.</li>
-  
 <li><tt>TO</tt> - Can be the words &#x201c;until&#x201d; or &#x201c;to&#x201d;</li>
-  
 <li><tt>AGO</tt> - Optionally the word &#x201c;ago&#x201d;</li>
 </ul>
-<p>The <tt>TO time_interval AGO</tt> portion is optional. If unspecified then it is expected that the time interval ends now.</p>
-<p>Due to the vagaries of the english language, the from and the to portions, if both specified, are interchangeable with regard to which one specifies the start and which specifies the end. </p>
+<p>The <tt>TO time_interval AGO</tt> portion is optional.  If unspecified then it is expected that the time interval ends now.</p>
+<p>Due to the vagaries of the english language, the from and the to portions, if both specified, are interchangeable with regard to which one specifies the start and which specifies the end.</p>
 <p>In other words &#x201c;<a href="#Total_Temporal_Duration"><span style="color:blue">starting from 1 hour ago to 30 minutes ago</span></a>&#x201d; and &#x201c;<a href="#Total_Temporal_Duration"><span style="color:blue">starting from 30 minutes ago to 1 hour ago</span></a>&#x201d; specify the same temporal duration.</p>
 <p><b>Examples</b></p>
-
 <ul>
-  
+
 <li>A duration starting 1 hour ago and ending now
-  
 <ul>
-    
+
 <li><a href="#Total_Temporal_Duration"><span style="color:blue">from 1 hour ago</span></a></li>
-    
 <li><a href="#Total_Temporal_Duration"><span style="color:blue">from 1 hour</span></a></li>
-    
 <li><a href="#Total_Temporal_Duration"><span style="color:blue">starting from 1 hour ago</span></a></li>
-    
 <li><a href="#Total_Temporal_Duration"><span style="color:blue">starting from 1 hour</span></a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>A duration starting 1 hour ago and ending 30 minutes ago:
-  
 <ul>
-    
+
 <li><a href="#Total_Temporal_Duration"><span style="color:blue">from 1 hour ago until 30 minutes ago</span></a></li>
-    
 <li><a href="#Total_Temporal_Duration"><span style="color:blue">from 30 minutes ago until 1 hour ago</span></a></li>
-    
 <li><a href="#Total_Temporal_Duration"><span style="color:blue">starting from 1 hour ago to 30 minutes ago</span></a></li>
-    
 <li><a href="#Total_Temporal_Duration"><span style="color:blue">starting from 1 hour to 30 minutes</span></a></li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h6><a name="Temporal_Window_Width"></a><span style="color:red">Temporal Window Width</span></h6>
-<p>Temporal window width is the specification of a window. A window is may either repeat within total temporal duration or may fill the total temporal duration. This is an inclusive window. A window is specified by the phrase: <tt>time_interval WINDOW</tt></p>
-
+<p>Temporal window width is the specification of a window. A window is may either repeat within total temporal duration or may fill the total temporal duration.  This is an inclusive window. A window is specified by the phrase: <tt>time_interval WINDOW</tt></p>
 <ul>
-  
-<li><tt>time_interval</tt> - A time amount followed by a unit (e.g. 1 hour). Fractional amounts are not supported. The unit may be &#x201c;minute&#x201d;, &#x201c;day&#x201d;, &#x201c;hour&#x201d; with any pluralization.</li>
-  
+
+<li><tt>time_interval</tt> - A time amount followed by a unit (e.g. 1 hour).  Fractional amounts are not supported. The unit may be &#x201c;minute&#x201d;, &#x201c;day&#x201d;, &#x201c;hour&#x201d; with any pluralization.</li>
 <li><tt>WINDOW</tt> - Optionally the word &#x201c;window&#x201d;</li>
 </ul>
 <p><b>Examples</b></p>
-
 <ul>
-  
+
 <li>A fixed window starting 2 hours ago and going until now
-  
 <ul>
-    
+
 <li><a href="#Temporal_Window_Width"><span style="color:red">2 hour</span></a></li>
-    
 <li><a href="#Temporal_Window_Width"><span style="color:red">2 hours</span></a></li>
-    
 <li><a href="#Temporal_Window_Width"><span style="color:red">2 hours window</span></a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>A repeating 30 minute window starting 2 hours ago and repeating every hour until now. This would result in 2 30-minute wide windows: 2 hours ago and 1 hour ago
-  
 <ul>
-    
+
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 2 hours ago</span></a></li>
-    
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minute windows</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 2 hours ago</span></a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>A repeating 30 minute window starting 2 hours ago and repeating every hour until 30 minutes ago. This would result in 2 30-minute wide windows: 2 hours ago and 1 hour ago
-  
 <ul>
-    
+
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 2 hours ago until 30 minutes ago</span></a></li>
-    
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minutes window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 2 hours ago to 30 minutes ago</span></a></li>
-    
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minutes window</span></a> <a href="#Skip_distance"><span style="color:green">for every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 30 minutes ago to 2 hours ago</span></a></li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h6><a name="Skip_distance"></a><span style="color:green">Skip distance</span></h6>
-<p>Skip distance is the amount of time between temporal window beginnings that the next window starts. It is, in effect, the window period. </p>
+<p>Skip distance is the amount of time between temporal window beginnings that the next window starts. It is, in effect, the window period.</p>
 <p>It is specified by the phrase <tt>EVERY time_interval</tt></p>
-
 <ul>
-  
-<li><tt>time_interval</tt> - A time amount followed by a unit (e.g. 1 hour). Fractional amounts are not supported. The unit may be &#x201c;minute&#x201d;, &#x201c;day&#x201d;, &#x201c;hour&#x201d; with any pluralization.</li>
-  
+
+<li><tt>time_interval</tt> - A time amount followed by a unit (e.g. 1 hour).  Fractional amounts are not supported. The unit may be &#x201c;minute&#x201d;, &#x201c;day&#x201d;, &#x201c;hour&#x201d; with any pluralization.</li>
 <li><tt>EVERY</tt> - The word/phrase &#x201c;every&#x201d; or &#x201c;for every&#x201d;</li>
 </ul>
 <p><b>Examples</b></p>
-
 <ul>
-  
+
 <li>A repeating 30 minute window starting 2 hours ago and repeating every hour until now. This would result in 2 30-minute wide windows: 2 hours ago and 1 hour ago
-  
 <ul>
-    
+
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 2 hours ago </span></a></li>
-    
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minutes window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 2 hours ago </span></a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>A repeating 30 minute window starting 2 hours ago and repeating every hour until 30 minutes ago. This would result in 2 30-minute wide windows: 2 hours ago and 1 hour ago
-  
 <ul>
-    
+
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 2 hours ago until 30 minutes ago</span></a></li>
-    
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minutes window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 2 hours ago to 30 minutes ago</span></a></li>
-    
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minutes window</span></a> <a href="#Skip_distance"><span style="color:green">for every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 30 minutes ago to 2 hours ago</span></a></li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h6><a name="InclusionExclusion_specifiers"></a><span style="color:purple">Inclusion/Exclusion specifiers</span></h6>
+<h6><a name="Inclusion.2FExclusion_specifiers"></a><span style="color:purple">Inclusion/Exclusion specifiers</span></h6>
 <p>Inclusion and Exclusion specifiers operate as filters on the set of windows. They operate on the window beginning timestamp.</p>
-<p>For inclusion specifiers, windows who are passed by <i>any</i> of the set of inclusion specifiers are included.<br />inclusion specifiers. Similarly, windows who are passed by <i>any</i> of the set of exclusion specifiers are excluded. Exclusion specifiers trump inclusion specifiers.</p>
+<p>For inclusion specifiers, windows who are passed by <i>any</i> of the set of inclusion specifiers are included.<br />
+inclusion specifiers. Similarly, windows who are passed by <i>any</i> of the set of exclusion specifiers are excluded. Exclusion specifiers trump inclusion specifiers.</p>
 <p>Specifiers follow one of the following formats depending on if it is an inclusion or exclusion specifier:</p>
-
 <ul>
-  
+
 <li><tt>INCLUSION specifier, specifier, ...</tt>
-  
 <ul>
-    
+
 <li><tt>INCLUSION</tt> can be &#x201c;include&#x201d;, &#x201c;includes&#x201d; or &#x201c;including&#x201d;</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>EXCLUSION specifier, specifier, ...</tt>
-  
 <ul>
-    
+
 <li><tt>EXCLUSION</tt> can be &#x201c;exclude&#x201d;, &#x201c;excludes&#x201d; or &#x201c;excluding&#x201d;</li>
-  </ul></li>
+</ul>
+</li>
 </ul>
 <p>The specifiers are a set of fixed specifiers available as part of the language:</p>
-
 <ul>
-  
+
 <li>Fixed day of week-based specifiers - includes or excludes if the window is on the specified day of the week
-  
 <ul>
-    
+
 <li>&#x201c;monday&#x201d; or &#x201c;mondays&#x201d;</li>
-    
 <li>&#x201c;tuesday&#x201d; or &#x201c;tuesdays&#x201d;</li>
-    
 <li>&#x201c;wednesday&#x201d; or &#x201c;wednesdays&#x201d;</li>
-    
 <li>&#x201c;thursday&#x201d; or &#x201c;thursdays&#x201d;</li>
-    
 <li>&#x201c;friday&#x201d; or &#x201c;fridays&#x201d;</li>
-    
 <li>&#x201c;saturday&#x201d; or &#x201c;saturdays&#x201d;</li>
-    
 <li>&#x201c;sunday&#x201d; or &#x201c;sundays&#x201d;</li>
-    
 <li>&#x201c;weekday&#x201d; or &#x201c;weekdays&#x201d;</li>
-    
 <li>&#x201c;weekend&#x201d; or &quot;&#x201c;weekends&#x201d;</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Relative day of week-based specifiers - includes or excludes based on the day of week relative to now
-  
 <ul>
-    
+
 <li>&#x201c;current day of the week&#x201d;</li>
-    
 <li>&#x201c;current day of week&#x201d;</li>
-    
 <li>&#x201c;this day of the week&#x201d;</li>
-    
 <li>&#x201c;this day of week&#x201d;</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Specified date - includes or excludes based on the specified date
-  
 <ul>
-    
+
 <li>&#x201c;date&#x201d; - Takes up to 2 arguments
-    
 <ul>
-      
+
 <li>The day in <tt>yyyy/MM/dd</tt> format if no second argument is provided</li>
-      
 <li>Optionally the format to specify the first argument in</li>
-      
 <li>Example: <tt>date:2017/12/25</tt> would include or exclude December 25, 2017</li>
-      
 <li>Example: <tt>date:20171225:yyyyMMdd</tt> would include or exclude December 25, 2017</li>
-    </ul></li>
-  </ul></li>
-  
+</ul>
+</li>
+</ul>
+</li>
 <li>Holidays - includes or excludes based on if the window starts during a holiday
-  
 <ul>
-    
+
 <li>&#x201c;holiday&#x201d; or &#x201c;holidays&#x201d;
-    
 <ul>
-      
-<li>Arguments form the jollyday hierarchy of holidays. e.g. &#x201c;us:nyc&#x201d; would be holidays for New York City, USA</li>
-      
+
+<li>Arguments form the jollyday hierarchy of holidays.  e.g. &#x201c;us:nyc&#x201d; would be holidays for New York City, USA</li>
 <li>If none is specified, it will choose based on locale.</li>
-      
 <li>Countries supported are those supported in <a class="externalLink" href="https://github.com/svendiedrichsen/jollyday/tree/master/src/main/resources/holidays">jollyday</a></li>
-      
 <li>Example: <tt>holiday:us:nyc</tt> would be the holidays of New York City, USA</li>
-      
 <li>Example: <tt>holiday:hu</tt> would be the holidays of Hungary</li>
-    </ul></li>
-  </ul></li>
+</ul>
+</li>
+</ul>
+</li>
 </ul>
 <p><b>WARNING: Daylight Savings Time effects</b></p>
-<p>While Universal Time (UTC) is nice and constant, many servers are set to local timezones that enable Daylight Savings Time (DST). This means that twice a year, on DST transition weekends, &#x201c;Sunday&#x201d; is either 23 or 25 hours long. However, durations specified as &#x201c;7 days ago&#x201d; are always interpreted as &#x201c;7*24 hours ago&#x201d;. This can lead to some surprising effects when using days of the week as inclusion or exclusion specifiers.</p>
+<p>While Universal Time (UTC) is nice and constant, many servers are set to local timezones that enable Daylight Savings Time (DST). This means that twice a year, on DST transition weekends, &#x201c;Sunday&#x201d; is either 23 or 25 hours long.  However, durations specified as &#x201c;7 days ago&#x201d; are always interpreted as &#x201c;7*24 hours ago&#x201d;.  This can lead to some surprising effects when using days of the week as inclusion or exclusion specifiers.</p>
 <p>For example, the profile window specified by the phrase &#x201c;30 minute window every 24 hours from 7 days ago&#x201d; will always have 7 thirty-minute intervals, and these will normally occur on 5 weekdays and 2 weekend days. However, if you invoke this window at 12:15am any day during the week following the start of DST, you will get these intervals (supposing you start early on a Wednesday morning):</p>
 
-<div class="source">
-<div class="source">
-<pre>Tuesday 12:15am-12:45am (yesterday)
+<div>
+<div>
+<pre class="source">Tuesday 12:15am-12:45am (yesterday)
 Monday 12:15am-12:45am
 Saturday 11:15pm-11:45pm (skipped Sunday!)
 Friday 11:15pm-11:45pm
@@ -674,61 +455,56 @@ Thursday 11:15pm-11:45pm
 Wednesday 11:15pm-11:45pm
 Tuesday 11:15pm-11:45pm
 </pre></div></div>
-<p>Sunday got skipped over because it was only 23 hours long; that is, there were 24 hours between Saturday 11:15pm and Monday 12:15am. So if you specified &#x201c;excluding weekends&#x201d;, you would get 6 days&#x2019; intervals instead of the expected 5. There are multiple variations on this theme.</p>
-<p>Remember that the underlying time is kept in UTC, so the data is always correct. It is only when attempting to interpret UTC as local time, date, and day, that these confusions may occur. They may be eliminated by setting your server timezone to UTC, or otherwise disabling DST.</p>
+
+<p>Sunday got skipped over because it was only 23 hours long; that is, there were 24 hours between Saturday 11:15pm and Monday 12:15am. So if you specified &#x201c;excluding weekends&#x201d;, you would get 6 days&#x2019; intervals instead of the expected 5.  There are multiple variations on this theme.</p>
+<p>Remember that the underlying time is kept in UTC, so the data is always correct.  It is only when attempting to interpret UTC as local time, date, and day, that these confusions may occur.  They may be eliminated by setting your server timezone to UTC, or otherwise disabling DST.</p>
 <p><b>Examples</b></p>
 <p>Assume these are executed at noon.</p>
-
 <ul>
-  
+
 <li>A 1 hour window for the past 8 &#x2018;current day of the week&#x2019;
-  
 <ul>
-    
+
 <li><a href="#Temporal_Window_Width"><span style="color:red">1 hour window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 56 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">including this day of the week</span></a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>A 1 hour window for the past 8 tuesdays
-  
 <ul>
-    
+
 <li><a href="#Temporal_Window_Width"><span style="color:red">1 hour window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 56 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">including tuesdays</span></a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>A 30 minute window every tuesday at noon starting 14 days ago until now
-  
 <ul>
-    
+
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 14 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">including tuesdays</span></a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>A 30 minute window every day except holidays and weekends at noon starting 14 days ago until now
-  
 <ul>
-    
+
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 14 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">excluding holidays:us, weekends</span></a></li>
-    
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 14 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">including weekdays excluding holidays:us, weekends</span></a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>A 30 minute window at noon every day from 7 days ago including saturdays and excluding weekends. Because exclusions trump inclusions, the following will never yield any windows
-  
 <ul>
-    
+
 <li><a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 7 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">including saturdays excluding weekends</span></a></li>
-  </ul></li>
+</ul>
+</li>
 </ul></div></div></div></div>
 <div class="section">
 <h3><a name="Errors"></a>Errors</h3>
 <p>The most common result of incorrect <tt>PROFILE_GET</tt> arguments or Client configuration parameters is an empty result set, rather than an error. The Client cannot effectively validate the arguments, because the Profiler configuration parameters may be changed and the profile itself does not store them. The person doing the querying must carry forward the knowledge of the Profiler configuration parameters from the time of profile creation, and use corresponding <tt>PROFILE_GET</tt> arguments and Client configuration parameters when querying the data.</p></div>
 <div class="section">
 <h3><a name="Examples"></a>Examples</h3>
-<p>The following are usage examples that show how the Stellar API can be used to read profiles generated by the <a href="../metron-profiler/index.html">Metron Profiler</a>. This API would be used in conjunction with other Stellar functions like <a href="../../metron-stellar/stellar-common/index.html#MAAS_MODEL_APPLY"><tt>MAAS_MODEL_APPLY</tt></a> to perform model scoring on streaming data.</p>
-<p>These examples assume a profile has been defined called &#x2018;snort-alerts&#x2019; that tracks the number of Snort alerts associated with an IP address over time. The profile definition might look similar to the following.</p>
+<p>The following are usage examples that show how the Stellar API can be used to read profiles generated by the <a href="../metron-profiler/index.html">Metron Profiler</a>.  This API would be used in conjunction with other Stellar functions like <a href="../../metron-stellar/stellar-common/index.html#MAAS_MODEL_APPLY"><tt>MAAS_MODEL_APPLY</tt></a> to perform model scoring on streaming data.</p>
+<p>These examples assume a profile has been defined called &#x2018;snort-alerts&#x2019; that tracks the number of Snort alerts associated with an IP address over time.  The profile definition might look similar to the following.</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;profiles&quot;: [
     {
       &quot;profile&quot;: &quot;snort-alerts&quot;,
@@ -740,46 +516,52 @@ Tuesday 11:15pm-11:45pm
   ]
 }
 </pre></div></div>
-<p>During model scoring the entity being scored, in this case a particular IP address, will be known. The following examples shows how this profile data might be retrieved. Retrieve all values of &#x2018;snort-alerts&#x2019; from &#x2018;10.0.0.1&#x2019; over the past 4 hours.</p>
 
-<div class="source">
-<div class="source">
-<pre>PROFILE_GET('snort-alerts', '10.0.0.1', PROFILE_FIXED(4, 'HOURS'))
+<p>During model scoring the entity being scored, in this case a particular IP address, will be known.  The following examples shows how this profile data might be retrieved. Retrieve all values of &#x2018;snort-alerts&#x2019; from &#x2018;10.0.0.1&#x2019; over the past 4 hours.</p>
+
+<div>
+<div>
+<pre class="source">PROFILE_GET('snort-alerts', '10.0.0.1', PROFILE_FIXED(4, 'HOURS'))
 </pre></div></div>
+
 <p>Retrieve all values of &#x2018;snort-alerts&#x2019; from &#x2018;10.0.0.1&#x2019; over the past 2 days.</p>
 
-<div class="source">
-<div class="source">
-<pre>PROFILE_GET('snort-alerts', '10.0.0.1', PROFILE_FIXED(2, 'DAYS'))
+<div>
+<div>
+<pre class="source">PROFILE_GET('snort-alerts', '10.0.0.1', PROFILE_FIXED(2, 'DAYS'))
 </pre></div></div>
+
 <p>If the profile had been defined to group the data by weekday versus weekend, then the following example would apply:</p>
 <p>Retrieve all values of &#x2018;snort-alerts&#x2019; from &#x2018;10.0.0.1&#x2019; that occurred on &#x2018;weekdays&#x2019; over the past 30 days.</p>
 
-<div class="source">
-<div class="source">
-<pre>PROFILE_GET('snort-alerts', '10.0.0.1', PROFILE_FIXED(30, 'DAYS'), ['weekdays'] )
+<div>
+<div>
+<pre class="source">PROFILE_GET('snort-alerts', '10.0.0.1', PROFILE_FIXED(30, 'DAYS'), ['weekdays'] )
 </pre></div></div>
-<p>The client may need to use a configuration different from the current Client configuration settings. For example, perhaps you are on a cluster shared with other analysts, and need to access a profile that was constructed 2 months ago using different period duration, while they are accessing more recent profiles constructed with the currently configured period duration. For this situation, you may use the <tt>config_overrides</tt> argument:</p>
+
+<p>The client may need to use a configuration different from the current Client configuration settings.  For example, perhaps you are on a cluster shared with other analysts, and need to access a profile that was constructed 2 months ago using different period duration, while they are accessing more recent profiles constructed with the currently configured period duration.  For this situation, you may use the <tt>config_overrides</tt> argument:</p>
 <p>Retrieve all values of &#x2018;snort-alerts&#x2019; from &#x2018;10.0.0.1&#x2019; over the past 2 days, with no <tt>groupBy</tt>, and overriding the usual global client configuration parameters for window duration.</p>
 
-<div class="source">
-<div class="source">
-<pre>PROFILE_GET('profile1', 'entity1', PROFILE_FIXED(2, 'DAYS', {'profiler.client.period.duration' : '2', 'profiler.client.period.duration.units' : 'MINUTES'}), [])
+<div>
+<div>
+<pre class="source">PROFILE_GET('profile1', 'entity1', PROFILE_FIXED(2, 'DAYS', {'profiler.client.period.duration' : '2', 'profiler.client.period.duration.units' : 'MINUTES'}), [])
 </pre></div></div>
+
 <p>Retrieve all values of &#x2018;snort-alerts&#x2019; from &#x2018;10.0.0.1&#x2019; that occurred on &#x2018;weekdays&#x2019; over the past 30 days, overriding the usual global client configuration parameters for window duration.</p>
 
-<div class="source">
-<div class="source">
-<pre>PROFILE_GET('profile1', 'entity1', PROFILE_FIXED(30, 'DAYS', {'profiler.client.period.duration' : '2', 'profiler.client.period.duration.units' : 'MINUTES'}), ['weekdays'] )
-</pre></div></div></div></div>
+<div>
+<div>
+<pre class="source">PROFILE_GET('profile1', 'entity1', PROFILE_FIXED(30, 'DAYS', {'profiler.client.period.duration' : '2', 'profiler.client.period.duration.units' : 'MINUTES'}), ['weekdays'] )
+</pre></div></div>
+</div></div>
 <div class="section">
 <h2><a name="Getting_Started"></a>Getting Started</h2>
-<p>These instructions step through the process of using the Stellar Client API on a live cluster. These instructions assume that the &#x2018;Getting Started&#x2019; instructions included with the <a href="../metron-profiler/index.html">Metron Profiler</a> have been followed. This will create a Profile called &#x2018;test&#x2019; whose data will be retrieved with the Stellar Client API.</p>
-<p>To validate that everything is working, login to the server hosting Metron. We will use the Stellar Shell to replicate the execution environment of Stellar running in a Storm topology, like Metron&#x2019;s Parser or Enrichment topology. Replace &#x2018;node1:2181&#x2019; with the URL to a Zookeeper Broker. </p>
+<p>These instructions step through the process of using the Stellar Client API on a live cluster.  These instructions assume that the &#x2018;Getting Started&#x2019; instructions included with the <a href="../metron-profiler/index.html">Metron Profiler</a> have been followed.  This will create a Profile called &#x2018;test&#x2019; whose data will be retrieved with the Stellar Client API.</p>
+<p>To validate that everything is working, login to the server hosting Metron.  We will use the Stellar Shell to replicate the execution environment of Stellar running in a Storm topology, like Metron&#x2019;s Parser or Enrichment topology.  Replace &#x2018;node1:2181&#x2019; with the URL to a Zookeeper Broker.</p>
 
-<div class="source">
-<div class="source">
-<pre>[root@node1 0.4.2]# bin/stellar -z node1:2181
+<div>
+<div>
+<pre class="source">[root@node1 0.4.2]# bin/stellar -z node1:2181
 Stellar, Go!
 Please note that functions are loading lazily in the background and will be unavailable until loaded fully.
 {es.clustername=metron, es.ip=node1, es.port=9300, es.date.format=yyyy.MM.dd.HH}
@@ -805,20 +587,21 @@ Returns: The selected profile measurements.
 [Stellar]&gt;&gt;&gt; PROFILE_GET('test','192.168.138.158', 1, 'HOURS')
 [12078.0, 8921.0, 12131.0]
 </pre></div></div>
+
 <p>The client API call above has retrieved the past hour of the &#x2018;test&#x2019; profile for the entity &#x2018;192.168.138.158&#x2019;.</p></div>
 <div class="section">
 <h2><a name="Developing_Profiles"></a>Developing Profiles</h2>
-<p>Troubleshooting issues when programming against a live stream of data can be difficult. The Stellar REPL is a powerful tool to help work out the kinds of enrichments and transformations that are needed. The Stellar REPL can also be used to help when developing profiles for the Profiler.</p>
+<p>Troubleshooting issues when programming against a live stream of data can be difficult.  The Stellar REPL is a powerful tool to help work out the kinds of enrichments and transformations that are needed.  The Stellar REPL can also be used to help when developing profiles for the Profiler.</p>
 <p>Follow these steps in the Stellar REPL to see how it can be used to help create profiles.</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Take a first pass at defining your profile. As an example, in the editor copy/paste the basic &#x201c;Hello, World&#x201d; profile below.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; conf := SHELL_EDIT()
+
+<p>Take a first pass at defining your profile.  As an example, in the editor copy/paste the basic &#x201c;Hello, World&#x201d; profile below.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; conf := SHELL_EDIT()
 [Stellar]&gt;&gt;&gt; conf
 {
   &quot;profiles&quot;: [
@@ -832,26 +615,29 @@ Returns: The selected profile measurements.
     }
   ]
 }
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Initialize the Profiler.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; profiler := PROFILER_INIT(conf)
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; profiler := PROFILER_INIT(conf)
 [Stellar]&gt;&gt;&gt; profiler
 Profiler{1 profile(s), 0 messages(s), 0 route(s)}
 </pre></div></div>
-<p>The profiler itself will show the number of profiles defined, the number of messages applied, and the number of routes taken. </p>
-<p>A route is defined when a message is applied to a specific profile. If a message is applied and not needed by any profile, then there are no routes. If a message is needed by one profile, then one route has been defined. If a message is needed by two profiles, then two routes have been defined. </p></li>
-  
+
+<p>The profiler itself will show the number of profiles defined, the number of messages applied, and the number of routes taken.</p>
+<p>A route is defined when a message is applied to a specific profile.  If a message is applied and not needed by any profile, then there are no routes.  If a message is needed by one profile, then one route has been defined.  If a message is needed by two profiles, then two routes have been defined.</p>
+</li>
 <li>
-<p>Create a message to simulate the type of telemetry that you expect to be profiled. As an example, in the editor copy/paste the JSON below.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; message := SHELL_EDIT()
+
+<p>Create a message to simulate the type of telemetry that you expect to be profiled.   As an example, in the editor copy/paste the JSON below.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; message := SHELL_EDIT()
 [Stellar]&gt;&gt;&gt; message
 {
   &quot;ip_src_addr&quot;: &quot;10.0.0.1&quot;,
@@ -859,62 +645,62 @@ Profiler{1 profile(s), 0 messages(s), 0 route(s)}
   &quot;length&quot;: &quot;10&quot;,
   &quot;bytes_in&quot;: &quot;234&quot;
 }
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Apply some telemetry messages to your profiles. The following applies the same message 3 times.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; PROFILER_APPLY(message, profiler)
+
+<p>Apply some telemetry messages to your profiles.  The following applies the same message 3 times.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; PROFILER_APPLY(message, profiler)
 Profiler{1 profile(s), 1 messages(s), 1 route(s)}
 </pre></div></div>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; PROFILER_APPLY(message, profiler)
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; PROFILER_APPLY(message, profiler)
 Profiler{1 profile(s), 2 messages(s), 2 route(s)}
 </pre></div></div>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; PROFILER_APPLY(message, profiler)
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; PROFILER_APPLY(message, profiler)
 Profiler{1 profile(s), 3 messages(s), 3 route(s)}
 </pre></div></div>
-<p>It is also possible to apply multiple messages at once. This is useful when testing against a larger set of data. To do this, create a string that contains a JSON array of messages and pass that to the <tt>PROFILER_APPLY</tt> function.</p></li>
-  
+
+<p>It is also possible to apply multiple messages at once.  This is useful when testing against a larger set of data.  To do this, create a string that contains a JSON array of messages and pass that to the <tt>PROFILER_APPLY</tt> function.</p>
+</li>
 <li>
-<p>Flush the Profiler to see what has been calculated. A flush is what occurs at the end of each 15 minute period in the Profiler. The result is a list of profile measurements. Each measurement is a map containing detailed information about the profile data that has been generated.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; values := PROFILER_FLUSH(profiler)
+
+<p>Flush the Profiler to see what has been calculated.  A flush is what occurs at the end of each 15 minute period in the Profiler.  The result is a list of profile measurements.  Each measurement is a map containing detailed information about the profile data that has been generated.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; values := PROFILER_FLUSH(profiler)
 [Stellar]&gt;&gt;&gt; values
 [{period={duration=900000, period=1669628, start=1502665200000, end=1502666100000}, 
    profile=hello-world, groups=[], value=3, entity=10.0.0.1}]
 </pre></div></div>
-<p>This profile simply counts the number of messages by IP source address. Notice that the value is &#x2018;3&#x2019; for the entity &#x2018;10.0.0.1&#x2019; as we applied 3 messages with an &#x2018;ip_src_addr&#x2019; of &#x2018;10.0.0.1&#x2019;. There will always be one measurement for each [profile, entity] pair.</p></li>
-  
+
+<p>This profile simply counts the number of messages by IP source address.  Notice that the value is &#x2018;3&#x2019; for the entity &#x2018;10.0.0.1&#x2019; as we applied 3 messages with an &#x2018;ip_src_addr&#x2019; of &#x2018;10.0.0.1&#x2019;.  There will always be one measurement for each [profile, entity] pair.</p>
+</li>
 <li>
-<p>If you are unhappy with the data that has been generated, then &#x2018;wash, rinse and repeat&#x2019; this process. Once you are happy with the profile that was created, follow the <a href="../metron-profiler/index.html#Getting_Started">Getting Started</a> guide to use the profile against your live, streaming data in a Metron cluster.</p></li>
-</ol></div>
-                  </div>
-            </div>
-          </div>
 
+<p>If you are unhappy with the data that has been generated, then &#x2018;wash, rinse and repeat&#x2019; this process.  Once you are happy with the profile that was created, follow the <a href="../metron-profiler/index.html#Getting_Started">Getting Started</a> guide to use the profile against your live, streaming data in a Metron cluster.</p>
+</li>
+</ol></div>
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[14/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-parsers/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-parsers/index.html b/site/current-book/metron-platform/metron-parsers/index.html
index 23955ac..807a24e 100644
--- a/site/current-book/metron-platform/metron-parsers/index.html
+++ b/site/current-book/metron-platform/metron-parsers/index.html
@@ -1,379 +1,211 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-parsers/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Parsers</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Parsers</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Parsers</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                            
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Parsers</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/parser-testing.html" title="parser-testing">
-          <i class="none"></i>
-        parser-testing</a>
-            </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Parsers</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/metron-parsers/3rdPartyParser.html" title="3rdPartyParser"><span class="none"></span>3rdPartyParser</a></li>
+    <li><a href="../../metron-platform/metron-parsers/parser-testing.html" title="parser-testing"><span class="none"></span>parser-testing</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Parsers</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Parsers</h1>
 <p><a name="Parsers"></a></p>
 <div class="section">
 <h2><a name="Introduction"></a>Introduction</h2>
-<p>Parsers are pluggable components which are used to transform raw data (textual or raw bytes) into JSON messages suitable for downstream enrichment and indexing. </p>
+<p>Parsers are pluggable components which are used to transform raw data (textual or raw bytes) into JSON messages suitable for downstream enrichment and indexing.</p>
 <p>There are two general types types of parsers:</p>
-
 <ul>
-  
-<li>A parser written in Java which conforms to the <tt>MessageParser</tt> interface. This kind of parser is optimized for speed and performance and is built for use with higher velocity topologies. These parsers are not easily modifiable and in order to make changes to them the entire topology need to be recompiled.</li>
-  
-<li>A general purpose parser. This type of parser is primarily designed for lower-velocity topologies or for quickly standing up a parser for a new telemetry before a permanent Java parser can be written for it. As of the time of this writing, we have:
-  
+
+<li>A parser written in Java which conforms to the <tt>MessageParser</tt> interface.  This kind of parser is optimized for speed and performance and is built for use with higher velocity topologies.  These parsers are not easily modifiable and in order to make changes to them the entire topology need to be recompiled.</li>
+<li>A general purpose parser.  This type of parser is primarily designed for lower-velocity topologies or for quickly standing up a parser for a new telemetry before a permanent Java parser can be written for it.  As of the time of this writing, we have:
 <ul>
-    
+
 <li>Grok parser: <tt>org.apache.metron.parsers.GrokParser</tt> with possible <tt>parserConfig</tt> entries of
-    
 <ul>
-      
+
 <li><tt>grokPath</tt> : The path in HDFS (or in the Jar) to the grok statement</li>
-      
 <li><tt>patternLabel</tt> : The pattern label to use from the grok statement</li>
-      
 <li><tt>timestampField</tt> : The field to use for timestamp</li>
-      
 <li><tt>timeFields</tt> : A list of fields to be treated as time</li>
-      
 <li><tt>dateFormat</tt> : The date format to use to parse the time fields</li>
-      
 <li><tt>timezone</tt> : The timezone to use. <tt>UTC</tt> is default.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>CSV Parser: <tt>org.apache.metron.parsers.csv.CSVParser</tt> with possible <tt>parserConfig</tt> entries of
-    
 <ul>
-      
-<li><tt>timestampFormat</tt> : The date format of the timestamp to use. If unspecified, the parser assumes the timestamp is ms since unix epoch.</li>
-      
-<li><tt>columns</tt> : A map of column names you wish to extract from the CSV to their offsets (e.g. <tt>{ 'name' : 1, 'profession' : 3}</tt> would be a column map for extracting the 2nd and 4th columns from a CSV)</li>
-      
+
+<li><tt>timestampFormat</tt> : The date format of the timestamp to use.  If unspecified, the parser assumes the timestamp is ms since unix epoch.</li>
+<li><tt>columns</tt> : A map of column names you wish to extract from the CSV to their offsets (e.g. <tt>{ 'name' : 1, 'profession' : 3}</tt>  would be a column map for extracting the 2nd and 4th columns from a CSV)</li>
 <li><tt>separator</tt> : The column separator, <tt>,</tt> by default.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>JSON Map Parser: <tt>org.apache.metron.parsers.json.JSONMapParser</tt> with possible <tt>parserConfig</tt> entries of
-    
 <ul>
-      
-<li><tt>mapStrategy</tt> : A strategy to indicate how to handle multi-dimensional Maps. This is one of
-      
+
+<li><tt>mapStrategy</tt> : A strategy to indicate how to handle multi-dimensional Maps.  This is one of
 <ul>
-        
+
 <li><tt>DROP</tt> : Drop fields which contain maps</li>
-        
-<li><tt>UNFOLD</tt> : Unfold inner maps. So <tt>{ &quot;foo&quot; : { &quot;bar&quot; : 1} }</tt> would turn into <tt>{&quot;foo.bar&quot; : 1}</tt></li>
-        
+<li><tt>UNFOLD</tt> : Unfold inner maps.  So <tt>{ &quot;foo&quot; : { &quot;bar&quot; : 1} }</tt> would turn into <tt>{&quot;foo.bar&quot; : 1}</tt></li>
 <li><tt>ALLOW</tt> : Allow multidimensional maps</li>
-        
 <li><tt>ERROR</tt> : Throw an error when a multidimensional map is encountered</li>
-      </ul></li>
-      
+</ul>
+</li>
+<li><tt>jsonpQuery</tt> : A <a href="#json_path">JSON Path</a> query string. If present, the result of the JSON Path query should be a list of messages. This is useful if you have a JSON document which contains a list or array of messages embedded in it, and you do not have another means of splitting the message.</li>
 <li>A field called <tt>timestamp</tt> is expected to exist and, if it does not, then current time is inserted.</li>
-    </ul></li>
-  </ul></li>
+</ul>
+</li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
+<h2><a name="Parser_Error_Routing"></a>Parser Error Routing</h2>
+<p>Currently, we have a few mechanisms for either deferring processing of messages or marking messages as invalid.</p>
+<div class="section">
+<h3><a name="Invalidation_Errors"></a>Invalidation Errors</h3>
+<p>There are two reasons a message will be marked as invalid:</p>
+<ul>
+
+<li>Fail <a href="../metron-common/index.html#validation-framework">global validation</a></li>
+<li>Fail the parser&#x2019;s validate function (generally that means to not have a <tt>timestamp</tt> field or a <tt>original_string</tt> field.</li>
+</ul>
+<p>Those messages which are marked as invalid are sent to the error queue with an indication that they are invalid in the error message.</p></div>
+<div class="section">
+<h3><a name="Parser_Errors"></a>Parser Errors</h3>
+<p>Errors, which are defined as unexpected exceptions happening during the parse, are sent along to the error queue with a message indicating that there was an error in parse along with a stacktrace.  This is to distinguish from the invalid messages.</p></div></div>
+<div class="section">
+<h2><a name="Filtered"></a>Filtered</h2>
+<p>One can also filter a message by specifying a <tt>filterClassName</tt> in the parser config.  Filtered messages are just dropped rather than passed through.</p></div>
+<div class="section">
 <h2><a name="Parser_Architecture"></a>Parser Architecture</h2>
 <p><img src="../../images/parser_arch.png" alt="Architecture" /></p>
-<p>Data flows through the parser bolt via kafka and into the <tt>enrichments</tt> topology in kafka. Errors are collected with the context of the error (e.g. stacktrace) and original message causing the error and sent to an <tt>error</tt> queue. Invalid messages as determined by global validation functions are also treated as errors and sent to an <tt>error</tt> queue. </p></div>
+<p>Data flows through the parser bolt via kafka and into the <tt>enrichments</tt> topology in kafka.  Errors are collected with the context of the error (e.g. stacktrace) and original message causing the error and sent to an <tt>error</tt> queue.  Invalid messages as determined by global validation functions are also treated as errors and sent to an <tt>error</tt> queue.</p></div>
 <div class="section">
 <h2><a name="Message_Format"></a>Message Format</h2>
-<p>All Metron messages follow a specific format in order to ingest a message. If a message does not conform to this format it will be dropped and put onto an error queue for further examination. The message must be of a JSON format and must have a JSON tag message like so:</p>
+<p>All Metron messages follow a specific format in order to ingest a message.  If a message does not conform to this format it will be dropped and put onto an error queue for further examination.  The message must be of a JSON format and must have a JSON tag message like so:</p>
 
-<div class="source">
-<div class="source">
-<pre>{&quot;message&quot; : message content}
+<div>
+<div>
+<pre class="source">{&quot;message&quot; : message content}
 </pre></div></div>
-<p>Where appropriate there is also a standardization around the 5-tuple JSON fields. This is done so the topology correlation engine further down stream can correlate messages from different topologies by these fields. We are currently working on expanding the message standardization beyond these fields, but this feature is not yet availabe. The standard field names are as follows:</p>
 
+<p>Where appropriate there is also a standardization around the 5-tuple JSON fields.  This is done so the topology correlation engine further down stream can correlate messages from different topologies by these fields.  We are currently working on expanding the message standardization beyond these fields, but this feature is not yet availabe.  The standard field names are as follows:</p>
 <ul>
-  
+
 <li>ip_src_addr: layer 3 source IP</li>
-  
 <li>ip_dst_addr: layer 3 dest IP</li>
-  
 <li>ip_src_port: layer 4 source port</li>
-  
 <li>ip_dst_port: layer 4 dest port</li>
-  
 <li>protocol: layer 4 protocol</li>
-  
 <li>timestamp (epoch)</li>
-  
 <li>original_string: A human friendly string representation of the message</li>
 </ul>
-<p>The timestamp and original_string fields are madatory. The remaining standard fields are optional. If any of the optional fields are not applicable then the field should be left out of the JSON.</p>
+<p>The timestamp and original_string fields are madatory. The remaining standard fields are optional.  If any of the optional fields are not applicable then the field should be left out of the JSON.</p>
 <p>So putting it all together a typical Metron message with all 5-tuple fields present would look like the following:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
 &quot;message&quot;: 
 {&quot;ip_src_addr&quot;: xxxx, 
 &quot;ip_dst_addr&quot;: xxxx, 
@@ -384,52 +216,43 @@
 &quot;additional-field 1&quot;: xxx,
 }
 }
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Global_Configuration"></a>Global Configuration</h2>
 <p>There are a few properties which can be managed in the global configuration that have pertinence to parsers and parsing in general.</p>
 <div class="section">
 <h3><a name="parser.error.topic"></a><tt>parser.error.topic</tt></h3>
 <p>The topic where messages which were unable to be parsed due to error are sent. Error messages will be indexed under a sensor type of <tt>error</tt> and the messages will have the following fields:</p>
-
 <ul>
-  
+
 <li><tt>sensor.type</tt>: <tt>error</tt></li>
-  
 <li><tt>failed_sensor_type</tt> : The sensor type of the message which wasn&#x2019;t able to be parsed</li>
-  
 <li><tt>error_type</tt> : The error type, in this case <tt>parser</tt>.</li>
-  
 <li><tt>stack</tt> : The stack trace of the error</li>
-  
 <li><tt>hostname</tt> : The hostname of the node where the error happened</li>
-  
 <li><tt>raw_message</tt> : The raw message in string form</li>
-  
 <li><tt>raw_message_bytes</tt> : The raw message bytes</li>
-  
 <li><tt>error_hash</tt> : A hash of the error message</li>
 </ul></div></div>
 <div class="section">
 <h2><a name="Parser_Configuration"></a>Parser Configuration</h2>
 <p>The configuration for the various parser topologies is defined by JSON documents stored in zookeeper.</p>
 <p>The document is structured in the following way</p>
-
 <ul>
-  
+
 <li><tt>parserClassName</tt> : The fully qualified classname for the parser to be used.</li>
-  
-<li><tt>filterClassName</tt> : The filter to use. This may be a fully qualified classname of a Class that implements the <tt>org.apache.metron.parsers.interfaces.MessageFilter&lt;JSONObject&gt;</tt> interface. Message Filters are intended to allow the user to ignore a set of messages via custom logic. The existing implementations are:
-  
+<li><tt>filterClassName</tt> : The filter to use.  This may be a fully qualified classname of a Class that implements the <tt>org.apache.metron.parsers.interfaces.MessageFilter&lt;JSONObject&gt;</tt> interface.  Message Filters are intended to allow the user to ignore a set of messages via custom logic.  The existing implementations are:
 <ul>
-    
-<li><tt>STELLAR</tt> : Allows you to apply a stellar statement which returns a boolean, which will pass every message for which the statement returns <tt>true</tt>. The Stellar statement that is to be applied is specified by the <tt>filter.query</tt> property in the <tt>parserConfig</tt>. Example Stellar Filter which includes messages which contain a the <tt>field1</tt> field:</li>
-  </ul></li>
+
+<li><tt>STELLAR</tt> : Allows you to apply a stellar statement which returns a boolean, which will pass every message for which the statement returns <tt>true</tt>.  The Stellar statement that is to be applied is specified by the <tt>filter.query</tt> property in the <tt>parserConfig</tt>. Example Stellar Filter which includes messages which contain a the <tt>field1</tt> field:</li>
+</ul>
+</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>   {
+<div>
+<div>
+<pre class="source">   {
     &quot;filterClassName&quot; : &quot;STELLAR&quot;
    ,&quot;parserConfig&quot; : {
     &quot;filter.query&quot; : &quot;exists(field1)&quot;
@@ -438,127 +261,119 @@
 </pre></div></div>
 
 <ul>
-  
-<li><tt>sensorTopic</tt> : The kafka topic to send the parsed messages to. If the topic is prefixed and suffixed by <tt>/</tt> then it is assumed to be a regex and will match any topic matching the pattern (e.g. <tt>/bro.*/</tt> would match <tt>bro_cust0</tt>, <tt>bro_cust1</tt> and <tt>bro_cust2</tt>)</li>
-  
-<li><tt>readMetadata</tt> : Boolean indicating whether to read metadata or not (<tt>false</tt> by default). See below for a discussion about metadata.</li>
-  
-<li><tt>mergeMetadata</tt> : Boolean indicating whether to merge metadata with the message or not (<tt>false</tt> by default). See below for a discussion about metadata.</li>
-  
+
+<li><tt>sensorTopic</tt> : The kafka topic to send the parsed messages to.  If the topic is prefixed and suffixed by <tt>/</tt> then it is assumed to be a regex and will match any topic matching the pattern (e.g. <tt>/bro.*/</tt> would match <tt>bro_cust0</tt>, <tt>bro_cust1</tt> and <tt>bro_cust2</tt>)</li>
+<li><tt>readMetadata</tt> : Boolean indicating whether to read metadata or not (<tt>false</tt> by default).  See below for a discussion about metadata.</li>
+<li><tt>mergeMetadata</tt> : Boolean indicating whether to merge metadata with the message or not (<tt>false</tt> by default).  See below for a discussion about metadata.</li>
 <li><tt>parserConfig</tt> : A JSON Map representing the parser implementation specific configuration.</li>
-  
 <li><tt>fieldTransformations</tt> : An array of complex objects representing the transformations to be done on the message generated from the parser before writing out to the kafka topic.</li>
-  
-<li><tt>spoutParallelism</tt> : The kafka spout parallelism (default to <tt>1</tt>). This can be overridden on the command line.</li>
-  
+<li><tt>spoutParallelism</tt> : The kafka spout parallelism (default to <tt>1</tt>).  This can be overridden on the command line.</li>
 <li><tt>spoutNumTasks</tt> : The number of tasks for the spout (default to <tt>1</tt>). This can be overridden on the command line.</li>
-  
 <li><tt>parserParallelism</tt> : The parser bolt parallelism (default to <tt>1</tt>). This can be overridden on the command line.</li>
-  
 <li><tt>parserNumTasks</tt> : The number of tasks for the parser bolt (default to <tt>1</tt>). This can be overridden on the command line.</li>
-  
 <li><tt>errorWriterParallelism</tt> : The error writer bolt parallelism (default to <tt>1</tt>). This can be overridden on the command line.</li>
-  
 <li><tt>errorWriterNumTasks</tt> : The number of tasks for the error writer bolt (default to <tt>1</tt>). This can be overridden on the command line.</li>
-  
 <li><tt>numWorkers</tt> : The number of workers to use in the topology (default is the storm default of <tt>1</tt>).</li>
-  
 <li><tt>numAckers</tt> : The number of acker executors to use in the topology (default is the storm default of <tt>1</tt>).</li>
-  
 <li><tt>spoutConfig</tt> : A map representing a custom spout config (this is a map). This can be overridden on the command line.</li>
-  
-<li><tt>securityProtocol</tt> : The security protocol to use for reading from kafka (this is a string). This can be overridden on the command line and also specified in the spout config via the <tt>security.protocol</tt> key. If both are specified, then they are merged and the CLI will take precedence.</li>
-  
-<li><tt>stormConfig</tt> : The storm config to use (this is a map). This can be overridden on the command line. If both are specified, they are merged with CLI properties taking precedence.</li>
+<li><tt>securityProtocol</tt> : The security protocol to use for reading from kafka (this is a string).  This can be overridden on the command line and also specified in the spout config via the <tt>security.protocol</tt> key.  If both are specified, then they are merged and the CLI will take precedence.</li>
+<li><tt>stormConfig</tt> : The storm config to use (this is a map).  This can be overridden on the command line.  If both are specified, they are merged with CLI properties taking precedence.</li>
+<li><tt>cacheConfig</tt> : Cache config for stellar field transformations.   This configures a least frequently used cache.  This is a map with the following keys.  If not explicitly configured (the default), then no cache will be used.
+<ul>
+
+<li><tt>stellar.cache.maxSize</tt> - The maximum number of elements in the cache. Default is to not use a cache.</li>
+<li><tt>stellar.cache.maxTimeRetain</tt> - The maximum amount of time an element is kept in the cache (in minutes). Default is to not use a cache.</li>
+</ul>
+<p>Example of a cache config to contain at max <tt>20000</tt> stellar expressions for at most <tt>20</tt> minutes.:</p>
+</li>
 </ul>
-<p>The <tt>fieldTransformations</tt> is a complex object which defines a transformation which can be done to a message. This transformation can </p>
 
+<div>
+<div>
+<pre class="source">{
+  &quot;cacheConfig&quot; : {
+    &quot;stellar.cache.maxSize&quot; : 20000,
+    &quot;stellar.cache.maxTimeRetain&quot; : 20
+  }
+}
+</pre></div></div>
+
+<p>The <tt>fieldTransformations</tt> is a complex object which defines a transformation which can be done to a message.  This transformation can</p>
 <ul>
-  
+
 <li>Modify existing fields to a message</li>
-  
 <li>Add new fields given the values of existing fields of a message</li>
-  
 <li>Remove existing fields of a message</li>
 </ul>
 <div class="section">
 <h3><a name="Metadata"></a>Metadata</h3>
-<p>Metadata is a useful thing to send to Metron and use during enrichment or threat intelligence.<br />Consider the following scenarios:</p>
-
+<p>Metadata is a useful thing to send to Metron and use during enrichment or threat intelligence.<br />
+Consider the following scenarios:</p>
 <ul>
-  
+
 <li>You have multiple telemetry sources of the same type that you want to
-  
 <ul>
-    
+
 <li>ensure downstream analysts can differentiate</li>
-    
 <li>ensure profiles consider independently as they have different seasonality or some other fundamental characteristic</li>
-  </ul></li>
+</ul>
+</li>
 </ul>
 <p>As such, there are two types of metadata that we seek to support in Metron:</p>
-
 <ul>
-  
+
 <li>Environmental metadata : Metadata about the system at large
-  
 <ul>
-    
+
 <li>Consider the possibility that you have multiple kafka topics being processed by one parser and you want to tag the messages with the kafka topic</li>
-    
 <li>At the moment, only the kafka topic is kept as the field name.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Custom metadata: Custom metadata from an individual telemetry source that one might want to use within Metron.</li>
 </ul>
 <p>Metadata is controlled by two fields in the parser:</p>
-
 <ul>
-  
-<li><tt>readMetadata</tt> : This is a boolean indicating whether metadata will be read and made available to Field transformations (i.e. Stellar field transformations). The default is <tt>false</tt>.</li>
-  
-<li>
-<p><tt>mergeMetadata</tt> : This is a boolean indicating whether metadata fields will be merged with the message automatically.<br />That is to say, if this property is set to <tt>true</tt> then every metadata field will become part of the messages and, consequently, also available for use in field transformations.</p>
-<div class="section">
-<h4><a name="Field_Naming"></a>Field Naming</h4></li>
+
+<li><tt>readMetadata</tt> : This is a boolean indicating whether metadata will be read and made available to Field transformations (i.e. Stellar field transformations).  The default is <tt>false</tt>.</li>
+<li><tt>mergeMetadata</tt> : This is a boolean indicating whether metadata fields will be merged with the message automatically.<br />
+That is to say, if this property is set to <tt>true</tt> then every metadata field will become part of the messages and, consequently, also available for use in field transformations.</li>
 </ul>
-<p>In order to avoid collisions from metadata fields, metadata fields will be prefixed with <tt>metron.metadata.</tt>.<br />So, for instance the kafka topic would be in the field <tt>metron.metadata.topic</tt>.</p></div>
+<div class="section">
+<h4><a name="Field_Naming"></a>Field Naming</h4>
+<p>In order to avoid collisions from metadata fields, metadata fields will be prefixed with <tt>metron.metadata.</tt>.<br />
+So, for instance the kafka topic would be in the field <tt>metron.metadata.topic</tt>.</p></div>
 <div class="section">
 <h4><a name="Specifying_Custom_Metadata"></a>Specifying Custom Metadata</h4>
-<p>Custom metadata is specified by sending a JSON Map in the key. If no key is sent, then, obviously, no metadata will be parsed. For instance, sending a metadata field called <tt>customer_id</tt> could be done by sending</p>
+<p>Custom metadata is specified by sending a JSON Map in the key.  If no key is sent, then, obviously, no metadata will be parsed. For instance, sending a metadata field called <tt>customer_id</tt> could be done by sending</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
 &quot;customer_id&quot; : &quot;my_customer_id&quot;
 }
 </pre></div></div>
-<p>in the kafka key. This would be exposed as the field <tt>metron.metadata.customer_id</tt> to stellar field transformations as well, if <tt>mergeMetadata</tt> is <tt>true</tt>, available as a field in its own right.</p></div></div>
+
+<p>in the kafka key.  This would be exposed as the field <tt>metron.metadata.customer_id</tt> to stellar field transformations as well, if <tt>mergeMetadata</tt> is <tt>true</tt>, available as a field in its own right.</p></div></div>
 <div class="section">
 <h3><a name="fieldTransformation_configuration"></a><tt>fieldTransformation</tt> configuration</h3>
 <p>The format of a <tt>fieldTransformation</tt> is as follows:</p>
-
 <ul>
-  
-<li><tt>input</tt> : An array of fields or a single field representing the input. This is optional; if unspecified, then the whole message is passed as input.</li>
-  
-<li><tt>output</tt> : The outputs to produce from the transformation. If unspecified, it is assumed to be the same as inputs.</li>
-  
-<li><tt>transformation</tt> : The fully qualified classname of the transformation to be used. This is either a class which implements <tt>FieldTransformation</tt> or a member of the <tt>FieldTransformations</tt> enum.</li>
-  
+
+<li><tt>input</tt> : An array of fields or a single field representing the input.  This is optional; if unspecified, then the whole message is passed as input.</li>
+<li><tt>output</tt> : The outputs to produce from the transformation.  If unspecified, it is assumed to be the same as inputs.</li>
+<li><tt>transformation</tt> : The fully qualified classname of the transformation to be used.  This is either a class which implements <tt>FieldTransformation</tt> or a member of the <tt>FieldTransformations</tt> enum.</li>
 <li><tt>config</tt> : A String to Object map of transformation specific configuration.</li>
 </ul>
 <p>The currently implemented fieldTransformations are:</p>
-
 <ul>
-  
-<li><tt>REMOVE</tt> : This transformation removes the specified input fields. If you want a conditional removal, you can pass a Metron Query Language statement to define the conditions under which you want to remove the fields.</li>
+
+<li><tt>REMOVE</tt> : This transformation removes the specified input fields.  If you want a conditional removal, you can pass a Metron Query Language statement to define the conditions under which you want to remove the fields.</li>
 </ul>
 <p>Consider the following simple configuration which will remove <tt>field1</tt> unconditionally:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
 ...
     &quot;fieldTransformations&quot; : [
           {
@@ -568,11 +383,12 @@
                       ]
 }
 </pre></div></div>
+
 <p>Consider the following simple sensor parser configuration which will remove <tt>field1</tt> whenever <tt>field2</tt> exists and whose corresponding equal to &#x2018;foo&#x2019;:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
 ...
   &quot;fieldTransformations&quot; : [
           {
@@ -587,14 +403,34 @@
 </pre></div></div>
 
 <ul>
-  
+
+<li><tt>SELECT</tt>: This transformation filters the fields in the message to include only the configured output fields, and drops any not explicitly included.</li>
+</ul>
+<p>For example:</p>
+
+<div>
+<div>
+<pre class="source">{
+...
+    &quot;fieldTransformations&quot; : [
+          {
+            &quot;output&quot; : [&quot;field1&quot;, &quot;field2&quot; ] 
+          , &quot;transformation&quot; : &quot;SELECT&quot;
+          }
+                      ]
+}
+</pre></div></div>
+
+<p>when applied to a message containing keys field1, field2 and field3, will only output the first two. It is also worth noting that two standard fields - timestamp and original_source - will always be passed along whether they are listed in output or not, since they are considered core required fields.</p>
+<ul>
+
 <li><tt>IP_PROTOCOL</tt> : This transformation maps IANA protocol numbers to consistent string representations.</li>
 </ul>
 <p>Consider the following sensor parser config to map the <tt>protocol</tt> field to a textual representation of the protocol:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
 ...
     &quot;fieldTransformations&quot; : [
           {
@@ -604,20 +440,45 @@
                       ]
 }
 </pre></div></div>
-<p>This transformation would transform <tt>{ &quot;protocol&quot; : 6, &quot;source.type&quot; : &quot;bro&quot;, ... }</tt> into <tt>{ &quot;protocol&quot; : &quot;TCP&quot;, &quot;source.type&quot; : &quot;bro&quot;, ...}</tt></p>
 
+<p>This transformation would transform <tt>{ &quot;protocol&quot; : 6, &quot;source.type&quot; : &quot;bro&quot;, ... }</tt> into <tt>{ &quot;protocol&quot; : &quot;TCP&quot;, &quot;source.type&quot; : &quot;bro&quot;, ...}</tt></p>
 <ul>
-  
-<li><tt>STELLAR</tt> : This transformation executes a set of transformations  expressed as <a href="../metron-common/index.html">Stellar Language</a> statements.</li>
-</ul></div>
+
+<li>
+
+<p><tt>STELLAR</tt> : This transformation executes a set of transformations expressed as <a href="../metron-common/index.html">Stellar Language</a> statements.</p>
+</li>
+<li>
+
+<p><tt>RENAME</tt> : This transformation allows users to rename a set of fields.  Specifically, the config is presumed to be the mapping.  The keys to the config are the existing field names and the values for the config map are the associated new field name.</p>
+</li>
+</ul>
+<p>The following config will rename the fields <tt>old_field</tt> and <tt>different_old_field</tt> to <tt>new_field</tt> and <tt>different_new_field</tt> respectively:</p>
+
+<div>
+<div>
+<pre class="source">{
+...
+    &quot;fieldTransformations&quot; : [
+          {
+            &quot;transformation&quot; : &quot;RENAME&quot;,
+          , &quot;config&quot; : {
+            &quot;old_field&quot; : &quot;new_field&quot;,
+            &quot;different_old_field&quot; : &quot;different_new_field&quot;
+                       }
+          }
+                      ]
+}
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Assignment_to_null"></a>Assignment to <tt>null</tt></h3>
-<p>If, in your field transformation, you assign a field to <tt>null</tt>, the field will be removed. You can use this capability to rename variables.</p>
+<p>If, in your field transformation, you assign a field to <tt>null</tt>, the field will be removed. You can use this capability to rename variables.  It is preferred, however, that the <tt>RENAME</tt> field transformation is used in this situation as it is less awkward.</p>
 <p>Consider this example:</p>
 
-<div class="source">
-<div class="source">
-<pre> &quot;fieldTransformations&quot; : [
+<div>
+<div>
+<pre class="source"> &quot;fieldTransformations&quot; : [
          { &quot;transformation&quot; : &quot;STELLAR&quot;
          ,&quot;output&quot; : [ &quot;new_field&quot;, &quot;old_field&quot;]
          ,&quot;config&quot; : {
@@ -627,15 +488,16 @@
          }
  ]
 </pre></div></div>
+
 <p>This would set <tt>new_field</tt> to the value of <tt>old_field</tt> and remove <tt>old_field</tt>.</p></div>
 <div class="section">
 <h3><a name="Warning:_Transforming_the_same_field_twice"></a>Warning: Transforming the same field twice</h3>
-<p>Currently, the stellar expressions are expressed in the form of a map where the keys define the fields and the values define the Stellar expressions. You order the expression evaluation in the <tt>output</tt> field. A consequence of this choice to store the assignments as a map is that the same field cannot appear in the map as a key twice.</p>
+<p>Currently, the stellar expressions are expressed in the form of a map where the keys define the fields and the values define the Stellar expressions.  You order the expression evaluation in the <tt>output</tt> field.  A consequence of this choice to store the assignments as a map is that the same field cannot appear in the map as a key twice.</p>
 <p>For instance, the following will not function as expected:</p>
 
-<div class="source">
-<div class="source">
-<pre> &quot;fieldTransformations&quot; : [
+<div>
+<div>
+<pre class="source"> &quot;fieldTransformations&quot; : [
          { &quot;transformation&quot; : &quot;STELLAR&quot;
          ,&quot;output&quot; : [ &quot;new_field&quot;]
          ,&quot;config&quot; : {
@@ -645,23 +507,21 @@
          }
  ]
 </pre></div></div>
+
 <p>In the above example, the last instance of <tt>new_field</tt> will win and <tt>TO_LOWER(new_field)</tt> will be evaluated while <tt>TO_UPPER(field1)</tt> will be skipped.</p></div>
 <div class="section">
 <h3><a name="Example"></a>Example</h3>
 <p>Consider the following sensor parser config to add three new fields to a message:</p>
-
 <ul>
-  
+
 <li><tt>utc_timestamp</tt> : The unix epoch timestamp based on the <tt>timestamp</tt> field, a <tt>dc</tt> field which is the data center the message comes from and a <tt>dc2tz</tt> map mapping data centers to timezones</li>
-  
 <li><tt>url_host</tt> : The host associated with the url in the <tt>url</tt> field</li>
-  
 <li><tt>url_protocol</tt> : The protocol associated with the url in the <tt>url</tt> field</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
 ...
     &quot;fieldTransformations&quot; : [
           {
@@ -684,14 +544,15 @@ HH:mm:ss', MAP_GET(dc, dc2tz, 'UTC') )&quot;
     }
 }
 </pre></div></div>
+
 <p>Note that the <tt>dc2tz</tt> map is in the parser config, so it is accessible in the functions.</p></div>
 <div class="section">
 <h3><a name="An_Example_Configuration_for_a_Sensor"></a>An Example Configuration for a Sensor</h3>
 <p>Consider the following example configuration for the <tt>yaf</tt> sensor:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;parserClassName&quot;:&quot;org.apache.metron.parsers.GrokParser&quot;,
   &quot;sensorTopic&quot;:&quot;yaf&quot;,
   &quot;fieldTransformations&quot; : [
@@ -709,42 +570,38 @@ HH:mm:ss', MAP_GET(dc, dc2tz, 'UTC') )&quot;
     &quot;dateFormat&quot;:&quot;yyyy-MM-dd HH:mm:ss.S&quot;
   }
 }
-</pre></div></div></div></div>
+</pre></div></div>
+</div></div>
 <div class="section">
 <h2><a name="Parser_Adapters"></a>Parser Adapters</h2>
-<p>Parser adapters are loaded dynamically in each Metron topology. They are defined in the Parser Config (defined above) JSON file in Zookeeper.</p>
+<p>Parser adapters are loaded dynamically in each Metron topology.  They are defined in the Parser Config (defined above) JSON file in Zookeeper.</p>
 <div class="section">
 <h3><a name="Java_Parser_Adapters"></a>Java Parser Adapters</h3>
-<p>Java parser adapters are indended for higher-velocity topologies and are not easily changed or extended. As the adoption of Metron continues we plan on extending our library of Java adapters to process more log formats. As of this moment the Java adapters included with Metron are:</p>
-
+<p>Java parser adapters are indended for higher-velocity topologies and are not easily changed or extended.  As the adoption of Metron continues we plan on extending our library of Java adapters to process more log formats.  As of this moment the Java adapters included with Metron are:</p>
 <ul>
-  
+
 <li>org.apache.metron.parsers.ise.BasicIseParser : Parse ISE messages</li>
-  
 <li>org.apache.metron.parsers.bro.BasicBroParser : Parse Bro messages</li>
-  
 <li>org.apache.metron.parsers.sourcefire.BasicSourcefireParser : Parse Sourcefire messages</li>
-  
 <li>org.apache.metron.parsers.lancope.BasicLancopeParser : Parse Lancope messages</li>
 </ul></div>
 <div class="section">
 <h3><a name="Grok_Parser_Adapters"></a>Grok Parser Adapters</h3>
-<p>Grok parser adapters are designed primarly for someone who is not a Java coder for quickly standing up a parser adapter for lower velocity topologies. Grok relies on Regex for message parsing, which is much slower than purpose-built Java parsers, but is more extensible. Grok parsers are defined via a config file and the topplogy does not need to be recombiled in order to make changes to them. An example of a Grok perser is:</p>
-
+<p>Grok parser adapters are designed primarly for someone who is not a Java coder for quickly standing up a parser adapter for lower velocity topologies.  Grok relies on Regex for message parsing, which is much slower than purpose-built Java parsers, but is more extensible.  Grok parsers are defined via a config file and the topplogy does not need to be recombiled in order to make changes to them.  An example of a Grok perser is:</p>
 <ul>
-  
+
 <li>org.apache.metron.parsers.GrokParser</li>
 </ul>
 <p>For more information on the Grok project please refer to the following link:</p>
 <p><a class="externalLink" href="https://github.com/thekrakken/java-grok">https://github.com/thekrakken/java-grok</a></p>
 <p><a name="Starting_the_Parser_Topology"></a></p>
 <h1>Starting the Parser Topology</h1>
-<p>Starting a particular parser topology on a running Metron deployment is as easy as running the <tt>start_parser_topology.sh</tt> script located in <tt>$METRON_HOME/bin</tt>. This utility will allow you to configure and start the running topology assuming that the sensor specific parser configuration exists within zookeeper.</p>
+<p>Starting a particular parser topology on a running Metron deployment is as easy as running the <tt>start_parser_topology.sh</tt> script located in <tt>$METRON_HOME/bin</tt>.  This utility will allow you to configure and start the running topology assuming that the sensor specific parser configuration exists within zookeeper.</p>
 <p>The usage for <tt>start_parser_topology.sh</tt> is as follows:</p>
 
-<div class="source">
-<div class="source">
-<pre>usage: start_parser_topology.sh
+<div>
+<div>
+<pre class="source">usage: start_parser_topology.sh
  -e,--extra_topology_options &lt;JSON_FILE&gt;               Extra options in the form
                                                        of a JSON file with a map
                                                        for content.
@@ -780,136 +637,129 @@ HH:mm:ss', MAP_GET(dc, dc2tz, 'UTC') )&quot;
  -t,--test &lt;TEST&gt;                                      Run in Test Mode
  -z,--zk &lt;ZK_QUORUM&gt;                                   Zookeeper Quroum URL
                                                        (zk1:2181,zk2:2181,...
-</pre></div></div></div></div>
+</pre></div></div>
+</div></div>
 <div class="section">
 <h2><a name="The_--extra_kafka_spout_config_Option"></a>The <tt>--extra_kafka_spout_config</tt> Option</h2>
-<p>These options are intended to configure the Storm Kafka Spout more completely. These options can be specified in a JSON file containing a map associating the kafka spout configuration parameter to a value. The range of values possible to configure are:</p>
-
+<p>These options are intended to configure the Storm Kafka Spout more completely.  These options can be specified in a JSON file containing a map associating the kafka spout configuration parameter to a value. The range of values possible to configure are:</p>
 <ul>
-  
-<li><tt>spout.pollTimeoutMs</tt> - Specifies the time, in milliseconds, spent waiting in poll if data is not available. Default is 2s</li>
-  
-<li><tt>spout.firstPollOffsetStrategy</tt> - Sets the offset used by the Kafka spout in the first poll to Kafka broker upon process start. One of
-  
+
+<li><tt>spout.pollTimeoutMs</tt> -  Specifies the time, in milliseconds, spent waiting in poll if data is not available. Default is 2s</li>
+<li><tt>spout.firstPollOffsetStrategy</tt> - Sets the offset used by the Kafka spout in the first poll to Kafka broker upon process start.  One of
 <ul>
-    
+
 <li><tt>EARLIEST</tt></li>
-    
 <li><tt>LATEST</tt></li>
-    
 <li><tt>UNCOMMITTED_EARLIEST</tt> - Last uncommitted and if offsets aren&#x2019;t found, defaults to earliest. NOTE: This is the default.</li>
-    
 <li><tt>UNCOMMITTED_LATEST</tt> - Last uncommitted and if offsets aren&#x2019;t found, defaults to latest.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>spout.offsetCommitPeriodMs</tt> - Specifies the period, in milliseconds, the offset commit task is periodically called. Default is 15s.</li>
-  
 <li><tt>spout.maxUncommittedOffsets</tt> - Defines the max number of polled offsets (records) that can be pending commit, before another poll can take place. Once this limit is reached, no more offsets (records) can be polled until the next successful commit(s) sets the number of pending offsets bellow the threshold. The default is 10,000,000.</li>
-  
-<li><tt>spout.maxRetries</tt> - Defines the max number of retrials in case of tuple failure. The default is to retry forever, which means that no new records are committed until the previous polled records have been acked. This guarantees at once delivery of all the previously polled records. By specifying a finite value for maxRetries, the user decides to sacrifice guarantee of delivery for the previous polled records in favor of processing more records.</li>
-  
+<li><tt>spout.maxRetries</tt> -  Defines the max number of retrials in case of tuple failure. The default is to retry forever, which means that no new records are committed until the previous polled records have been acked. This guarantees at once delivery of all the previously polled records.  By specifying a finite value for maxRetries, the user decides to sacrifice guarantee of delivery for the previous polled records in favor of processing more records.</li>
 <li>Any of the configs in the Consumer API for <a class="externalLink" href="http://kafka.apache.org/0100/documentation.html#newconsumerconfigs">Kafka 0.10.x</a></li>
 </ul>
 <p>For instance, creating a JSON file which will set the offsets to <tt>UNCOMMITTED_EARLIEST</tt></p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;spout.firstPollOffsetStrategy&quot; : &quot;UNCOMMITTED_EARLIEST&quot;
 }
 </pre></div></div>
+
 <p>This would be loaded by passing the file as argument to <tt>--extra_kafka_spout_config</tt></p></div>
 <div class="section">
 <h2><a name="The_--extra_topology_options_Option"></a>The <tt>--extra_topology_options</tt> Option</h2>
-<p>These options are intended to be Storm configuration options and will live in a JSON file which will be loaded into the Storm config. For instance, if you wanted to set a storm property on the config called <tt>topology.ticks.tuple.freq.secs</tt> to 1000 and <tt>storm.local.dir</tt> to <tt>/opt/my/path</tt> you could create a file called <tt>custom_config.json</tt> containing </p>
+<p>These options are intended to be Storm configuration options and will live in a JSON file which will be loaded into the Storm config.  For instance, if you wanted to set a storm property on the config called <tt>topology.ticks.tuple.freq.secs</tt> to 1000 and <tt>storm.local.dir</tt> to <tt>/opt/my/path</tt> you could create a file called <tt>custom_config.json</tt> containing</p>
 
-<div class="source">
-<div class="source">
-<pre>{ 
+<div>
+<div>
+<pre class="source">{ 
   &quot;topology.ticks.tuple.freq.secs&quot; : 1000,
   &quot;storm.local.dir&quot; : &quot;/opt/my/path&quot;
 }
 </pre></div></div>
+
 <p>and pass <tt>--extra_topology_options custom_config.json</tt> to <tt>start_parser_topology.sh</tt>.</p>
 <p><a name="Notes_on_Performance_Tuning"></a></p>
 <h1>Notes on Performance Tuning</h1>
-<p>Default installed Metron is untuned for production deployment. There are a few knobs to tune to get the most out of your system.</p>
+<p>Default installed Metron is untuned for production deployment.  There are a few knobs to tune to get the most out of your system.</p>
 <p><a name="Notes_on_Adding_a_New_Sensor"></a></p>
 <h1>Notes on Adding a New Sensor</h1>
 <p>In order to allow for meta alerts to be queries alongside regular alerts in Elasticsearch 2.x, it is necessary to add an additional field to the templates and mapping for existing sensors.</p>
 <p>Please see a description of the steps necessary to make this change in the metron-elasticsearch <a href="../../metron-platform/metron-elasticsearch/index.html#Using_Metron_with_Elasticsearch_2.x">Using Metron with Elasticsearch 2.x</a></p></div>
 <div class="section">
 <h2><a name="Kafka_Queue"></a>Kafka Queue</h2>
-<p>The kafka queue associated with your parser is a collection point for all of the data sent to your parser. As such, make sure that the number of partitions in the kafka topic is sufficient to handle the throughput that you expect from your parser topology.</p></div>
+<p>The kafka queue associated with your parser is a collection point for all of the data sent to your parser.  As such, make sure that the number of partitions in the kafka topic is sufficient to handle the throughput that you expect from your parser topology.</p></div>
 <div class="section">
 <h2><a name="Parser_Topology"></a>Parser Topology</h2>
-<p>The enrichment topology as started by the <tt>$METRON_HOME/bin/start_parser_topology.sh</tt> script uses a default of one executor per bolt. In a real production system, this should be customized by modifying the arguments sent to this utility.</p>
-
+<p>The enrichment topology as started by the <tt>$METRON_HOME/bin/start_parser_topology.sh</tt> script uses a default of one executor per bolt.  In a real production system, this should be customized by modifying the arguments sent to this utility.</p>
 <ul>
-  
+
 <li>Topology Wide
-  
 <ul>
-    
+
 <li><tt>--num_workers</tt> : The number of workers for the topology</li>
-    
 <li><tt>--num_ackers</tt> : The number of ackers for the topology</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>The Kafka Spout
-  
 <ul>
-    
+
 <li><tt>--spout_num_tasks</tt> : The number of tasks for the spout</li>
-    
 <li><tt>--spout_p</tt> : The parallelism hint for the spout</li>
-    
 <li>Ensure that the spout has enough parallelism so that it can dedicate a worker per partition in your kafka topic.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>The Parser Bolt
-  
 <ul>
-    
+
 <li><tt>--parser_num_tasks</tt> : The number of tasks for the parser bolt</li>
-    
 <li><tt>--parser_p</tt> : The parallelism hint for the spout</li>
-    
 <li>This is bolt that gets the most processing, so ensure that it is configured with sufficient parallelism to match your throughput expectations.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>The Error Message Writer Bolt
-  
 <ul>
-    
+
 <li><tt>--error_writer_num_tasks</tt> : The number of tasks for the error writer bolt</li>
-    
 <li><tt>--error_writer_p</tt> : The parallelism hint for the error writer bolt</li>
-  </ul></li>
+</ul>
+</li>
 </ul>
 <p>Finally, if workers and executors are new to you, the following might be of use to you:</p>
-
 <ul>
-  
+
 <li><a class="externalLink" href="http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/">Understanding the Parallelism of a Storm Topology</a></li>
 </ul></div>
-                  </div>
-            </div>
-          </div>
+<div class="section">
+<h2><a name="JSON_Path"></a>JSON Path</h2>
+<blockquote>
 
-    <hr/>
+<dl>
 
+<dt>&#x201c;JSONPath expressions always refer to a JSON structure in the same way as XPath expression are used in combination with an XML document.&#x201d;</dt>
+<dd>Stefan Goessner</dd>
+</dl>
+</blockquote>
+<ul>
+
+<li><a class="externalLink" href="http://goessner.net/articles/JsonPath/">JSON Path concept</a></li>
+<li><a class="externalLink" href="https://github.com/json-path/JsonPath">Read about JSON Path library Apache Metron uses</a></li>
+<li><a class="externalLink" href="http://jsonpath.herokuapp.com">Try JSON Path expressions online</a></li>
+</ul></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[44/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/css/apache-maven-fluido-1.3.0.min.css
----------------------------------------------------------------------
diff --git a/site/current-book/css/apache-maven-fluido-1.3.0.min.css b/site/current-book/css/apache-maven-fluido-1.3.0.min.css
deleted file mode 100644
index 7db4133..0000000
--- a/site/current-book/css/apache-maven-fluido-1.3.0.min.css
+++ /dev/null
@@ -1,9 +0,0 @@
-/*!
- * Bootstrap v2.1.0
- *
- * Copyright 2012 Twitter, Inc
- * Licensed under the Apache License v2.0
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Designed and built with all the love in the world @twitter by @mdo and @fat.
- */article,aside,details,figcaption,figure,footer,header,hgroup,nav,section{display:block}audio,canvas,video{display:inline-block;*display:inline;*zoom:1}audio:not([controls]){display:none}html{font-size:100%;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}a:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}a:hover,a:active{outline:0}sub,sup{position:relative;font-size:75%;line-height:0;vertical-align:baseline}sup{top:-0.5em}sub{bottom:-0.25em}img{height:auto;max-width:100%;vertical-align:middle;border:0;-ms-interpolation-mode:bicubic}#map_canvas img{max-width:none}button,input,select,textarea{margin:0;font-size:100%;vertical-align:middle}button,input{*overflow:visible;line-height:normal}button::-moz-focus-inner,input::-moz-focus-inner{padding:0;border:0}button,input[type="button"],input[type="reset"],input[type="submit"]{cursor:pointer;-webkit-appearance:button}input[type="search"]{-webkit-box-sizing:content-box;-moz-box-sizing:con
 tent-box;box-sizing:content-box;-webkit-appearance:textfield}input[type="search"]::-webkit-search-decoration,input[type="search"]::-webkit-search-cancel-button{-webkit-appearance:none}textarea{overflow:auto;vertical-align:top}.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;line-height:0;content:""}.clearfix:after{clear:both}.hide-text{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.input-block-level{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}body{margin:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:14px;line-height:20px;color:#333;background-color:#fff}a{color:#08c;text-decoration:none}a:hover{color:#005580;text-decoration:underline}.img-rounded{-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.img-polaroid{padding:4px;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);-webkit-box-shadow:0
  1px 3px rgba(0,0,0,0.1);-moz-box-shadow:0 1px 3px rgba(0,0,0,0.1);box-shadow:0 1px 3px rgba(0,0,0,0.1)}.img-circle{-webkit-border-radius:500px;-moz-border-radius:500px;border-radius:500px}.row{margin-left:-20px;*zoom:1}.row:before,.row:after{display:table;line-height:0;content:""}.row:after{clear:both}[class*="span"]{float:left;margin-left:20px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:940px}.span12{width:940px}.span11{width:860px}.span10{width:780px}.span9{width:700px}.span8{width:620px}.span7{width:540px}.span6{width:460px}.span5{width:380px}.span4{width:300px}.span3{width:220px}.span2{width:140px}.span1{width:60px}.offset12{margin-left:980px}.offset11{margin-left:900px}.offset10{margin-left:820px}.offset9{margin-left:740px}.offset8{margin-left:660px}.offset7{margin-left:580px}.offset6{margin-left:500px}.offset5{margin-left:420px}.offset4{margin-left:340px}.offset3{margin-left:260px}.offset2{margin-left:180px}.offs
 et1{margin-left:100px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid [class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.127659574468085%;*margin-left:2.074468085106383%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.48936170212765%;*width:91.43617021276594%}.row-fluid .span10{width:82.97872340425532%;*width:82.92553191489361%}.row-fluid .span9{width:74.46808510638297%;*width:74.41489361702126%}.row-fluid .span8{width:65.95744680851064%;*width:65.90425531914893%}.row-fluid .span7{width:57.44680851063829%;*width:57.39361702127659%}.row-fluid .span6{width:48.93617021276595%;*width:48.88297872340425%}.row-fluid .span5{width:40.42553191489362%;*width:40.37234042553192%}.row-fluid .span4{width:31.9
 14893617021278%;*width:31.861702127659576%}.row-fluid .span3{width:23.404255319148934%;*width:23.351063829787233%}.row-fluid .span2{width:14.893617021276595%;*width:14.840425531914894%}.row-fluid .span1{width:6.382978723404255%;*width:6.329787234042553%}.row-fluid .offset12{margin-left:104.25531914893617%;*margin-left:104.14893617021275%}.row-fluid .offset12:first-child{margin-left:102.12765957446808%;*margin-left:102.02127659574467%}.row-fluid .offset11{margin-left:95.74468085106382%;*margin-left:95.6382978723404%}.row-fluid .offset11:first-child{margin-left:93.61702127659574%;*margin-left:93.51063829787232%}.row-fluid .offset10{margin-left:87.23404255319149%;*margin-left:87.12765957446807%}.row-fluid .offset10:first-child{margin-left:85.1063829787234%;*margin-left:84.99999999999999%}.row-fluid .offset9{margin-left:78.72340425531914%;*margin-left:78.61702127659572%}.row-fluid .offset9:first-child{margin-left:76.59574468085106%;*margin-left:76.48936170212764%}.row-fluid .offset8{mar
 gin-left:70.2127659574468%;*margin-left:70.10638297872339%}.row-fluid .offset8:first-child{margin-left:68.08510638297872%;*margin-left:67.9787234042553%}.row-fluid .offset7{margin-left:61.70212765957446%;*margin-left:61.59574468085106%}.row-fluid .offset7:first-child{margin-left:59.574468085106375%;*margin-left:59.46808510638297%}.row-fluid .offset6{margin-left:53.191489361702125%;*margin-left:53.085106382978715%}.row-fluid .offset6:first-child{margin-left:51.063829787234035%;*margin-left:50.95744680851063%}.row-fluid .offset5{margin-left:44.68085106382979%;*margin-left:44.57446808510638%}.row-fluid .offset5:first-child{margin-left:42.5531914893617%;*margin-left:42.4468085106383%}.row-fluid .offset4{margin-left:36.170212765957444%;*margin-left:36.06382978723405%}.row-fluid .offset4:first-child{margin-left:34.04255319148936%;*margin-left:33.93617021276596%}.row-fluid .offset3{margin-left:27.659574468085104%;*margin-left:27.5531914893617%}.row-fluid .offset3:first-child{margin-left:25
 .53191489361702%;*margin-left:25.425531914893618%}.row-fluid .offset2{margin-left:19.148936170212764%;*margin-left:19.04255319148936%}.row-fluid .offset2:first-child{margin-left:17.02127659574468%;*margin-left:16.914893617021278%}.row-fluid .offset1{margin-left:10.638297872340425%;*margin-left:10.53191489361702%}.row-fluid .offset1:first-child{margin-left:8.51063829787234%;*margin-left:8.404255319148938%}[class*="span"].hide,.row-fluid [class*="span"].hide{display:none}[class*="span"].pull-right,.row-fluid [class*="span"].pull-right{float:right}.container{margin-right:auto;margin-left:auto;*zoom:1}.container:before,.container:after{display:table;line-height:0;content:""}.container:after{clear:both}.container-fluid{padding-right:20px;padding-left:20px;*zoom:1}.container-fluid:before,.container-fluid:after{display:table;line-height:0;content:""}.container-fluid:after{clear:both}p{margin:0 0 10px}.lead{margin-bottom:20px;font-size:20px;font-weight:200;line-height:30px}small{font-size:8
 5%}strong{font-weight:bold}em{font-style:italic}cite{font-style:normal}.muted{color:#999}h1,h2,h3,h4,h5,h6{margin:10px 0;font-family:inherit;font-weight:bold;line-height:1;color:inherit;text-rendering:optimizelegibility}h1 small,h2 small,h3 small,h4 small,h5 small,h6 small{font-weight:normal;line-height:1;color:#999}h1{font-size:36px;line-height:40px}h2{font-size:30px;line-height:40px}h3{font-size:24px;line-height:40px}h4{font-size:18px;line-height:20px}h5{font-size:14px;line-height:20px}h6{font-size:12px;line-height:20px}h1 small{font-size:24px}h2 small{font-size:18px}h3 small{font-size:14px}h4 small{font-size:14px}.page-header{padding-bottom:9px;margin:20px 0 30px;border-bottom:1px solid #eee}ul,ol{padding:0;margin:0 0 10px 25px}ul ul,ul ol,ol ol,ol ul{margin-bottom:0}li{line-height:20px}ul.unstyled,ol.unstyled{margin-left:0;list-style:none}dl{margin-bottom:20px}dt,dd{line-height:20px}dt{font-weight:bold}dd{margin-left:10px}.dl-horizontal dt{float:left;width:120px;overflow:hidden;
 clear:left;text-align:right;text-overflow:ellipsis;white-space:nowrap}.dl-horizontal dd{margin-left:130px}hr{margin:20px 0;border:0;border-top:1px solid #eee;border-bottom:1px solid #fff}abbr[title]{cursor:help;border-bottom:1px dotted #999}abbr.initialism{font-size:90%;text-transform:uppercase}blockquote{padding:0 0 0 15px;margin:0 0 20px;border-left:5px solid #eee}blockquote p{margin-bottom:0;font-size:16px;font-weight:300;line-height:25px}blockquote small{display:block;line-height:20px;color:#999}blockquote small:before{content:'\2014 \00A0'}blockquote.pull-right{float:right;padding-right:15px;padding-left:0;border-right:5px solid #eee;border-left:0}blockquote.pull-right p,blockquote.pull-right small{text-align:right}blockquote.pull-right small:before{content:''}blockquote.pull-right small:after{content:'\00A0 \2014'}q:before,q:after,blockquote:before,blockquote:after{content:""}address{display:block;margin-bottom:20px;font-style:normal;line-height:20px}code,pre{padding:0 3px 2px
 ;font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:12px;color:#333;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}code{padding:2px 4px;color:#d14;background-color:#f7f7f9;border:1px solid #e1e1e8}pre{display:block;padding:9.5px;margin:0 0 10px;font-size:13px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre;white-space:pre-wrap;background-color:#f5f5f5;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.15);-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}pre.prettyprint{margin-bottom:20px}pre code{padding:0;color:inherit;background-color:transparent;border:0}.pre-scrollable{max-height:340px;overflow-y:scroll}form{margin:0 0 20px}fieldset{padding:0;margin:0;border:0}legend{display:block;width:100%;padding:0;margin-bottom:20px;font-size:21px;line-height:40px;color:#333;border:0;border-bottom:1px solid #e5e5e5}legend small{font-size:15px;color:#999}label,input,button,select,textarea{font-size:14px;font-we
 ight:normal;line-height:20px}input,button,select,textarea{font-family:"Helvetica Neue",Helvetica,Arial,sans-serif}label{display:block;margin-bottom:5px}select,textarea,input[type="text"],input[type="password"],input[type="datetime"],input[type="datetime-local"],input[type="date"],input[type="month"],input[type="time"],input[type="week"],input[type="number"],input[type="email"],input[type="url"],input[type="search"],input[type="tel"],input[type="color"],.uneditable-input{display:inline-block;height:20px;padding:4px 6px;margin-bottom:9px;font-size:14px;line-height:20px;color:#555;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}input,textarea{width:210px}textarea{height:auto}textarea,input[type="text"],input[type="password"],input[type="datetime"],input[type="datetime-local"],input[type="date"],input[type="month"],input[type="time"],input[type="week"],input[type="number"],input[type="email"],input[type="url"],input[type="search"],input[type="tel"],input[type="color"]
 ,.uneditable-input{background-color:#fff;border:1px solid #ccc;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-webkit-transition:border linear .2s,box-shadow linear .2s;-moz-transition:border linear .2s,box-shadow linear .2s;-o-transition:border linear .2s,box-shadow linear .2s;transition:border linear .2s,box-shadow linear .2s}textarea:focus,input[type="text"]:focus,input[type="password"]:focus,input[type="datetime"]:focus,input[type="datetime-local"]:focus,input[type="date"]:focus,input[type="month"]:focus,input[type="time"]:focus,input[type="week"]:focus,input[type="number"]:focus,input[type="email"]:focus,input[type="url"]:focus,input[type="search"]:focus,input[type="tel"]:focus,input[type="color"]:focus,.uneditable-input:focus{border-color:rgba(82,168,236,0.8);outline:0;outline:thin dotted \9;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6);-moz-
 box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6)}input[type="radio"],input[type="checkbox"]{margin:4px 0 0;margin-top:1px \9;*margin-top:0;line-height:normal;cursor:pointer}input[type="file"],input[type="image"],input[type="submit"],input[type="reset"],input[type="button"],input[type="radio"],input[type="checkbox"]{width:auto}select,input[type="file"]{height:30px;*margin-top:4px;line-height:30px}select{width:220px;background-color:#fff;border:1px solid #bbb}select[multiple],select[size]{height:auto}select:focus,input[type="file"]:focus,input[type="radio"]:focus,input[type="checkbox"]:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}.uneditable-input,.uneditable-textarea{color:#999;cursor:not-allowed;background-color:#fcfcfc;border-color:#ccc;-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.025);-moz-box-shadow:inset 0 1px 2px rgba(0,0,0,0.025)
 ;box-shadow:inset 0 1px 2px rgba(0,0,0,0.025)}.uneditable-input{overflow:hidden;white-space:nowrap}.uneditable-textarea{width:auto;height:auto}input:-moz-placeholder,textarea:-moz-placeholder{color:#999}input:-ms-input-placeholder,textarea:-ms-input-placeholder{color:#999}input::-webkit-input-placeholder,textarea::-webkit-input-placeholder{color:#999}.radio,.checkbox{min-height:18px;padding-left:18px}.radio input[type="radio"],.checkbox input[type="checkbox"]{float:left;margin-left:-18px}.controls>.radio:first-child,.controls>.checkbox:first-child{padding-top:5px}.radio.inline,.checkbox.inline{display:inline-block;padding-top:5px;margin-bottom:0;vertical-align:middle}.radio.inline+.radio.inline,.checkbox.inline+.checkbox.inline{margin-left:10px}.input-mini{width:60px}.input-small{width:90px}.input-medium{width:150px}.input-large{width:210px}.input-xlarge{width:270px}.input-xxlarge{width:530px}input[class*="span"],select[class*="span"],textarea[class*="span"],.uneditable-input[class*
 ="span"],.row-fluid input[class*="span"],.row-fluid select[class*="span"],.row-fluid textarea[class*="span"],.row-fluid .uneditable-input[class*="span"]{float:none;margin-left:0}.input-append input[class*="span"],.input-append .uneditable-input[class*="span"],.input-prepend input[class*="span"],.input-prepend .uneditable-input[class*="span"],.row-fluid input[class*="span"],.row-fluid select[class*="span"],.row-fluid textarea[class*="span"],.row-fluid .uneditable-input[class*="span"],.row-fluid .input-prepend [class*="span"],.row-fluid .input-append [class*="span"]{display:inline-block}input,textarea,.uneditable-input{margin-left:0}.controls-row [class*="span"]+[class*="span"]{margin-left:20px}input.span12,textarea.span12,.uneditable-input.span12{width:926px}input.span11,textarea.span11,.uneditable-input.span11{width:846px}input.span10,textarea.span10,.uneditable-input.span10{width:766px}input.span9,textarea.span9,.uneditable-input.span9{width:686px}input.span8,textarea.span8,.unedit
 able-input.span8{width:606px}input.span7,textarea.span7,.uneditable-input.span7{width:526px}input.span6,textarea.span6,.uneditable-input.span6{width:446px}input.span5,textarea.span5,.uneditable-input.span5{width:366px}input.span4,textarea.span4,.uneditable-input.span4{width:286px}input.span3,textarea.span3,.uneditable-input.span3{width:206px}input.span2,textarea.span2,.uneditable-input.span2{width:126px}input.span1,textarea.span1,.uneditable-input.span1{width:46px}.controls-row{*zoom:1}.controls-row:before,.controls-row:after{display:table;line-height:0;content:""}.controls-row:after{clear:both}.controls-row [class*="span"]{float:left}input[disabled],select[disabled],textarea[disabled],input[readonly],select[readonly],textarea[readonly]{cursor:not-allowed;background-color:#eee}input[type="radio"][disabled],input[type="checkbox"][disabled],input[type="radio"][readonly],input[type="checkbox"][readonly]{background-color:transparent}.control-group.warning>label,.control-group.warning .h
 elp-block,.control-group.warning .help-inline{color:#c09853}.control-group.warning .checkbox,.control-group.warning .radio,.control-group.warning input,.control-group.warning select,.control-group.warning textarea{color:#c09853;border-color:#c09853;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.warning .checkbox:focus,.control-group.warning .radio:focus,.control-group.warning input:focus,.control-group.warning select:focus,.control-group.warning textarea:focus{border-color:#a47e3c;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e}.control-group.warning .input-prepend .add-on,.control-group.warning .input-append .add-on{color:#c09853;background-color:#fcf8e3;border-color:#c09853}.control-group.error>label,.control-group.error 
 .help-block,.control-group.error .help-inline{color:#b94a48}.control-group.error .checkbox,.control-group.error .radio,.control-group.error input,.control-group.error select,.control-group.error textarea{color:#b94a48;border-color:#b94a48;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.error .checkbox:focus,.control-group.error .radio:focus,.control-group.error input:focus,.control-group.error select:focus,.control-group.error textarea:focus{border-color:#953b39;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392}.control-group.error .input-prepend .add-on,.control-group.error .input-append .add-on{color:#b94a48;background-color:#f2dede;border-color:#b94a48}.control-group.success>label,.control-group.success .help-block,.control
 -group.success .help-inline{color:#468847}.control-group.success .checkbox,.control-group.success .radio,.control-group.success input,.control-group.success select,.control-group.success textarea{color:#468847;border-color:#468847;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.success .checkbox:focus,.control-group.success .radio:focus,.control-group.success input:focus,.control-group.success select:focus,.control-group.success textarea:focus{border-color:#356635;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b}.control-group.success .input-prepend .add-on,.control-group.success .input-append .add-on{color:#468847;background-color:#dff0d8;border-color:#468847}input:focus:required:invalid,textarea:focus:required:invalid,selec
 t:focus:required:invalid{color:#b94a48;border-color:#ee5f5b}input:focus:required:invalid:focus,textarea:focus:required:invalid:focus,select:focus:required:invalid:focus{border-color:#e9322d;-webkit-box-shadow:0 0 6px #f8b9b7;-moz-box-shadow:0 0 6px #f8b9b7;box-shadow:0 0 6px #f8b9b7}.form-actions{padding:19px 20px 20px;margin-top:20px;margin-bottom:20px;background-color:#f5f5f5;border-top:1px solid #e5e5e5;*zoom:1}.form-actions:before,.form-actions:after{display:table;line-height:0;content:""}.form-actions:after{clear:both}.help-block,.help-inline{color:#595959}.help-block{display:block;margin-bottom:10px}.help-inline{display:inline-block;*display:inline;padding-left:5px;vertical-align:middle;*zoom:1}.input-append,.input-prepend{margin-bottom:5px;font-size:0;white-space:nowrap}.input-append input,.input-prepend input,.input-append select,.input-prepend select,.input-append .uneditable-input,.input-prepend .uneditable-input{position:relative;margin-bottom:0;*margin-left:0;font-size:1
 4px;vertical-align:top;-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}.input-append input:focus,.input-prepend input:focus,.input-append select:focus,.input-prepend select:focus,.input-append .uneditable-input:focus,.input-prepend .uneditable-input:focus{z-index:2}.input-append .add-on,.input-prepend .add-on{display:inline-block;width:auto;height:20px;min-width:16px;padding:4px 5px;font-size:14px;font-weight:normal;line-height:20px;text-align:center;text-shadow:0 1px 0 #fff;background-color:#eee;border:1px solid #ccc}.input-append .add-on,.input-prepend .add-on,.input-append .btn,.input-prepend .btn{margin-left:-1px;vertical-align:top;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.input-append .active,.input-prepend .active{background-color:#a9dba9;border-color:#46a546}.input-prepend .add-on,.input-prepend .btn{margin-right:-1px}.input-prepend .add-on:first-child,.input-prepend .btn:first-child{-webkit-border-radius:3px 0 0 
 3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.input-append input,.input-append select,.input-append .uneditable-input{-webkit-border-radius:3px 0 0 3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.input-append .add-on:last-child,.input-append .btn:last-child{-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}.input-prepend.input-append input,.input-prepend.input-append select,.input-prepend.input-append .uneditable-input{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.input-prepend.input-append .add-on:first-child,.input-prepend.input-append .btn:first-child{margin-right:-1px;-webkit-border-radius:3px 0 0 3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.input-prepend.input-append .add-on:last-child,.input-prepend.input-append .btn:last-child{margin-left:-1px;-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}input.search-query{padding-right:14px;pad
 ding-right:4px \9;padding-left:14px;padding-left:4px \9;margin-bottom:0;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.form-search .input-append .search-query,.form-search .input-prepend .search-query{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.form-search .input-append .search-query{-webkit-border-radius:14px 0 0 14px;-moz-border-radius:14px 0 0 14px;border-radius:14px 0 0 14px}.form-search .input-append .btn{-webkit-border-radius:0 14px 14px 0;-moz-border-radius:0 14px 14px 0;border-radius:0 14px 14px 0}.form-search .input-prepend .search-query{-webkit-border-radius:0 14px 14px 0;-moz-border-radius:0 14px 14px 0;border-radius:0 14px 14px 0}.form-search .input-prepend .btn{-webkit-border-radius:14px 0 0 14px;-moz-border-radius:14px 0 0 14px;border-radius:14px 0 0 14px}.form-search input,.form-inline input,.form-horizontal input,.form-search textarea,.form-inline textarea,.form-horizontal textarea,.form-search select,.form-inline select,.for
 m-horizontal select,.form-search .help-inline,.form-inline .help-inline,.form-horizontal .help-inline,.form-search .uneditable-input,.form-inline .uneditable-input,.form-horizontal .uneditable-input,.form-search .input-prepend,.form-inline .input-prepend,.form-horizontal .input-prepend,.form-search .input-append,.form-inline .input-append,.form-horizontal .input-append{display:inline-block;*display:inline;margin-bottom:0;vertical-align:middle;*zoom:1}.form-search .hide,.form-inline .hide,.form-horizontal .hide{display:none}.form-search label,.form-inline label,.form-search .btn-group,.form-inline .btn-group{display:inline-block}.form-search .input-append,.form-inline .input-append,.form-search .input-prepend,.form-inline .input-prepend{margin-bottom:0}.form-search .radio,.form-search .checkbox,.form-inline .radio,.form-inline .checkbox{padding-left:0;margin-bottom:0;vertical-align:middle}.form-search .radio input[type="radio"],.form-search .checkbox input[type="checkbox"],.form-inli
 ne .radio input[type="radio"],.form-inline .checkbox input[type="checkbox"]{float:left;margin-right:3px;margin-left:0}.control-group{margin-bottom:10px}legend+.control-group{margin-top:20px;-webkit-margin-top-collapse:separate}.form-horizontal .control-group{margin-bottom:20px;*zoom:1}.form-horizontal .control-group:before,.form-horizontal .control-group:after{display:table;line-height:0;content:""}.form-horizontal .control-group:after{clear:both}.form-horizontal .control-label{float:left;width:140px;padding-top:5px;text-align:right}.form-horizontal .controls{*display:inline-block;*padding-left:20px;margin-left:160px;*margin-left:0}.form-horizontal .controls:first-child{*padding-left:160px}.form-horizontal .help-block{margin-top:10px;margin-bottom:0}.form-horizontal .form-actions{padding-left:160px}table{max-width:100%;background-color:transparent;border-collapse:collapse;border-spacing:0}.table{width:100%;margin-bottom:20px}.table th,.table td{padding:8px;line-height:20px;text-alig
 n:left;vertical-align:top;border-top:1px solid #ddd}.table th{font-weight:bold}.table thead th{vertical-align:bottom}.table caption+thead tr:first-child th,.table caption+thead tr:first-child td,.table colgroup+thead tr:first-child th,.table colgroup+thead tr:first-child td,.table thead:first-child tr:first-child th,.table thead:first-child tr:first-child td{border-top:0}.table tbody+tbody{border-top:2px solid #ddd}.table-condensed th,.table-condensed td{padding:4px 5px}.table-bordered{border:1px solid #ddd;border-collapse:separate;*border-collapse:collapse;border-left:0;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.table-bordered th,.table-bordered td{border-left:1px solid #ddd}.table-bordered caption+thead tr:first-child th,.table-bordered caption+tbody tr:first-child th,.table-bordered caption+tbody tr:first-child td,.table-bordered colgroup+thead tr:first-child th,.table-bordered colgroup+tbody tr:first-child th,.table-bordered colgroup+tbody tr:first-child
  td,.table-bordered thead:first-child tr:first-child th,.table-bordered tbody:first-child tr:first-child th,.table-bordered tbody:first-child tr:first-child td{border-top:0}.table-bordered thead:first-child tr:first-child th:first-child,.table-bordered tbody:first-child tr:first-child td:first-child{-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-topleft:4px}.table-bordered thead:first-child tr:first-child th:last-child,.table-bordered tbody:first-child tr:first-child td:last-child{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-moz-border-radius-topright:4px}.table-bordered thead:last-child tr:last-child th:first-child,.table-bordered tbody:last-child tr:last-child td:first-child,.table-bordered tfoot:last-child tr:last-child td:first-child{-webkit-border-radius:0 0 0 4px;-moz-border-radius:0 0 0 4px;border-radius:0 0 0 4px;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-moz-border-radius-bottomleft:4px}.table-b
 ordered thead:last-child tr:last-child th:last-child,.table-bordered tbody:last-child tr:last-child td:last-child,.table-bordered tfoot:last-child tr:last-child td:last-child{-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-moz-border-radius-bottomright:4px}.table-bordered caption+thead tr:first-child th:first-child,.table-bordered caption+tbody tr:first-child td:first-child,.table-bordered colgroup+thead tr:first-child th:first-child,.table-bordered colgroup+tbody tr:first-child td:first-child{-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-topleft:4px}.table-bordered caption+thead tr:first-child th:last-child,.table-bordered caption+tbody tr:first-child td:last-child,.table-bordered colgroup+thead tr:first-child th:last-child,.table-bordered colgroup+tbody tr:first-child td:last-child{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-moz-border-right-topleft:4px}.table-striped tbody tr:nth-child(odd) td,.table-s
 triped tbody tr:nth-child(odd) th{background-color:#f9f9f9}.table-hover tbody tr:hover td,.table-hover tbody tr:hover th{background-color:#f5f5f5}table [class*=span],.row-fluid table [class*=span]{display:table-cell;float:none;margin-left:0}table .span1{float:none;width:44px;margin-left:0}table .span2{float:none;width:124px;margin-left:0}table .span3{float:none;width:204px;margin-left:0}table .span4{float:none;width:284px;margin-left:0}table .span5{float:none;width:364px;margin-left:0}table .span6{float:none;width:444px;margin-left:0}table .span7{float:none;width:524px;margin-left:0}table .span8{float:none;width:604px;margin-left:0}table .span9{float:none;width:684px;margin-left:0}table .span10{float:none;width:764px;margin-left:0}table .span11{float:none;width:844px;margin-left:0}table .span12{float:none;width:924px;margin-left:0}table .span13{float:none;width:1004px;margin-left:0}table .span14{float:none;width:1084px;margin-left:0}table .span15{float:none;width:1164px;margin-left:
 0}table .span16{float:none;width:1244px;margin-left:0}table .span17{float:none;width:1324px;margin-left:0}table .span18{float:none;width:1404px;margin-left:0}table .span19{float:none;width:1484px;margin-left:0}table .span20{float:none;width:1564px;margin-left:0}table .span21{float:none;width:1644px;margin-left:0}table .span22{float:none;width:1724px;margin-left:0}table .span23{float:none;width:1804px;margin-left:0}table .span24{float:none;width:1884px;margin-left:0}.table tbody tr.success td{background-color:#dff0d8}.table tbody tr.error td{background-color:#f2dede}.table tbody tr.info td{background-color:#d9edf7}[class^="icon-"],[class*=" icon-"]{display:inline-block;width:14px;height:14px;margin-top:1px;*margin-right:.3em;line-height:14px;vertical-align:text-top;background-image:url("../img/glyphicons-halflings.png");background-position:14px 14px;background-repeat:no-repeat}.icon-white,.nav>.active>a>[class^="icon-"],.nav>.active>a>[class*=" icon-"],.dropdown-menu>li>a:hover>[clas
 s^="icon-"],.dropdown-menu>li>a:hover>[class*=" icon-"],.dropdown-menu>.active>a>[class^="icon-"],.dropdown-menu>.active>a>[class*=" icon-"]{background-image:url("../img/glyphicons-halflings-white.png")}.icon-glass{background-position:0 0}.icon-music{background-position:-24px 0}.icon-search{background-position:-48px 0}.icon-envelope{background-position:-72px 0}.icon-heart{background-position:-96px 0}.icon-star{background-position:-120px 0}.icon-star-empty{background-position:-144px 0}.icon-user{background-position:-168px 0}.icon-film{background-position:-192px 0}.icon-th-large{background-position:-216px 0}.icon-th{background-position:-240px 0}.icon-th-list{background-position:-264px 0}.icon-ok{background-position:-288px 0}.icon-remove{background-position:-312px 0}.icon-zoom-in{background-position:-336px 0}.icon-zoom-out{background-position:-360px 0}.icon-off{background-position:-384px 0}.icon-signal{background-position:-408px 0}.icon-cog{background-position:-432px 0}.icon-trash{back
 ground-position:-456px 0}.icon-home{background-position:0 -24px}.icon-file{background-position:-24px -24px}.icon-time{background-position:-48px -24px}.icon-road{background-position:-72px -24px}.icon-download-alt{background-position:-96px -24px}.icon-download{background-position:-120px -24px}.icon-upload{background-position:-144px -24px}.icon-inbox{background-position:-168px -24px}.icon-play-circle{background-position:-192px -24px}.icon-repeat{background-position:-216px -24px}.icon-refresh{background-position:-240px -24px}.icon-list-alt{background-position:-264px -24px}.icon-lock{background-position:-287px -24px}.icon-flag{background-position:-312px -24px}.icon-headphones{background-position:-336px -24px}.icon-volume-off{background-position:-360px -24px}.icon-volume-down{background-position:-384px -24px}.icon-volume-up{background-position:-408px -24px}.icon-qrcode{background-position:-432px -24px}.icon-barcode{background-position:-456px -24px}.icon-tag{background-position:0 -48px}.ic
 on-tags{background-position:-25px -48px}.icon-book{background-position:-48px -48px}.icon-bookmark{background-position:-72px -48px}.icon-print{background-position:-96px -48px}.icon-camera{background-position:-120px -48px}.icon-font{background-position:-144px -48px}.icon-bold{background-position:-167px -48px}.icon-italic{background-position:-192px -48px}.icon-text-height{background-position:-216px -48px}.icon-text-width{background-position:-240px -48px}.icon-align-left{background-position:-264px -48px}.icon-align-center{background-position:-288px -48px}.icon-align-right{background-position:-312px -48px}.icon-align-justify{background-position:-336px -48px}.icon-list{background-position:-360px -48px}.icon-indent-left{background-position:-384px -48px}.icon-indent-right{background-position:-408px -48px}.icon-facetime-video{background-position:-432px -48px}.icon-picture{background-position:-456px -48px}.icon-pencil{background-position:0 -72px}.icon-map-marker{background-position:-24px -72p
 x}.icon-adjust{background-position:-48px -72px}.icon-tint{background-position:-72px -72px}.icon-edit{background-position:-96px -72px}.icon-share{background-position:-120px -72px}.icon-check{background-position:-144px -72px}.icon-move{background-position:-168px -72px}.icon-step-backward{background-position:-192px -72px}.icon-fast-backward{background-position:-216px -72px}.icon-backward{background-position:-240px -72px}.icon-play{background-position:-264px -72px}.icon-pause{background-position:-288px -72px}.icon-stop{background-position:-312px -72px}.icon-forward{background-position:-336px -72px}.icon-fast-forward{background-position:-360px -72px}.icon-step-forward{background-position:-384px -72px}.icon-eject{background-position:-408px -72px}.icon-chevron-left{background-position:-432px -72px}.icon-chevron-right{background-position:-456px -72px}.icon-plus-sign{background-position:0 -96px}.icon-minus-sign{background-position:-24px -96px}.icon-remove-sign{background-position:-48px -96px
 }.icon-ok-sign{background-position:-72px -96px}.icon-question-sign{background-position:-96px -96px}.icon-info-sign{background-position:-120px -96px}.icon-screenshot{background-position:-144px -96px}.icon-remove-circle{background-position:-168px -96px}.icon-ok-circle{background-position:-192px -96px}.icon-ban-circle{background-position:-216px -96px}.icon-arrow-left{background-position:-240px -96px}.icon-arrow-right{background-position:-264px -96px}.icon-arrow-up{background-position:-289px -96px}.icon-arrow-down{background-position:-312px -96px}.icon-share-alt{background-position:-336px -96px}.icon-resize-full{background-position:-360px -96px}.icon-resize-small{background-position:-384px -96px}.icon-plus{background-position:-408px -96px}.icon-minus{background-position:-433px -96px}.icon-asterisk{background-position:-456px -96px}.icon-exclamation-sign{background-position:0 -120px}.icon-gift{background-position:-24px -120px}.icon-leaf{background-position:-48px -120px}.icon-fire{backgrou
 nd-position:-72px -120px}.icon-eye-open{background-position:-96px -120px}.icon-eye-close{background-position:-120px -120px}.icon-warning-sign{background-position:-144px -120px}.icon-plane{background-position:-168px -120px}.icon-calendar{background-position:-192px -120px}.icon-random{width:16px;background-position:-216px -120px}.icon-comment{background-position:-240px -120px}.icon-magnet{background-position:-264px -120px}.icon-chevron-up{background-position:-288px -120px}.icon-chevron-down{background-position:-313px -119px}.icon-retweet{background-position:-336px -120px}.icon-shopping-cart{background-position:-360px -120px}.icon-folder-close{background-position:-384px -120px}.icon-folder-open{width:16px;background-position:-408px -120px}.icon-resize-vertical{background-position:-432px -119px}.icon-resize-horizontal{background-position:-456px -118px}.icon-hdd{background-position:0 -144px}.icon-bullhorn{background-position:-24px -144px}.icon-bell{background-position:-48px -144px}.icon-
 certificate{background-position:-72px -144px}.icon-thumbs-up{background-position:-96px -144px}.icon-thumbs-down{background-position:-120px -144px}.icon-hand-right{background-position:-144px -144px}.icon-hand-left{background-position:-168px -144px}.icon-hand-up{background-position:-192px -144px}.icon-hand-down{background-position:-216px -144px}.icon-circle-arrow-right{background-position:-240px -144px}.icon-circle-arrow-left{background-position:-264px -144px}.icon-circle-arrow-up{background-position:-288px -144px}.icon-circle-arrow-down{background-position:-312px -144px}.icon-globe{background-position:-336px -144px}.icon-wrench{background-position:-360px -144px}.icon-tasks{background-position:-384px -144px}.icon-filter{background-position:-408px -144px}.icon-briefcase{background-position:-432px -144px}.icon-fullscreen{background-position:-456px -144px}.dropup,.dropdown{position:relative}.dropdown-toggle{*margin-bottom:-3px}.dropdown-toggle:active,.open .dropdown-toggle{outline:0}.car
 et{display:inline-block;width:0;height:0;vertical-align:top;border-top:4px solid #000;border-right:4px solid transparent;border-left:4px solid transparent;content:""}.dropdown .caret{margin-top:8px;margin-left:2px}.dropdown-menu{position:absolute;top:100%;left:0;z-index:1000;display:none;float:left;min-width:160px;padding:5px 0;margin:2px 0 0;list-style:none;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);*border-right-width:2px;*border-bottom-width:2px;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 5px 10px rgba(0,0,0,0.2);-moz-box-shadow:0 5px 10px rgba(0,0,0,0.2);box-shadow:0 5px 10px rgba(0,0,0,0.2);-webkit-background-clip:padding-box;-moz-background-clip:padding;background-clip:padding-box}.dropdown-menu.pull-right{right:0;left:auto}.dropdown-menu .divider{*width:100%;height:1px;margin:9px 1px;*margin:-5px 0 5px;overflow:hidden;background-color:#e5e5e5;border-bottom:1px solid #fff}.dropdown-menu a{display:blo
 ck;padding:3px 20px;clear:both;font-weight:normal;line-height:20px;color:#333;white-space:nowrap}.dropdown-menu li>a:hover,.dropdown-menu li>a:focus,.dropdown-submenu:hover>a{color:#fff;text-decoration:none;background-color:#08c;background-color:#0081c2;background-image:-moz-linear-gradient(top,#08c,#0077b3);background-image:-webkit-gradient(linear,0 0,0 100%,from(#08c),to(#0077b3));background-image:-webkit-linear-gradient(top,#08c,#0077b3);background-image:-o-linear-gradient(top,#08c,#0077b3);background-image:linear-gradient(to bottom,#08c,#0077b3);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0077b3',GradientType=0)}.dropdown-menu .active>a,.dropdown-menu .active>a:hover{color:#fff;text-decoration:none;background-color:#08c;background-color:#0081c2;background-image:linear-gradient(to bottom,#08c,#0077b3);background-image:-moz-linear-gradient(top,#08c,#0077b3);background-image:-webkit-gradient(linear,0 0,0 100
 %,from(#08c),to(#0077b3));background-image:-webkit-linear-gradient(top,#08c,#0077b3);background-image:-o-linear-gradient(top,#08c,#0077b3);background-repeat:repeat-x;outline:0;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0077b3',GradientType=0)}.dropdown-menu .disabled>a,.dropdown-menu .disabled>a:hover{color:#999}.dropdown-menu .disabled>a:hover{text-decoration:none;cursor:default;background-color:transparent}.open{*z-index:1000}.open>.dropdown-menu{display:block}.pull-right>.dropdown-menu{right:0;left:auto}.dropup .caret,.navbar-fixed-bottom .dropdown .caret{border-top:0;border-bottom:4px solid #000;content:"\2191"}.dropup .dropdown-menu,.navbar-fixed-bottom .dropdown .dropdown-menu{top:auto;bottom:100%;margin-bottom:1px}.dropdown-submenu{position:relative}.dropdown-submenu>.dropdown-menu{top:0;left:100%;margin-top:-6px;margin-left:-1px;-webkit-border-radius:0 6px 6px 6px;-moz-border-radius:0 6px 6px 6px;border-radius:0 6px 6px 6px}.d
 ropdown-submenu:hover .dropdown-menu{display:block}.dropdown-submenu>a:after{display:block;float:right;width:0;height:0;margin-top:5px;margin-right:-10px;border-color:transparent;border-left-color:#ccc;border-style:solid;border-width:5px 0 5px 5px;content:" "}.dropdown-submenu:hover>a:after{border-left-color:#fff}.dropdown .dropdown-menu .nav-header{padding-right:20px;padding-left:20px}.typeahead{margin-top:2px;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.well{min-height:20px;padding:19px;margin-bottom:20px;background-color:#f5f5f5;border:1px solid #e3e3e3;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.05);box-shadow:inset 0 1px 1px rgba(0,0,0,0.05)}.well blockquote{border-color:#ddd;border-color:rgba(0,0,0,0.15)}.well-large{padding:24px;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.well-small{padding:9px;-webkit-border-rad
 ius:3px;-moz-border-radius:3px;border-radius:3px}.fade{opacity:0;-webkit-transition:opacity .15s linear;-moz-transition:opacity .15s linear;-o-transition:opacity .15s linear;transition:opacity .15s linear}.fade.in{opacity:1}.collapse{position:relative;height:0;overflow:hidden;overflow:visible \9;-webkit-transition:height .35s ease;-moz-transition:height .35s ease;-o-transition:height .35s ease;transition:height .35s ease}.collapse.in{height:auto}.close{float:right;font-size:20px;font-weight:bold;line-height:20px;color:#000;text-shadow:0 1px 0 #fff;opacity:.2;filter:alpha(opacity=20)}.close:hover{color:#000;text-decoration:none;cursor:pointer;opacity:.4;filter:alpha(opacity=40)}button.close{padding:0;cursor:pointer;background:transparent;border:0;-webkit-appearance:none}.btn{display:inline-block;*display:inline;padding:4px 14px;margin-bottom:0;*margin-left:.3em;font-size:14px;line-height:20px;*line-height:20px;color:#333;text-align:center;text-shadow:0 1px 1px rgba(255,255,255,0.75);
 vertical-align:middle;cursor:pointer;background-color:#f5f5f5;*background-color:#e6e6e6;background-image:-webkit-gradient(linear,0 0,0 100%,from(#fff),to(#e6e6e6));background-image:-webkit-linear-gradient(top,#fff,#e6e6e6);background-image:-o-linear-gradient(top,#fff,#e6e6e6);background-image:linear-gradient(to bottom,#fff,#e6e6e6);background-image:-moz-linear-gradient(top,#fff,#e6e6e6);background-repeat:repeat-x;border:1px solid #bbb;*border:0;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);border-color:#e6e6e6 #e6e6e6 #bfbfbf;border-bottom-color:#a2a2a2;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffffffff',endColorstr='#ffe6e6e6',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false);*zoom:1;-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);box-
 shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05)}.btn:hover,.btn:active,.btn.active,.btn.disabled,.btn[disabled]{color:#333;background-color:#e6e6e6;*background-color:#d9d9d9}.btn:active,.btn.active{background-color:#ccc \9}.btn:first-child{*margin-left:0}.btn:hover{color:#333;text-decoration:none;background-color:#e6e6e6;*background-color:#d9d9d9;background-position:0 -15px;-webkit-transition:background-position .1s linear;-moz-transition:background-position .1s linear;-o-transition:background-position .1s linear;transition:background-position .1s linear}.btn:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}.btn.active,.btn:active{background-color:#e6e6e6;background-color:#d9d9d9 \9;background-image:none;outline:0;-webkit-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2p
 x rgba(0,0,0,0.05)}.btn.disabled,.btn[disabled]{cursor:default;background-color:#e6e6e6;background-image:none;opacity:.65;filter:alpha(opacity=65);-webkit-box-shadow:none;-moz-box-shadow:none;box-shadow:none}.btn-large{padding:9px 14px;font-size:16px;line-height:normal;-webkit-border-radius:5px;-moz-border-radius:5px;border-radius:5px}.btn-large [class^="icon-"]{margin-top:2px}.btn-small{padding:3px 9px;font-size:12px;line-height:18px}.btn-small [class^="icon-"]{margin-top:0}.btn-mini{padding:2px 6px;font-size:11px;line-height:16px}.btn-block{display:block;width:100%;padding-right:0;padding-left:0;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.btn-block+.btn-block{margin-top:5px}.btn-primary.active,.btn-warning.active,.btn-danger.active,.btn-success.active,.btn-info.active,.btn-inverse.active{color:rgba(255,255,255,0.75)}.btn{border-color:#c5c5c5;border-color:rgba(0,0,0,0.15) rgba(0,0,0,0.15) rgba(0,0,0,0.25)}.btn-primary{color:#fff;text-shadow:0 -1p
 x 0 rgba(0,0,0,0.25);background-color:#006dcc;*background-color:#04c;background-image:-webkit-gradient(linear,0 0,0 100%,from(#08c),to(#04c));background-image:-webkit-linear-gradient(top,#08c,#04c);background-image:-o-linear-gradient(top,#08c,#04c);background-image:linear-gradient(to bottom,#08c,#04c);background-image:-moz-linear-gradient(top,#08c,#04c);background-repeat:repeat-x;border-color:#04c #0044cc #002a80;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0044cc',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-primary:hover,.btn-primary:active,.btn-primary.active,.btn-primary.disabled,.btn-primary[disabled]{color:#fff;background-color:#04c;*background-color:#003bb3}.btn-primary:active,.btn-primary.active{background-color:#039 \9}.btn-warning{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#faa732;*background-color:#f89
 406;background-image:-webkit-gradient(linear,0 0,0 100%,from(#fbb450),to(#f89406));background-image:-webkit-linear-gradient(top,#fbb450,#f89406);background-image:-o-linear-gradient(top,#fbb450,#f89406);background-image:linear-gradient(to bottom,#fbb450,#f89406);background-image:-moz-linear-gradient(top,#fbb450,#f89406);background-repeat:repeat-x;border-color:#f89406 #f89406 #ad6704;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#fffbb450',endColorstr='#fff89406',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-warning:hover,.btn-warning:active,.btn-warning.active,.btn-warning.disabled,.btn-warning[disabled]{color:#fff;background-color:#f89406;*background-color:#df8505}.btn-warning:active,.btn-warning.active{background-color:#c67605 \9}.btn-danger{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#da4f49;*background-color:#bd362f;background-image:-webki
 t-gradient(linear,0 0,0 100%,from(#ee5f5b),to(#bd362f));background-image:-webkit-linear-gradient(top,#ee5f5b,#bd362f);background-image:-o-linear-gradient(top,#ee5f5b,#bd362f);background-image:linear-gradient(to bottom,#ee5f5b,#bd362f);background-image:-moz-linear-gradient(top,#ee5f5b,#bd362f);background-repeat:repeat-x;border-color:#bd362f #bd362f #802420;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffee5f5b',endColorstr='#ffbd362f',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-danger:hover,.btn-danger:active,.btn-danger.active,.btn-danger.disabled,.btn-danger[disabled]{color:#fff;background-color:#bd362f;*background-color:#a9302a}.btn-danger:active,.btn-danger.active{background-color:#942a25 \9}.btn-success{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#5bb75b;*background-color:#51a351;background-image:-webkit-gradient(linear,0 0,0 100%,from
 (#62c462),to(#51a351));background-image:-webkit-linear-gradient(top,#62c462,#51a351);background-image:-o-linear-gradient(top,#62c462,#51a351);background-image:linear-gradient(to bottom,#62c462,#51a351);background-image:-moz-linear-gradient(top,#62c462,#51a351);background-repeat:repeat-x;border-color:#51a351 #51a351 #387038;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff62c462',endColorstr='#ff51a351',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-success:hover,.btn-success:active,.btn-success.active,.btn-success.disabled,.btn-success[disabled]{color:#fff;background-color:#51a351;*background-color:#499249}.btn-success:active,.btn-success.active{background-color:#408140 \9}.btn-info{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#49afcd;*background-color:#2f96b4;background-image:-webkit-gradient(linear,0 0,0 100%,from(#5bc0de),to(#2f96b4));backgr
 ound-image:-webkit-linear-gradient(top,#5bc0de,#2f96b4);background-image:-o-linear-gradient(top,#5bc0de,#2f96b4);background-image:linear-gradient(to bottom,#5bc0de,#2f96b4);background-image:-moz-linear-gradient(top,#5bc0de,#2f96b4);background-repeat:repeat-x;border-color:#2f96b4 #2f96b4 #1f6377;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff5bc0de',endColorstr='#ff2f96b4',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-info:hover,.btn-info:active,.btn-info.active,.btn-info.disabled,.btn-info[disabled]{color:#fff;background-color:#2f96b4;*background-color:#2a85a0}.btn-info:active,.btn-info.active{background-color:#24748c \9}.btn-inverse{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#363636;*background-color:#222;background-image:-webkit-gradient(linear,0 0,0 100%,from(#444),to(#222));background-image:-webkit-linear-gradient(top,#444,#222);backgr
 ound-image:-o-linear-gradient(top,#444,#222);background-image:linear-gradient(to bottom,#444,#222);background-image:-moz-linear-gradient(top,#444,#222);background-repeat:repeat-x;border-color:#222 #222222 #000;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff444444',endColorstr='#ff222222',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-inverse:hover,.btn-inverse:active,.btn-inverse.active,.btn-inverse.disabled,.btn-inverse[disabled]{color:#fff;background-color:#222;*background-color:#151515}.btn-inverse:active,.btn-inverse.active{background-color:#080808 \9}button.btn,input[type="submit"].btn{*padding-top:3px;*padding-bottom:3px}button.btn::-moz-focus-inner,input[type="submit"].btn::-moz-focus-inner{padding:0;border:0}button.btn.btn-large,input[type="submit"].btn.btn-large{*padding-top:7px;*padding-bottom:7px}button.btn.btn-small,input[type="submit"].btn.btn-small{
 *padding-top:3px;*padding-bottom:3px}button.btn.btn-mini,input[type="submit"].btn.btn-mini{*padding-top:1px;*padding-bottom:1px}.btn-link,.btn-link:active{background-color:transparent;background-image:none;-webkit-box-shadow:none;-moz-box-shadow:none;box-shadow:none}.btn-link{color:#08c;cursor:pointer;border-color:transparent;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-link:hover{color:#005580;text-decoration:underline;background-color:transparent}.btn-group{position:relative;*margin-left:.3em;font-size:0;white-space:nowrap}.btn-group:first-child{*margin-left:0}.btn-group+.btn-group{margin-left:5px}.btn-toolbar{margin-top:10px;margin-bottom:10px;font-size:0}.btn-toolbar .btn-group{display:inline-block;*display:inline;*zoom:1}.btn-toolbar .btn+.btn,.btn-toolbar .btn-group+.btn,.btn-toolbar .btn+.btn-group{margin-left:5px}.btn-group>.btn{position:relative;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-group>.btn+.btn{margin-left:-1px}.btn-group
 >.btn,.btn-group>.dropdown-menu{font-size:14px}.btn-group>.btn-mini{font-size:11px}.btn-group>.btn-small{font-size:12px}.btn-group>.btn-large{font-size:16px}.btn-group>.btn:first-child{margin-left:0;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-bottomleft:4px;-moz-border-radius-topleft:4px}.btn-group>.btn:last-child,.btn-group>.dropdown-toggle{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-moz-border-radius-topright:4px;-moz-border-radius-bottomright:4px}.btn-group>.btn.large:first-child{margin-left:0;-webkit-border-bottom-left-radius:6px;border-bottom-left-radius:6px;-webkit-border-top-left-radius:6px;border-top-left-radius:6px;-moz-border-radius-bottomleft:6px;-moz-border-radius-topleft:6px}.btn-group>.btn.large:last-child,.btn-group>.large.dropdown-toggle{-webkit-border-top-right-radius:6px;b
 order-top-right-radius:6px;-webkit-border-bottom-right-radius:6px;border-bottom-right-radius:6px;-moz-border-radius-topright:6px;-moz-border-radius-bottomright:6px}.btn-group>.btn:hover,.btn-group>.btn:focus,.btn-group>.btn:active,.btn-group>.btn.active{z-index:2}.btn-group .dropdown-toggle:active,.btn-group.open .dropdown-toggle{outline:0}.btn-group>.btn+.dropdown-toggle{*padding-top:5px;padding-right:8px;*padding-bottom:5px;padding-left:8px;-webkit-box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05)}.btn-group>.btn-mini+.dropdown-toggle{*padding-top:2px;padding-right:5px;*padding-bottom:2px;padding-left:5px}.btn-group>.btn-small+.dropdown-toggle{*padding-top:5px;*padding-bottom:4px}.btn-group>.btn
 -large+.dropdown-toggle{*padding-top:7px;padding-right:12px;*padding-bottom:7px;padding-left:12px}.btn-group.open .dropdown-toggle{background-image:none;-webkit-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05)}.btn-group.open .btn.dropdown-toggle{background-color:#e6e6e6}.btn-group.open .btn-primary.dropdown-toggle{background-color:#04c}.btn-group.open .btn-warning.dropdown-toggle{background-color:#f89406}.btn-group.open .btn-danger.dropdown-toggle{background-color:#bd362f}.btn-group.open .btn-success.dropdown-toggle{background-color:#51a351}.btn-group.open .btn-info.dropdown-toggle{background-color:#2f96b4}.btn-group.open .btn-inverse.dropdown-toggle{background-color:#222}.btn .caret{margin-top:8px;margin-left:0}.btn-mini .caret,.btn-small .caret,.btn-large .caret{margin-top:6px}.btn-large .caret{border-top-width:5p
 x;border-right-width:5px;border-left-width:5px}.dropup .btn-large .caret{border-top:0;border-bottom:5px solid #000}.btn-primary .caret,.btn-warning .caret,.btn-danger .caret,.btn-info .caret,.btn-success .caret,.btn-inverse .caret{border-top-color:#fff;border-bottom-color:#fff}.btn-group-vertical{display:inline-block;*display:inline;*zoom:1}.btn-group-vertical .btn{display:block;float:none;width:100%;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-group-vertical .btn+.btn{margin-top:-1px;margin-left:0}.btn-group-vertical .btn:first-child{-webkit-border-radius:4px 4px 0 0;-moz-border-radius:4px 4px 0 0;border-radius:4px 4px 0 0}.btn-group-vertical .btn:last-child{-webkit-border-radius:0 0 4px 4px;-moz-border-radius:0 0 4px 4px;border-radius:0 0 4px 4px}.btn-group-vertical .btn-large:first-child{-webkit-border-radius:6px 6px 0 0;-moz-border-radius:6px 6px 0 0;border-radius:6px 6px 0 0}.btn-group-vertical .btn-large:last-child{-webkit-border-radius:0 0 6px 6px;-moz-bo
 rder-radius:0 0 6px 6px;border-radius:0 0 6px 6px}.alert{padding:8px 35px 8px 14px;margin-bottom:20px;color:#c09853;text-shadow:0 1px 0 rgba(255,255,255,0.5);background-color:#fcf8e3;border:1px solid #fbeed5;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.alert h4{margin:0}.alert .close{position:relative;top:-2px;right:-21px;line-height:20px}.alert-success{color:#468847;background-color:#dff0d8;border-color:#d6e9c6}.alert-danger,.alert-error{color:#b94a48;background-color:#f2dede;border-color:#eed3d7}.alert-info{color:#3a87ad;background-color:#d9edf7;border-color:#bce8f1}.alert-block{padding-top:14px;padding-bottom:14px}.alert-block>p,.alert-block>ul{margin-bottom:0}.alert-block p+p{margin-top:5px}.nav{margin-bottom:20px;margin-left:0;list-style:none}.nav>li>a{display:block}.nav>li>a:hover{text-decoration:none;background-color:#eee}.nav>.pull-right{float:right}.nav-header{display:block;padding:3px 15px;font-size:11px;font-weight:bold;line-height:20px;color:#999;t
 ext-shadow:0 1px 0 rgba(255,255,255,0.5);text-transform:uppercase}.nav li+.nav-header{margin-top:9px}.nav-list{padding-right:15px;padding-left:15px;margin-bottom:0}.nav-list>li>a,.nav-list .nav-header{margin-right:-15px;margin-left:-15px;text-shadow:0 1px 0 rgba(255,255,255,0.5)}.nav-list>li>a{padding:3px 15px}.nav-list>.active>a,.nav-list>.active>a:hover{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.2);background-color:#08c}.nav-list [class^="icon-"]{margin-right:2px}.nav-list .divider{*width:100%;height:1px;margin:9px 1px;*margin:-5px 0 5px;overflow:hidden;background-color:#e5e5e5;border-bottom:1px solid #fff}.nav-tabs,.nav-pills{*zoom:1}.nav-tabs:before,.nav-pills:before,.nav-tabs:after,.nav-pills:after{display:table;line-height:0;content:""}.nav-tabs:after,.nav-pills:after{clear:both}.nav-tabs>li,.nav-pills>li{float:left}.nav-tabs>li>a,.nav-pills>li>a{padding-right:12px;padding-left:12px;margin-right:2px;line-height:14px}.nav-tabs{border-bottom:1px solid #ddd}.nav-tabs>li{margin-
 bottom:-1px}.nav-tabs>li>a{padding-top:8px;padding-bottom:8px;line-height:20px;border:1px solid transparent;-webkit-border-radius:4px 4px 0 0;-moz-border-radius:4px 4px 0 0;border-radius:4px 4px 0 0}.nav-tabs>li>a:hover{border-color:#eee #eeeeee #ddd}.nav-tabs>.active>a,.nav-tabs>.active>a:hover{color:#555;cursor:default;background-color:#fff;border:1px solid #ddd;border-bottom-color:transparent}.nav-pills>li>a{padding-top:8px;padding-bottom:8px;margin-top:2px;margin-bottom:2px;-webkit-border-radius:5px;-moz-border-radius:5px;border-radius:5px}.nav-pills>.active>a,.nav-pills>.active>a:hover{color:#fff;background-color:#08c}.nav-stacked>li{float:none}.nav-stacked>li>a{margin-right:0}.nav-tabs.nav-stacked{border-bottom:0}.nav-tabs.nav-stacked>li>a{border:1px solid #ddd;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.nav-tabs.nav-stacked>li:first-child>a{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-webkit-border-top-left-radius:4px;border-top-left-radi
 us:4px;-moz-border-radius-topright:4px;-moz-border-radius-topleft:4px}.nav-tabs.nav-stacked>li:last-child>a{-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-moz-border-radius-bottomright:4px;-moz-border-radius-bottomleft:4px}.nav-tabs.nav-stacked>li>a:hover{z-index:2;border-color:#ddd}.nav-pills.nav-stacked>li>a{margin-bottom:3px}.nav-pills.nav-stacked>li:last-child>a{margin-bottom:1px}.nav-tabs .dropdown-menu{-webkit-border-radius:0 0 6px 6px;-moz-border-radius:0 0 6px 6px;border-radius:0 0 6px 6px}.nav-pills .dropdown-menu{-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.nav .dropdown-toggle .caret{margin-top:6px;border-top-color:#08c;border-bottom-color:#08c}.nav .dropdown-toggle:hover .caret{border-top-color:#005580;border-bottom-color:#005580}.nav-tabs .dropdown-toggle .caret{margin-top:8px}.nav .active .dropdown-toggle .caret{border-top-color:#fff;border-bottom-color:#f
 ff}.nav-tabs .active .dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.nav>.dropdown.active>a:hover{cursor:pointer}.nav-tabs .open .dropdown-toggle,.nav-pills .open .dropdown-toggle,.nav>li.dropdown.open.active>a:hover{color:#fff;background-color:#999;border-color:#999}.nav li.dropdown.open .caret,.nav li.dropdown.open.active .caret,.nav li.dropdown.open a:hover .caret{border-top-color:#fff;border-bottom-color:#fff;opacity:1;filter:alpha(opacity=100)}.tabs-stacked .open>a:hover{border-color:#999}.tabbable{*zoom:1}.tabbable:before,.tabbable:after{display:table;line-height:0;content:""}.tabbable:after{clear:both}.tab-content{overflow:auto}.tabs-below>.nav-tabs,.tabs-right>.nav-tabs,.tabs-left>.nav-tabs{border-bottom:0}.tab-content>.tab-pane,.pill-content>.pill-pane{display:none}.tab-content>.active,.pill-content>.active{display:block}.tabs-below>.nav-tabs{border-top:1px solid #ddd}.tabs-below>.nav-tabs>li{margin-top:-1px;margin-bottom:0}.tabs-below>.nav-tabs>li>a
 {-webkit-border-radius:0 0 4px 4px;-moz-border-radius:0 0 4px 4px;border-radius:0 0 4px 4px}.tabs-below>.nav-tabs>li>a:hover{border-top-color:#ddd;border-bottom-color:transparent}.tabs-below>.nav-tabs>.active>a,.tabs-below>.nav-tabs>.active>a:hover{border-color:transparent #ddd #ddd #ddd}.tabs-left>.nav-tabs>li,.tabs-right>.nav-tabs>li{float:none}.tabs-left>.nav-tabs>li>a,.tabs-right>.nav-tabs>li>a{min-width:74px;margin-right:0;margin-bottom:3px}.tabs-left>.nav-tabs{float:left;margin-right:19px;border-right:1px solid #ddd}.tabs-left>.nav-tabs>li>a{margin-right:-1px;-webkit-border-radius:4px 0 0 4px;-moz-border-radius:4px 0 0 4px;border-radius:4px 0 0 4px}.tabs-left>.nav-tabs>li>a:hover{border-color:#eee #dddddd #eee #eeeeee}.tabs-left>.nav-tabs .active>a,.tabs-left>.nav-tabs .active>a:hover{border-color:#ddd transparent #ddd #ddd;*border-right-color:#fff}.tabs-right>.nav-tabs{float:right;margin-left:19px;border-left:1px solid #ddd}.tabs-right>.nav-tabs>li>a{margin-left:-1px;-webkit-
 border-radius:0 4px 4px 0;-moz-border-radius:0 4px 4px 0;border-radius:0 4px 4px 0}.tabs-right>.nav-tabs>li>a:hover{border-color:#eee #eeeeee #eee #dddddd}.tabs-right>.nav-tabs .active>a,.tabs-right>.nav-tabs .active>a:hover{border-color:#ddd #ddd #ddd transparent;*border-left-color:#fff}.nav>.disabled>a{color:#999}.nav>.disabled>a:hover{text-decoration:none;cursor:default;background-color:transparent}.navbar{*position:relative;*z-index:2;margin-bottom:20px;overflow:visible;color:#555}.navbar-inner{min-height:40px;padding-right:20px;padding-left:20px;background-color:#fafafa;background-image:-moz-linear-gradient(top,#fff,#f2f2f2);background-image:-webkit-gradient(linear,0 0,0 100%,from(#fff),to(#f2f2f2));background-image:-webkit-linear-gradient(top,#fff,#f2f2f2);background-image:-o-linear-gradient(top,#fff,#f2f2f2);background-image:linear-gradient(to bottom,#fff,#f2f2f2);background-repeat:repeat-x;border:1px solid #d4d4d4;-webkit-border-radius:4px;-moz-border-radius:4px;border-radiu
 s:4px;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffffffff',endColorstr='#fff2f2f2',GradientType=0);-webkit-box-shadow:0 1px 4px rgba(0,0,0,0.065);-moz-box-shadow:0 1px 4px rgba(0,0,0,0.065);box-shadow:0 1px 4px rgba(0,0,0,0.065)}.navbar .container{width:auto}.nav-collapse.collapse{height:auto}.navbar .brand{display:block;float:left;padding:10px 20px 10px;margin-left:-20px;font-size:20px;font-weight:200;color:#555;text-shadow:0 1px 0 #fff}.navbar .brand:hover{text-decoration:none}.navbar-text{margin-bottom:0;line-height:40px}.navbar-link{color:#555}.navbar-link:hover{color:#333}.navbar .divider-vertical{height:40px;margin:0 9px;border-right:1px solid #fff;border-left:1px solid #f2f2f2}.navbar .btn,.navbar .btn-group{margin-top:6px}.navbar .btn-group .btn{margin:0}.navbar-form{margin-bottom:0;*zoom:1}.navbar-form:before,.navbar-form:after{display:table;line-height:0;content:""}.navbar-form:after{clear:both}.navbar-form input,.navbar-form select,.navbar-form .rad
 io,.navbar-form .checkbox{margin-top:5px}.navbar-form input,.navbar-form select,.navbar-form .btn{display:inline-block;margin-bottom:0}.navbar-form input[type="image"],.navbar-form input[type="checkbox"],.navbar-form input[type="radio"]{margin-top:3px}.navbar-form .input-append,.navbar-form .input-prepend{margin-top:6px;white-space:nowrap}.navbar-form .input-append input,.navbar-form .input-prepend input{margin-top:0}.navbar-search{position:relative;float:left;margin-top:5px;margin-bottom:0}.navbar-search .search-query{padding:4px 14px;margin-bottom:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:13px;font-weight:normal;line-height:1;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.navbar-static-top{position:static;width:100%;margin-bottom:0}.navbar-static-top .navbar-inner{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.navbar-fixed-top,.navbar-fixed-bottom{position:fixed;right:0;left:0;z-index:1030;margin-bottom:0}.navbar-fix
 ed-top .navbar-inner,.navbar-fixed-bottom .navbar-inner,.navbar-static-top .navbar-inner{border:0}.navbar-fixed-top .navbar-inner,.navbar-fixed-bottom .navbar-inner{padding-right:0;padding-left:0;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:940px}.navbar-fixed-top{top:0}.navbar-fixed-top .navbar-inner,.navbar-static-top .navbar-inner{-webkit-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.1),0 1px 10px rgba(0,0,0,0.1);-moz-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.1),0 1px 10px rgba(0,0,0,0.1);box-shadow:inset 0 -1px 0 rgba(0,0,0,0.1),0 1px 10px rgba(0,0,0,0.1)}.navbar-fixed-bottom{bottom:0}.navbar-fixed-bottom .navbar-inner{-webkit-box-shadow:inset 0 1px 0 rgba(0,0,0,0.1),0 -1px 10px rgba(0,0,0,0.1);-moz-box-shadow:inset 0 1px 0 rgba(0,0,0,0.1),0 -1px 10px rgba(0,0,0,0.1);box-shadow:inset 0 1px 0 rgba(0,0,0,0.1),0 -1px 10px rgba(0,0,0,0.1)}.navbar .nav{position:relative;left:0;displ
 ay:block;float:left;margin:0 10px 0 0}.navbar .nav.pull-right{float:right}.navbar .nav>li{float:left}.navbar .nav>li>a{float:none;padding:10px 15px 10px;color:#555;text-decoration:none;text-shadow:0 1px 0 #fff}.navbar .nav .dropdown-toggle .caret{margin-top:8px}.navbar .nav>li>a:focus,.navbar .nav>li>a:hover{color:#333;text-decoration:none;background-color:transparent}.navbar .nav>.active>a,.navbar .nav>.active>a:hover,.navbar .nav>.active>a:focus{color:#555;text-decoration:none;background-color:#e5e5e5;-webkit-box-shadow:inset 0 3px 8px rgba(0,0,0,0.125);-moz-box-shadow:inset 0 3px 8px rgba(0,0,0,0.125);box-shadow:inset 0 3px 8px rgba(0,0,0,0.125)}.navbar .btn-navbar{display:none;float:right;padding:7px 10px;margin-right:5px;margin-left:5px;color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#ededed;*background-color:#e5e5e5;background-image:-webkit-gradient(linear,0 0,0 100%,from(#f2f2f2),to(#e5e5e5));background-image:-webkit-linear-gradient(top,#f2f2f2,#e5e5e5);back
 ground-image:-o-linear-gradient(top,#f2f2f2,#e5e5e5);background-image:linear-gradient(to bottom,#f2f2f2,#e5e5e5);background-image:-moz-linear-gradient(top,#f2f2f2,#e5e5e5);background-repeat:repeat-x;border-color:#e5e5e5 #e5e5e5 #bfbfbf;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#fff2f2f2',endColorstr='#ffe5e5e5',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false);-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075);-moz-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075);box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075)}.navbar .btn-navbar:hover,.navbar .btn-navbar:active,.navbar .btn-navbar.active,.navbar .btn-navbar.disabled,.navbar .btn-navbar[disabled]{color:#fff;background-color:#e5e5e5;*background-color:#d9d9d9}.navbar .btn-navbar:active,.navbar .btn-navbar.active{background-co
 lor:#ccc \9}.navbar .btn-navbar .icon-bar{display:block;width:18px;height:2px;background-color:#f5f5f5;-webkit-border-radius:1px;-moz-border-radius:1px;border-radius:1px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,0.25);-moz-box-shadow:0 1px 0 rgba(0,0,0,0.25);box-shadow:0 1px 0 rgba(0,0,0,0.25)}.btn-navbar .icon-bar+.icon-bar{margin-top:3px}.navbar .nav>li>.dropdown-menu:before{position:absolute;top:-7px;left:9px;display:inline-block;border-right:7px solid transparent;border-bottom:7px solid #ccc;border-left:7px solid transparent;border-bottom-color:rgba(0,0,0,0.2);content:''}.navbar .nav>li>.dropdown-menu:after{position:absolute;top:-6px;left:10px;display:inline-block;border-right:6px solid transparent;border-bottom:6px solid #fff;border-left:6px solid transparent;content:''}.navbar-fixed-bottom .nav>li>.dropdown-menu:before{top:auto;bottom:-7px;border-top:7px solid #ccc;border-bottom:0;border-top-color:rgba(0,0,0,0.2)}.navbar-fixed-bottom .nav>li>.dropdown-menu:after{top:auto;bottom:-6
 px;border-top:6px solid #fff;border-bottom:0}.navbar .nav li.dropdown.open>.dropdown-toggle,.navbar .nav li.dropdown.active>.dropdown-toggle,.navbar .nav li.dropdown.open.active>.dropdown-toggle{color:#555;background-color:#e5e5e5}.navbar .nav li.dropdown>.dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.navbar .nav li.dropdown.open>.dropdown-toggle .caret,.navbar .nav li.dropdown.active>.dropdown-toggle .caret,.navbar .nav li.dropdown.open.active>.dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.navbar .pull-right>li>.dropdown-menu,.navbar .nav>li>.dropdown-menu.pull-right{right:0;left:auto}.navbar .pull-right>li>.dropdown-menu:before,.navbar .nav>li>.dropdown-menu.pull-right:before{right:12px;left:auto}.navbar .pull-right>li>.dropdown-menu:after,.navbar .nav>li>.dropdown-menu.pull-right:after{right:13px;left:auto}.navbar .pull-right>li>.dropdown-menu .dropdown-menu,.navbar .nav>li>.dropdown-menu.pull-right .dropdown-menu{right:100%;left:
 auto;margin-right:-1px;margin-left:0;-webkit-border-radius:6px 0 6px 6px;-moz-border-radius:6px 0 6px 6px;border-radius:6px 0 6px 6px}.navbar-inverse{color:#999}.navbar-inverse .navbar-inner{background-color:#1b1b1b;background-image:-moz-linear-gradient(top,#222,#111);background-image:-webkit-gradient(linear,0 0,0 100%,from(#222),to(#111));background-image:-webkit-linear-gradient(top,#222,#111);background-image:-o-linear-gradient(top,#222,#111);background-image:linear-gradient(to bottom,#222,#111);background-repeat:repeat-x;border-color:#252525;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff222222',endColorstr='#ff111111',GradientType=0)}.navbar-inverse .brand,.navbar-inverse .nav>li>a{color:#999;text-shadow:0 -1px 0 rgba(0,0,0,0.25)}.navbar-inverse .brand:hover,.navbar-inverse .nav>li>a:hover{color:#fff}.navbar-inverse .nav>li>a:focus,.navbar-inverse .nav>li>a:hover{color:#fff;background-color:transparent}.navbar-inverse .nav .active>a,.navbar-inverse .nav .act
 ive>a:hover,.navbar-inverse .nav .active>a:focus{color:#fff;background-color:#111}.navbar-inverse .navbar-link{color:#999}.navbar-inverse .navbar-link:hover{color:#fff}.navbar-inverse .divider-vertical{border-right-color:#222;border-left-color:#111}.navbar-inverse .nav li.dropdown.open>.dropdown-toggle,.navbar-inverse .nav li.dropdown.active>.dropdown-toggle,.navbar-inverse .nav li.dropdown.open.active>.dropdown-toggle{color:#fff;background-color:#111}.navbar-inverse .nav li.dropdown>.dropdown-toggle .caret{border-top-color:#999;border-bottom-color:#999}.navbar-inverse .nav li.dropdown.open>.dropdown-toggle .caret,.navbar-inverse .nav li.dropdown.active>.dropdown-toggle .caret,.navbar-inverse .nav li.dropdown.open.active>.dropdown-toggle .caret{border-top-color:#fff;border-bottom-color:#fff}.navbar-inverse .navbar-search .search-query{color:#fff;background-color:#515151;border-color:#111;-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);-moz-box-shado
 w:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);box-shadow:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);-webkit-transition:none;-moz-transition:none;-o-transition:none;transition:none}.navbar-inverse .navbar-search .search-query:-moz-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query:-ms-input-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query::-webkit-input-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query:focus,.navbar-inverse .navbar-search .search-query.focused{padding:5px 15px;color:#333;text-shadow:0 1px 0 #fff;background-color:#fff;border:0;outline:0;-webkit-box-shadow:0 0 3px rgba(0,0,0,0.15);-moz-box-shadow:0 0 3px rgba(0,0,0,0.15);box-shadow:0 0 3px rgba(0,0,0,0.15)}.navbar-inverse .btn-navbar{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#0e0e0e;*background-color:#040404;background-image:-webkit-gradient(linear,0 0,0 100%,from(#151515),to(#040404));background-image:-
 webkit-linear-gradient(top,#151515,#040404);background-image:-o-linear-gradient(top,#151515,#040404);background-image:linear-gradient(to bottom,#151515,#040404);background-image:-moz-linear-gradient(top,#151515,#040404);background-repeat:repeat-x;border-color:#040404 #040404 #000;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff151515',endColorstr='#ff040404',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.navbar-inverse .btn-navbar:hover,.navbar-inverse .btn-navbar:active,.navbar-inverse .btn-navbar.active,.navbar-inverse .btn-navbar.disabled,.navbar-inverse .btn-navbar[disabled]{color:#fff;background-color:#040404;*background-color:#000}.navbar-inverse .btn-navbar:active,.navbar-inverse .btn-navbar.active{background-color:#000 \9}.breadcrumb{padding:8px 15px;margin:0 0 20px;list-style:none;background-color:#f5f5f5;-webkit-border-radius:4px;-moz-border-radius:4px;borde
 r-radius:4px}.breadcrumb li{display:inline-block;*display:inline;text-shadow:0 1px 0 #fff;*zoom:1}.breadcrumb .divider{padding:0 5px;color:#ccc}.breadcrumb .active{color:#999}.pagination{height:40px;margin:20px 0}.pagination ul{display:inline-block;*display:inline;margin-bottom:0;margin-left:0;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px;*zoom:1;-webkit-box-shadow:0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:0 1px 2px rgba(0,0,0,0.05);box-shadow:0 1px 2px rgba(0,0,0,0.05)}.pagination li{display:inline}.pagination a,.pagination span{float:left;padding:0 14px;line-height:38px;text-decoration:none;background-color:#fff;border:1px solid #ddd;border-left-width:0}.pagination a:hover,.pagination .active a,.pagination .active span{background-color:#f5f5f5}.pagination .active a,.pagination .active span{color:#999;cursor:default}.pagination .disabled span,.pagination .disabled a,.pagination .disabled a:hover{color:#999;cursor:default;background-color:transparent}.paginatio
 n li:first-child a,.pagination li:first-child span{border-left-width:1px;-webkit-border-radius:3px 0 0 3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.pagination li:last-child a,.pagination li:last-child span{-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}.pagination-centered{text-align:center}.pagination-right{text-align:right}.pager{margin:20px 0;text-align:center;list-style:none;*zoom:1}.pager:before,.pager:after{display:table;line-height:0;content:""}.pager:after{clear:both}.pager li{display:inline}.pager a{display:inline-block;padding:5px 14px;background-color:#fff;border:1px solid #ddd;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.pager a:hover{text-decoration:none;background-color:#f5f5f5}.pager .next a{float:right}.pager .previous a{float:left}.pager .disabled a,.pager .disabled a:hover{color:#999;cursor:default;background-color:#fff}.modal-open .dropdown-menu{z-index:2050}.modal-open .dropdow
 n.open{*z-index:2050}.modal-open .popover{z-index:2060}.modal-open .tooltip{z-index:2080}.modal-backdrop{position:fixed;top:0;right:0;bottom:0;left:0;z-index:1040;background-color:#000}.modal-backdrop.fade{opacity:0}.modal-backdrop,.modal-backdrop.fade.in{opacity:.8;filter:alpha(opacity=80)}.modal{position:fixed;top:50%;left:50%;z-index:1050;width:560px;margin:-250px 0 0 -280px;overflow:auto;background-color:#fff;border:1px solid #999;border:1px solid rgba(0,0,0,0.3);*border:1px solid #999;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 3px 7px rgba(0,0,0,0.3);-moz-box-shadow:0 3px 7px rgba(0,0,0,0.3);box-shadow:0 3px 7px rgba(0,0,0,0.3);-webkit-background-clip:padding-box;-moz-background-clip:padding-box;background-clip:padding-box}.modal.fade{top:-25%;-webkit-transition:opacity .3s linear,top .3s ease-out;-moz-transition:opacity .3s linear,top .3s ease-out;-o-transition:opacity .3s linear,top .3s ease-out;transition:opacity .3s linear,top .3
 s ease-out}.modal.fade.in{top:50%}.modal-header{padding:9px 15px;border-bottom:1px solid #eee}.modal-header .close{margin-top:2px}.modal-header h3{margin:0;line-height:30px}.modal-body{max-height:400px;padding:15px;overflow-y:auto}.modal-form{margin-bottom:0}.modal-footer{padding:14px 15px 15px;margin-bottom:0;text-align:right;background-color:#f5f5f5;border-top:1px solid #ddd;-webkit-border-radius:0 0 6px 6px;-moz-border-radius:0 0 6px 6px;border-radius:0 0 6px 6px;*zoom:1;-webkit-box-shadow:inset 0 1px 0 #fff;-moz-box-shadow:inset 0 1px 0 #fff;box-shadow:inset 0 1px 0 #fff}.modal-footer:before,.modal-footer:after{display:table;line-height:0;content:""}.modal-footer:after{clear:both}.modal-footer .btn+.btn{margin-bottom:0;margin-left:5px}.modal-footer .btn-group .btn+.btn{margin-left:-1px}.tooltip{position:absolute;z-index:1030;display:block;padding:5px;font-size:11px;opacity:0;filter:alpha(opacity=0);visibility:visible}.tooltip.in{opacity:.8;filter:alpha(opacity=80)}.tooltip.top{m
 argin-top:-3px}.tooltip.right{margin-left:3px}.tooltip.bottom{margin-top:3px}.tooltip.left{margin-left:-3px}.tooltip-inner{max-width:200px;padding:3px 8px;color:#fff;text-align:center;text-decoration:none;background-color:#000;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.tooltip-arrow{position:absolute;width:0;height:0;border-color:transparent;border-style:solid}.tooltip.top .tooltip-arrow{bottom:0;left:50%;margin-left:-5px;border-top-color:#000;border-width:5px 5px 0}.tooltip.right .tooltip-arrow{top:50%;left:0;margin-top:-5px;border-right-color:#000;border-width:5px 5px 5px 0}.tooltip.left .tooltip-arrow{top:50%;right:0;margin-top:-5px;border-left-color:#000;border-width:5px 0 5px 5px}.tooltip.bottom .tooltip-arrow{top:0;left:50%;margin-left:-5px;border-bottom-color:#000;border-width:0 5px 5px}.popover{position:absolute;top:0;left:0;z-index:1010;display:none;width:236px;padding:1px;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);
 -webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 5px 10px rgba(0,0,0,0.2);-moz-box-shadow:0 5px 10px rgba(0,0,0,0.2);box-shadow:0 5px 10px rgba(0,0,0,0.2);-webkit-background-clip:padding-box;-moz-background-clip:padding;background-clip:padding-box}.popover.top{margin-bottom:10px}.popover.right{margin-left:10px}.popover.bottom{margin-top:10px}.popover.left{margin-right:10px}.popover-title{padding:8px 14px;margin:0;font-size:14px;font-weight:normal;line-height:18px;background-color:#f7f7f7;border-bottom:1px solid #ebebeb;-webkit-border-radius:5px 5px 0 0;-moz-border-radius:5px 5px 0 0;border-radius:5px 5px 0 0}.popover-content{padding:9px 14px}.popover-content p,.popover-content ul,.popover-content ol{margin-bottom:0}.popover .arrow,.popover .arrow:after{position:absolute;display:inline-block;width:0;height:0;border-color:transparent;border-style:solid}.popover .arrow:after{z-index:-1;content:""}.popover.top .arrow{bottom:-10px;left:50%;margin-le
 ft:-10px;border-top-color:#fff;border-width:10px 10px 0}.popover.top .arrow:after{bottom:-1px;left:-11px;border-top-color:rgba(0,0,0,0.25);border-width:11px 11px 0}.popover.right .arrow{top:50%;left:-10px;margin-top:-10px;border-right-color:#fff;border-width:10px 10px 10px 0}.popover.right .arrow:after{bottom:-11px;left:-1px;border-right-color:rgba(0,0,0,0.25);border-width:11px 11px 11px 0}.popover.bottom .arrow{top:-10px;left:50%;margin-left:-10px;border-bottom-color:#fff;border-width:0 10px 10px}.popover.bottom .arrow:after{top:-1px;left:-11px;border-bottom-color:rgba(0,0,0,0.25);border-width:0 11px 11px}.popover.left .arrow{top:50%;right:-10px;margin-top:-10px;border-left-color:#fff;border-width:10px 0 10px 10px}.popover.left .arrow:after{right:-1px;bottom:-11px;border-left-color:rgba(0,0,0,0.25);border-width:11px 0 11px 11px}.thumbnails{margin-left:-20px;list-style:none;*zoom:1}.thumbnails:before,.thumbnails:after{display:table;line-height:0;content:""}.thumbnails:after{clear:bo
 th}.row-fluid .thumbnails{margin-left:0}.thumbnails>li{float:left;margin-bottom:20px;margin-left:20px}.thumbnail{display:block;padding:4px;line-height:20px;border:1px solid #ddd;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;-webkit-box-shadow:0 1px 3px rgba(0,0,0,0.055);-moz-box-shadow:0 1px 3px rgba(0,0,0,0.055);box-shadow:0 1px 3px rgba(0,0,0,0.055);-webkit-transition:all .2s ease-in-out;-moz-transition:all .2s ease-in-out;-o-transition:all .2s ease-in-out;transition:all .2s ease-in-out}a.thumbnail:hover{border-color:#08c;-webkit-box-shadow:0 1px 4px rgba(0,105,214,0.25);-moz-box-shadow:0 1px 4px rgba(0,105,214,0.25);box-shadow:0 1px 4px rgba(0,105,214,0.25)}.thumbnail>img{display:block;max-width:100%;margin-right:auto;margin-left:auto}.thumbnail .caption{padding:9px;color:#555}.label,.badge{font-size:11.844px;font-weight:bold;line-height:14px;color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);white-space:nowrap;vertical-align:baseline;background-color:#999}.lab
 el{padding:1px 4px 2px;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}.badge{padding:1px 9px 2px;-webkit-border-radius:9px;-moz-border-radius:9px;border-radius:9px}a.label:hover,a.badge:hover{color:#fff;text-decoration:none;cursor:pointer}.label-important,.badge-important{background-color:#b94a48}.label-important[href],.badge-important[href]{background-color:#953b39}.label-warning,.badge-warning{background-color:#f89406}.label-warning[href],.badge-warning[href]{background-color:#c67605}.label-success,.badge-success{background-color:#468847}.label-success[href],.badge-success[href]{background-color:#356635}.label-info,.badge-info{background-color:#3a87ad}.label-info[href],.badge-info[href]{background-color:#2d6987}.label-inverse,.badge-inverse{background-color:#333}.label-inverse[href],.badge-inverse[href]{background-color:#1a1a1a}.btn .label,.btn .badge{position:relative;top:-1px}.btn-mini .label,.btn-mini .badge{top:0}@-webkit-keyframes progress-bar-stripes{from
 {background-position:40px 0}to{background-position:0 0}}@-moz-keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}@-ms-keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}@-o-keyframes progress-bar-stripes{from{background-position:0 0}to{background-position:40px 0}}@keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}.progress{height:20px;margin-bottom:20px;overflow:hidden;background-color:#f7f7f7;background-image:-moz-linear-gradient(top,#f5f5f5,#f9f9f9);background-image:-webkit-gradient(linear,0 0,0 100%,from(#f5f5f5),to(#f9f9f9));background-image:-webkit-linear-gradient(top,#f5f5f5,#f9f9f9);background-image:-o-linear-gradient(top,#f5f5f5,#f9f9f9);background-image:linear-gradient(to bottom,#f5f5f5,#f9f9f9);background-repeat:repeat-x;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff
 f5f5f5',endColorstr='#fff9f9f9',GradientType=0);-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1);-moz-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1);box-shadow:inset 0 1px 2px rgba(0,0,0,0.1)}.progress .bar{float:left;width:0;height:100%;font-size:12px;color:#fff;text-align:center;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#0e90d2;background-image:-moz-linear-gradient(top,#149bdf,#0480be);background-image:-webkit-gradient(linear,0 0,0 100%,from(#149bdf),to(#0480be));background-image:-webkit-linear-gradient(top,#149bdf,#0480be);background-image:-o-linear-gradient(top,#149bdf,#0480be);background-image:linear-gradient(to bottom,#149bdf,#0480be);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff149bdf',endColorstr='#ff0480be',GradientType=0);-webkit-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.15);-moz-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.15);box-shadow:inset 0 -1px 0 rgba(0,0,0,0.15);-webkit-box-sizing:border-box;-moz-box-sizing:border
 -box;box-sizing:border-box;-webkit-transition:width .6s ease;-moz-transition:width .6s ease;-o-transition:width .6s ease;transition:width .6s ease}.progress .bar+.bar{-webkit-box-shadow:inset 1px 0 0 rgba(0,0,0,0.15),inset 0 -1px 0 rgba(0,0,0,0.15);-moz-box-shadow:inset 1px 0 0 rgba(0,0,0,0.15),inset 0 -1px 0 rgba(0,0,0,0.15);box-shadow:inset 1px 0 0 rgba(0,0,0,0.15),inset 0 -1px 0 rgba(0,0,0,0.15)}.progress-striped .bar{background-color:#149bdf;background-image:-webkit-gradient(linear,0 100%,100% 0,color-stop(0.25,rgba(255,255,255,0.15)),color-stop(0.25,transparent),color-stop(0.5,transparent),color-stop(0.5,rgba(255,255,255,0.15)),color-stop(0.75,rgba(255,255,255,0.15)),color-stop(0.75,transparent),to(transparent));background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-moz-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparen
 t 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);-webkit-background-size:40px 40px;-moz-background-size:40px 40px;-o-background-size:40px 40px;background-size:40px 40px}.progress.active .bar{-webkit-animation:progress-bar-stripes 2s linear infinite;-moz-animation:progress-bar-stripes 2s linear infinite;-ms-animation:progress-bar-stripes 2s linear infinite;-o-animation:progress-bar-stripes 2s linear infinite;animation:progress-bar-stripes 2s linear infinite}.progress-danger .bar,.progress .bar-danger{background-color:#dd514c;background-image:-moz-linear-gradient(top,#ee5
 f5b,#c43c35);background-image:-webkit-gradient(linear,0 0,0 100%,from(#ee5f5b),to(#c43c35));background-image:-webkit-linear-gradient(top,#ee5f5b,#c43c35);background-image:-o-linear-gradient(top,#ee5f5b,#c43c35);background-image:linear-gradient(to bottom,#ee5f5b,#c43c35);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffee5f5b',endColorstr='#ffc43c35',GradientType=0)}.progress-danger.progress-striped .bar,.progress-striped .bar-danger{background-color:#ee5f5b;background-image:-webkit-gradient(linear,0 100%,100% 0,color-stop(0.25,rgba(255,255,255,0.15)),color-stop(0.25,transparent),color-stop(0.5,transparent),color-stop(0.5,rgba(255,255,255,0.15)),color-stop(0.75,rgba(255,255,255,0.15)),color-stop(0.75,transparent),to(transparent));background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-moz-l
 inear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent)}.progress-success .bar,.progress .bar-success{background-color:#5eb95e;background-image:-moz-linear-gradient(top,#62c462,#57a957);background-image:-webkit-gradient(linear,0 0,0 100%,from(#62c462),to(#57a957));background-image:-webkit-linear-gradient(top,#62c462,#57a957);background-image:-o-linear-gradient(top,#62c462,#57a957);background-image:linear-gradient(to bottom,#62c462,#57a957);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(star
 tColorstr='#ff62c462',endColorstr='#ff57a957',GradientType=0)}.

<TRUNCATED>

[30/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/ansible/roles/opentaxii/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/ansible/roles/opentaxii/index.html b/site/current-book/metron-deployment/ansible/roles/opentaxii/index.html
new file mode 100644
index 0000000..c3c8fdd
--- /dev/null
+++ b/site/current-book/metron-deployment/ansible/roles/opentaxii/index.html
@@ -0,0 +1,301 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/ansible/roles/opentaxii/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; OpenTAXII</title>
+    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../../css/site.css" />
+    <link rel="stylesheet" href="../../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">OpenTAXII</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-down"></span>Ansible</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/ansible/roles/index.html" title="Roles"><span class="icon-chevron-down"></span>Roles</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>Opentaxii</a></li>
+    <li><a href="../../../../metron-deployment/ansible/roles/pcap_replay/index.html" title="Pcap_replay"><span class="none"></span>Pcap_replay</a></li>
+    <li><a href="../../../../metron-deployment/ansible/roles/sensor-stubs/index.html" title="Sensor-stubs"><span class="none"></span>Sensor-stubs</a></li>
+    <li><a href="../../../../metron-deployment/ansible/roles/sensor-test-mode/index.html" title="Sensor-test-mode"><span class="none"></span>Sensor-test-mode</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>OpenTAXII</h1>
+<p><a name="OpenTAXII"></a></p>
+<p>Installs <a class="externalLink" href="https://github.com/EclecticIQ/OpenTAXII">OpenTAXII</a> as a deamon that can be launched via a SysV service script.  The complementary client implementation, <a class="externalLink" href="https://github.com/EclecticIQ/cabby">Cabby</a> is also installed.</p>
+<p>OpenTAXII is a robust Python implementation of TAXII Services that delivers a rich feature set and friendly pythonic API.  <a class="externalLink" href="https://stixproject.github.io/">TAXII</a> (Trusted Automated eXchange of Indicator Information) is a collection of specifications defining a set of services and message exchanges used for sharing cyber threat intelligence information between parties.</p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>After deployment completes the OpenTAXII service is installed and running.  A set of <a class="externalLink" href="http://hailataxii.com/">Hail a TAXII</a> threat intel collections have been defined and configured.  Use the <tt>status</tt> option to view the collections that have been defined.</p>
+
+<div>
+<div>
+<pre class="source">$ service opentaxii status
+Checking opentaxii...                             Running
+guest.phishtank_com                                0
+guest.Abuse_ch                                     0
+guest.CyberCrime_Tracker                           0
+guest.EmergingThreats_rules                        0
+guest.Lehigh_edu                                   0
+guest.MalwareDomainList_Hostlist                   0
+guest.blutmagie_de_torExits                        0
+guest.dataForLast_7daysOnly                        0
+guest.dshield_BlockList                            0
+</pre></div></div>
+
+<p>Notice that each collections contain zero records.  None of the data is automatically synced during deployment.  To sync the data manually use the <tt>sync</tt> option as defined below.  The following example does not provide a begin and end time so the data will be fetched for the current day only.</p>
+
+<div>
+<div>
+<pre class="source"># service opentaxii sync guest.blutmagie_de_torExits
+2016-04-21 20:34:42,511 INFO: Starting new HTTP connection (1): localhost
+2016-04-21 20:34:42,540 INFO: Response received for Inbox_Message from http://localhost:9000/services/inbox
+2016-04-21 20:34:42,542 INFO: Sending Inbox_Message to http://localhost:9000/services/inbox
+...
+2016-04-21 20:34:42,719 INFO: Response received for Poll_Request from http://localhost:9000/services/poll
+2016-04-21 20:34:42,719 INFO: Content blocks count: 1618, is partial: False
+</pre></div></div>
+
+<p>The OpenTAXII service now contains 1,618 threat intel records indicating Tor Exit nodes.</p>
+
+<div>
+<div>
+<pre class="source">[root@source ~]# service opentaxii status
+Checking opentaxii...                             Running
+guest.phishtank_com                                0
+guest.Abuse_ch                                     0
+guest.CyberCrime_Tracker                           0
+guest.EmergingThreats_rules                        0
+guest.Lehigh_edu                                   0
+guest.MalwareDomainList_Hostlist                   0
+guest.blutmagie_de_torExits                        1618
+guest.dataForLast_7daysOnly                        0
+guest.dshield_BlockList                            0
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Usage"></a>Usage</h2>
+<p>A standard SysV script has been installed to manage OpenTAXII.  The following functions are available.</p>
+<p><tt>start</tt> <tt>stop</tt> <tt>restart</tt> the OpenTAXII service</p>
+<p><tt>status</tt> of the OpenTAXII service.  The command displays the collections that have been defined and the number of records in each.</p>
+
+<div>
+<div>
+<pre class="source">$ service opentaxii status
+Checking opentaxii...                             Running
+guest.phishtank_com                                984
+guest.Abuse_ch                                     45
+guest.CyberCrime_Tracker                           482
+guest.EmergingThreats_rules                        0
+guest.Lehigh_edu                                   1030
+guest.MalwareDomainList_Hostlist                   84
+guest.blutmagie_de_torExits                        3236
+guest.dataForLast_7daysOnly                        3377
+guest.dshield_BlockList                            0
+</pre></div></div>
+
+<p><tt>setup</tt> Initializes the services and collections required to operate the OpenTAXII service.  This will destroy all existing data.  The user is prompted to continue before any data is destroyed.</p>
+
+<div>
+<div>
+<pre class="source"># service opentaxii setup
+WARNING: force reset and destroy all opentaxii data? [Ny]: y
+Stopping opentaxii                                ..Ok
+2016-04-21T19:56:01.886157Z [opentaxii.server] info: api.persistence.loaded {timestamp=2016-04-21T19:56:01.886157Z, logger=opentaxii.server, api_class=SQLDatabaseAPI, event=api.persistence.loaded, level=info}
+2016-04-21T19:56:01.896503Z [opentaxii.server] info: api.auth.loaded {timestamp=2016-04-21T19:56:01.896503Z, logger=opentaxii.server, api_class=SQLDatabaseAPI, event=api.auth.loaded, level=info}
+2016-04-21T19:56:01.896655Z [opentaxii.server] info: taxiiserver.configured {timestamp=2016-04-21T19:56:01.896655Z, logger=opentaxii.server, event=taxiiserver.configured, level=info}
+...
+Ok
+</pre></div></div>
+
+<p><tt>sync [collection] [begin-at] [end-at]</tt> Syncs the threat intel data available at <a class="externalLink" href="http://hailataxii.com/">Hail a TAXII</a>.  If no begin and end date is provided then data is synced over the current day only.</p>
+<ul>
+
+<li><tt>collection</tt> Name of the collection to sync.</li>
+<li><tt>begin-at</tt> Exclusive begin of time window; ISO8601</li>
+<li><tt>end-at</tt> Inclusive end of time window; ISO8601</li>
+</ul>
+
+<div>
+<div>
+<pre class="source">$ service opentaxii sync guest.phishtank_com
++ /usr/local/opentaxii/opentaxii-venv/bin/taxii-proxy --poll-path http://hailataxii.com/taxii-data --poll-collection guest.phishtank_com --inbox-path http://localhost:9000/services/guest.phishtank_com-inbox --inbox-collection guest.phishtank_com --binding urn:stix.mitre.org:xml:1.1.1 --begin 2016-04-21 --end 2016-04-22
+2016-04-21 17:36:23,778 INFO: Sending Poll_Request to http://hailataxii.com/taxii-data
+2016-04-21 17:36:23,784 INFO: Starting new HTTP connection (1): hailataxii.com
+2016-04-21 17:36:24,175 INFO: Response received for Poll_Request from http://hailataxii.com/taxii-data
+2016-04-21 17:36:24,274 INFO: Sending Inbox_Message to http://localhost:9000/services/guest.phishtank_com-inbox
+...
+2016-04-21 17:36:34,867 INFO: Response received for Poll_Request from http://localhost:9000/services/guest.phishtank_com-poll
+2016-04-21 17:36:34,868 INFO: Content blocks count: 6993, is partial: False
+</pre></div></div>
+
+<div class="section">
+<h3><a name="Troubleshooting"></a>Troubleshooting</h3>
+<p>Should you need to explore the installation, here are instructions on doing so.</p>
+<p>OpenTAXII is installed in a virtual environment.  Before exploring the environment run the following commands to perform the necessary setup.  The specific paths may change depending on your Ansible settings.</p>
+
+<div>
+<div>
+<pre class="source">export LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64
+export OPENTAXII_CONFIG=/usr/local/opentaxii/etc/opentaxii-conf.yml
+cd /usr/local/opentaxii
+. opentaxii-venv/bin/activate
+</pre></div></div>
+
+<p>Discover available services.</p>
+
+<div>
+<div>
+<pre class="source">taxii-discovery --discovery http://localhost:9000/services/discovery
+taxii-discovery --discovery http://hailataxii.com/taxii-data
+</pre></div></div>
+
+<p>Explore available collections.</p>
+
+<div>
+<div>
+<pre class="source">taxii-collections --discovery http://localhost:9000/services/discovery
+taxii-collections --discovery http://hailataxii.com/taxii-data
+</pre></div></div>
+
+<p>Read data from a collection.</p>
+
+<div>
+<div>
+<pre class="source">taxii-poll --discovery http://localhost:9000/services/discovery -c guest.phishtank_com
+taxii-poll --discovery http://hailataxii.com/taxii-data -c guest.phishtank_com --begin 2016-04-20
+</pre></div></div>
+
+<p>Manually load data into a collection.</p>
+
+<div>
+<div>
+<pre class="source">taxii-push \
+  --discovery http://localhost:9000/services/discovery \
+  --dest phishtank \
+  --content-file data.xml \
+  --username guest \
+  --password guest
+</pre></div></div>
+
+<p>Fetch data from a remote service and mirror it locally.</p>
+
+<div>
+<div>
+<pre class="source">taxii-proxy --poll-path http://hailataxii.com/taxii-data \
+            --poll-collection guest.phishtank_com \
+            --inbox-path http://localhost:9000/services/guest.phishtank_com-inbox \
+            --inbox-collection guest.phishtank_com \
+            --binding urn:stix.mitre.org:xml:1.1.1 \
+            --inbox-username guest \
+            --inbox-password guest \
+            --begin 2016-04-20
+</pre></div></div></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/ansible/roles/pcap_replay/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/ansible/roles/pcap_replay/index.html b/site/current-book/metron-deployment/ansible/roles/pcap_replay/index.html
new file mode 100644
index 0000000..d97a97d
--- /dev/null
+++ b/site/current-book/metron-deployment/ansible/roles/pcap_replay/index.html
@@ -0,0 +1,172 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/ansible/roles/pcap_replay/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Pcap Replay</title>
+    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../../css/site.css" />
+    <link rel="stylesheet" href="../../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Pcap Replay</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-down"></span>Ansible</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/ansible/roles/index.html" title="Roles"><span class="icon-chevron-down"></span>Roles</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/ansible/roles/opentaxii/index.html" title="Opentaxii"><span class="none"></span>Opentaxii</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Pcap_replay</a></li>
+    <li><a href="../../../../metron-deployment/ansible/roles/sensor-stubs/index.html" title="Sensor-stubs"><span class="none"></span>Sensor-stubs</a></li>
+    <li><a href="../../../../metron-deployment/ansible/roles/sensor-test-mode/index.html" title="Sensor-test-mode"><span class="none"></span>Sensor-test-mode</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Pcap Replay</h1>
+<p>This project enables packet capture data to be replayed through a network interface to simulate live network traffic.  This can be used to support functional, performance, and load testing of Apache Metron.</p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>To replay packet capture data, simply start the <tt>pcap-replay</tt> SysV service.  To do this run the following command.</p>
+
+<div>
+<div>
+<pre class="source">service pcap-replay start
+</pre></div></div>
+
+<p>All additional options accepted by <tt>tcpreplay</tt> can be passed to the service script to modify how the network data is replayed.  For example, this makes it simple to control the amount and rate of data replayed during functional, performance and load testing.</p>
+<p>Example: Replay data at a rate of 10 mbps.</p>
+
+<div>
+<div>
+<pre class="source">service pcap-replay start --mbps 10
+</pre></div></div>
+
+<p>Example: Replay data at a rate of 10 packets per second.</p>
+
+<div>
+<div>
+<pre class="source">service pcap-replay start --pps 10
+</pre></div></div>
+
+<p>All nodes on the same subnet with their network interface set to promiscuous mode will then be able to capture the network traffic being replayed.  To validate, simply run something like the following.</p>
+
+<div>
+<div>
+<pre class="source">tcpdump -i eth1
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Data"></a>Data</h2>
+<p>An example packet capture file has been installed at <tt>/opt/pcap-replay/example.pcap</tt>.  By default, the network traffic contained within this file is continually replayed.</p>
+<p>To replay your own packet capture data, simply add any number of files containing <tt>libpcap</tt> formatted packet capture data to <tt>/opt/pcap-replay</tt>.  The files must end with the <tt>.pcap</tt> extension.  To pick up newly installed files, simply restart the service.</p>
+
+<div>
+<div>
+<pre class="source">service pcap-replay restart
+</pre></div></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/ansible/roles/sensor-stubs/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/ansible/roles/sensor-stubs/index.html b/site/current-book/metron-deployment/ansible/roles/sensor-stubs/index.html
new file mode 100644
index 0000000..650fc89
--- /dev/null
+++ b/site/current-book/metron-deployment/ansible/roles/sensor-stubs/index.html
@@ -0,0 +1,221 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/ansible/roles/sensor-stubs/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Sensor Stubs</title>
+    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../../css/site.css" />
+    <link rel="stylesheet" href="../../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Sensor Stubs</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-down"></span>Ansible</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/ansible/roles/index.html" title="Roles"><span class="icon-chevron-down"></span>Roles</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/ansible/roles/opentaxii/index.html" title="Opentaxii"><span class="none"></span>Opentaxii</a></li>
+    <li><a href="../../../../metron-deployment/ansible/roles/pcap_replay/index.html" title="Pcap_replay"><span class="none"></span>Pcap_replay</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Sensor-stubs</a></li>
+    <li><a href="../../../../metron-deployment/ansible/roles/sensor-test-mode/index.html" title="Sensor-test-mode"><span class="none"></span>Sensor-test-mode</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Sensor Stubs</h1>
+<p><a name="Sensor_Stubs"></a></p>
+<p>A service has been created to simulate the behavior of a sensor by sending canned telemetry data to a Kafka topic.  These &#x201c;Sensor Stubs&#x201d; consume fewer resources than the actual sensor that they replace.</p>
+<div class="section">
+<div class="section">
+<h3><a name="a.28Q.29_How_do_the_sensor_stubs_work.3F"></a>(Q) How do the sensor stubs work?</h3>
+<p>The stubs are installed with a set of canned data for each sensor type; Bro, Snort and YAF.  A subset of this canned data is randomly selected and sent to the Kafka topic in batches.  The timestamp of each message is updated to match current system time.</p></div>
+<div class="section">
+<h3><a name="a.28Q.29_How_do_I_configure_the_message_rate.3F"></a>(Q) How do I configure the message rate?</h3>
+<p>The number of telemetry messages sent in each batch, along with the time delay between batches is configurable.  Before installation, these values can be configured by redefining <tt>sensor_stubs_delay</tt> and <tt>sensor_stubs_count</tt>.  The values can also be configured by altering the deployed system service script at <tt>/etc/init.d/sensor-stubs</tt>.</p></div>
+<div class="section">
+<h3><a name="a.28Q.29_How_do_I_install_the_sensor_stubs.3F"></a>(Q) How do I install the sensor stubs?</h3>
+<p>Using the default playbooks, this role can be installed by using the Ansible tag <tt>sensor-stubs</tt>.  This service is installed on the same hosts where the sensors would be; defined by the <tt>sensors</tt> host group.</p>
+<p>The defaults for the &#x201c;Full Dev&#x201d; environment have been changed so that the Sensor Stubs are installed by default, rather than the sensors themselves.  The Amazon EC2 environment continues to install the original sensors by default.</p></div>
+<div class="section">
+<h3><a name="a.28Q.29_How_do_I_use_the_sensor_stubs.3F"></a>(Q) How do I use the sensor stubs?</h3>
+<p>Start all sensor stubs.  The output includes the PID for each running sensor stub.</p>
+
+<div>
+<div>
+<pre class="source">$ service sensor-stubs start
+Starting sensor-stubs...
+       bro: Ok [26505]
+       yaf: Ok [26507]
+     snort: Ok [26509]
+</pre></div></div>
+
+<p>Check the status of each sensor stub.</p>
+
+<div>
+<div>
+<pre class="source">$ service sensor-stubs status
+Checking sensor-stubs...
+       bro: Running [26505]
+       yaf: Running [26507]
+     snort: Running [26509]
+</pre></div></div>
+
+<p>Stop all sensor stubs.</p>
+
+<div>
+<div>
+<pre class="source">$ service sensor-stubs stop
+Stopping sensor-stubs...
+..       bro: Ok [26505]
+..       yaf: Ok [26507]
+..     snort: Ok [26509]
+</pre></div></div>
+
+<p>Check the status.  All sensor stubs should be stopped.</p>
+
+<div>
+<div>
+<pre class="source">$ service sensor-stubs status
+Checking sensor-stubs...
+       bro: Not running
+       yaf: Not running
+     snort: Not running
+</pre></div></div>
+
+<p>Start only the Bro sensor stub.</p>
+
+<div>
+<div>
+<pre class="source">$ service sensor-stubs start bro
+Starting sensor-stubs...
+       bro: OK [11616]
+</pre></div></div>
+
+<p>Stop the Bro sensor stub.</p>
+
+<div>
+<div>
+<pre class="source">$ service sensor-stubs stop bro
+Stopping sensor-stubs...
+..       bro: Ok [11616]
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="a.28Q.29_How_do_I_install_the_original_sensors.3F"></a>(Q) How do I install the original sensors?</h3>
+<p>The default behavior can be changed by skipping the <tt>sensor-stubs</tt> flag and including the <tt>sensors</tt> flag. For example, to deploy &#x201c;Full Dev&#x201d; with the original sensors run the following command.</p>
+
+<div>
+<div>
+<pre class="source">cd metron-deployment/development/centos6
+vagrant --ansible-skip-tags=&quot;sensor-stubs,solr&quot; up
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="a.28Q.29_Where_does_the_mock_data_come_from.3F"></a>(Q) Where does the mock data come from?</h3>
+<p>The data produced by the sensor stubs was generated by running the sensors against the example pcap file that is distributed with Metron.  This ensures that the data produced by the sensor stubs is similar to the data produced when using the actual sensors.</p></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/ansible/roles/sensor-test-mode/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/ansible/roles/sensor-test-mode/index.html b/site/current-book/metron-deployment/ansible/roles/sensor-test-mode/index.html
new file mode 100644
index 0000000..b989235
--- /dev/null
+++ b/site/current-book/metron-deployment/ansible/roles/sensor-test-mode/index.html
@@ -0,0 +1,156 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/ansible/roles/sensor-test-mode/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Sensor Test Mode</title>
+    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../../css/site.css" />
+    <link rel="stylesheet" href="../../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Sensor Test Mode</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-down"></span>Ansible</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/ansible/roles/index.html" title="Roles"><span class="icon-chevron-down"></span>Roles</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/ansible/roles/opentaxii/index.html" title="Opentaxii"><span class="none"></span>Opentaxii</a></li>
+    <li><a href="../../../../metron-deployment/ansible/roles/pcap_replay/index.html" title="Pcap_replay"><span class="none"></span>Pcap_replay</a></li>
+    <li><a href="../../../../metron-deployment/ansible/roles/sensor-stubs/index.html" title="Sensor-stubs"><span class="none"></span>Sensor-stubs</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Sensor-test-mode</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Sensor Test Mode</h1>
+<p>A role that configures each of the sensors to produce the maximum amount of telemetry data.  This role is useful only for testing.  It can be useful to support functional, performance, and load testing of Apache Metron.</p>
+<p>The role does the following to maximize the amount of telemetry data produced by each Metron sensor.</p>
+<ul>
+
+<li>Plays a packet capture file through a network interface to simulate live network traffic.</li>
+<li>Configures <a class="externalLink" href="https://tools.netsa.cert.org/yaf/yaf.html">YAF</a> with <tt>idle-timeout=0</tt>.  This causes a flow record to be produced for every network packet received.</li>
+<li>Configures <a class="externalLink" href="https://www.snort.org/">Snort</a> to produce an alert for every network packet received.</li>
+</ul>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>To enable the <tt>sensor-test-mode</tt> role apply the role to the <tt>sensors</tt> host group in your Ansible playbook.</p>
+
+<div>
+<div>
+<pre class="source">- hosts: sensors
+  roles:
+    - role: sensor-test-mode
+</pre></div></div>
+
+<p>The role has also been added to the default <tt>metron_install.yml</tt> playbook so that it can be turned on/off with a property in both the local Virtualbox and the remote EC2 deployments.</p>
+
+<div>
+<div>
+<pre class="source">sensor_test_mode: True
+</pre></div></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/development/centos6/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/development/centos6/index.html b/site/current-book/metron-deployment/development/centos6/index.html
new file mode 100644
index 0000000..b036e37
--- /dev/null
+++ b/site/current-book/metron-deployment/development/centos6/index.html
@@ -0,0 +1,233 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/development/centos6/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Metron on CentOS 6</title>
+    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../css/site.css" />
+    <link rel="stylesheet" href="../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron on CentOS 6</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-down"></span>Development</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>Centos6</a></li>
+    <li><a href="../../../metron-deployment/development/fastcapa/index.html" title="Fastcapa"><span class="none"></span>Fastcapa</a></li>
+    <li><a href="../../../metron-deployment/development/ubuntu14/index.html" title="Ubuntu14"><span class="none"></span>Ubuntu14</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron on CentOS 6</h1>
+<p>This project fully automates the provisioning and deployment of Apache Metron and all necessary prerequisites on a single, virtualized host running CentOS 6.</p>
+<p>Metron is composed of many components and installing all of these on a single host, especially a virtualized one, will greatly stress the resources of the host.   The host will require at least 8 GB of RAM and a fair amount of patience.  It is highly recommended that you shut down all unnecessary services.</p>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<p>The computer used to deploy Apache Metron will need to have the following components installed.</p>
+<ul>
+
+<li><a class="externalLink" href="https://github.com/ansible/ansible">Ansible</a> (2.0.0.2, 2.2.2.0, or 2.5.0)</li>
+<li><a class="externalLink" href="https://www.docker.com/community-edition">Docker</a></li>
+<li><a class="externalLink" href="https://www.vagrantup.com">Vagrant</a> 2.0+</li>
+<li><a class="externalLink" href="https://github.com/devopsgroup-io/vagrant-hostmanager">Vagrant Hostmanager Plugin</a></li>
+<li><a class="externalLink" href="https://virtualbox.org">Virtualbox</a> 5.0+</li>
+<li>Python 2.7</li>
+<li>Maven 3.3.9</li>
+<li>C++11 compliant compiler, like <a class="externalLink" href="https://gcc.gnu.org/projects/cxx-status.html#cxx11">GCC</a></li>
+</ul>
+<p>Running the following script can help validate whether you have all the prerequisites installed and running correctly.</p>
+
+<div>
+<div>
+<pre class="source">  metron-deployment/scripts/platform-info.sh
+</pre></div></div>
+
+<div class="section">
+<h4><a name="How_do_I_install_these_on_MacOS.3F"></a>How do I install these on MacOS?</h4>
+<p>Any platform that supports these tools is suitable, but the following instructions cover installation on macOS.  The easiest means of installing these tools on a Mac is to use the excellent <a class="externalLink" href="http://brew.sh/">Homebrew</a> project.</p>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Install Homebrew by following the instructions at <a class="externalLink" href="http://brew.sh/">Homebrew</a>.</p>
+</li>
+<li>
+
+<p>Run the following command in a terminal to install all of the required tools.</p>
+
+<div>
+<div>
+<pre class="source">brew cask install vagrant virtualbox docker
+brew cask install caskroom/versions/java8
+brew install maven@3.3 git
+pip install ansible==2.2.2.0
+vagrant plugin install vagrant-hostmanager
+open /Applications/Docker.app
+</pre></div></div>
+</li>
+</ol></div></div>
+<div class="section">
+<h3><a name="Deploy_Metron"></a>Deploy Metron</h3>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Ensure that the Docker service is running.</p>
+</li>
+<li>
+
+<p>Deploy Metron</p>
+
+<div>
+<div>
+<pre class="source">cd metron-deployment/development/centos6
+vagrant up
+</pre></div></div>
+
+<p>Should the process fail before completing the deployment, the following command will continue the deployment process without re-instantiating the host.</p>
+
+<div>
+<div>
+<pre class="source">vagrant provision
+</pre></div></div>
+</li>
+</ol></div>
+<div class="section">
+<h3><a name="Explore_Metron"></a>Explore Metron</h3>
+<p>Navigate to the following resources to explore your newly minted Apache Metron environment.</p>
+<ul>
+
+<li><a class="externalLink" href="http://node1:4201">Metron Alerts</a></li>
+<li><a class="externalLink" href="http://node1:8080">Ambari</a></li>
+</ul>
+<p>Connecting to the host through SSH is as simple as running the following command.</p>
+
+<div>
+<div>
+<pre class="source">vagrant ssh
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="Working_with_Metron"></a>Working with Metron</h3>
+<p>In addition to re-running the entire provisioning play book, you may now re-run an individual Ansible tag or a collection of tags in the following ways.  The following commands will re-run the <tt>sensor-stubs</tt> role on the Vagrant image. This will install and start the sensor stub components.</p>
+
+<div>
+<div>
+<pre class="source">vagrant --ansible-tags=&quot;sensor-stubs&quot; provision
+</pre></div></div>
+
+<p>Tags are listed in the playbooks, some frequently used tags:</p>
+<ul>
+
+<li><tt>hdp-install</tt> - Install HDP</li>
+<li><tt>hdp-deploy</tt> - Deploy and Start HDP Services (will start all Hadoop Services)</li>
+<li><tt>sensors</tt> - Deploy and start the sensors.</li>
+<li><tt>sensor-stubs</tt> - Deploy and start the sensor stubs.</li>
+</ul></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/development/fastcapa/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/development/fastcapa/index.html b/site/current-book/metron-deployment/development/fastcapa/index.html
new file mode 100644
index 0000000..d471576
--- /dev/null
+++ b/site/current-book/metron-deployment/development/fastcapa/index.html
@@ -0,0 +1,249 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/development/fastcapa/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Fastcapa Test Environment</title>
+    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../css/site.css" />
+    <link rel="stylesheet" href="../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Fastcapa Test Environment</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-down"></span>Development</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/development/centos6/index.html" title="Centos6"><span class="none"></span>Centos6</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Fastcapa</a></li>
+    <li><a href="../../../metron-deployment/development/ubuntu14/index.html" title="Ubuntu14"><span class="none"></span>Ubuntu14</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Fastcapa Test Environment</h1>
+<p>Provides a test environment for the development and testing of Fastcapa.  The environment is automatically validated after it is created to ensure that Fastcapa is behaving correctly.</p>
+<p>Two virtualized nodes are launched with Vagrant that can communicate with one another over a private network.</p>
+<ul>
+
+<li>The <tt>source</tt> node uses Metron&#x2019;s <tt>pcap_replay</tt> functionality to transmit raw network packet data over a private network.</li>
+<li>The <tt>sink</tt> node is running <tt>fastcapa</tt> and is capturing these network packets.</li>
+<li>Fastcapa then transforms and bundles the packets into a message.</li>
+<li>The message is sent to a Kafka broker running on the <tt>source</tt> node.</li>
+</ul>
+<div class="section">
+<h2><a name="Getting_Started"></a>Getting Started</h2>
+<p>The Fastcapa test environment can be executed on different operating systems.  There is a sub-directory for each operating system that Fastcapa can be tested on.</p>
+<p>To run, simply execute <tt>vagrant up</tt> within the appropriate directory.  For example, to run the tests on CentOS 7.1 then execute the following commands.</p>
+
+<div>
+<div>
+<pre class="source">cd centos-7.1
+vagrant up
+</pre></div></div>
+
+<p>Automated tests are executed after provisioning completes to ensure that Fastcapa and the rest of the environment is functioning properly.  If you see something like the following, then the tests have passed.</p>
+
+<div>
+<div>
+<pre class="source">$ vagrant up
+==&gt; source: Running provisioner: ansible...
+    source: Running ansible-playbook...
+...
+TASK [debug] *******************************************************************
+ok: [source] =&gt; {
+    &quot;msg&quot;: &quot;Successfully received packets sent from pcap-replay!&quot;
+}
+...
+TASK [debug] *******************************************************************
+ok: [source] =&gt; {
+    &quot;msg&quot;: &quot;Successfully received a Kafka message from fastcapa!&quot;
+}
+</pre></div></div>
+
+<p>If the deployment process fails mid-course, running <tt>vagrant provision</tt> will continue the process from where it left off.  This can sometimes occur when the VM reboots as part of the deployment process.  The error might look like the following.</p>
+
+<div>
+<div>
+<pre class="source">TASK [fastcapa : Restart for modified kernel params] ***************************
+fatal: [sink]: UNREACHABLE! =&gt; {&quot;changed&quot;: false, &quot;msg&quot;: &quot;Failed to connect to the host via ssh: Shared connection to 127.0.0.1 closed.\r\n&quot;, &quot;unreachable&quot;: true}
+	to retry, use: --limit @/Users/nallen/Development/metron/metron-deployment/vagrant/fastcapa-test-platform/playbook.retry
+
+PLAY RECAP *********************************************************************
+sink                       : ok=11   changed=9    unreachable=1    failed=0
+source                     : ok=29   changed=25   unreachable=0    failed=0
+
+Ansible failed to complete successfully. Any error output should be
+visible above. Please fix these errors and try again.
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Going_Deeper"></a>Going Deeper</h2>
+<p>This section will outline in more detail the environment and how to interact with it.</p>
+<div class="section">
+<h3><a name="source"></a><tt>source</tt></h3>
+<p>To validate that the <tt>source</tt> node is functioning properly, run the following commands.</p>
+<p>First, ensure that the <tt>pcap-replay</tt> service is running.</p>
+
+<div>
+<div>
+<pre class="source">vagrant ssh source
+sudo service pcap-replay status
+</pre></div></div>
+
+<p>Use <tt>tcpdump</tt> to ensure that the raw packet data is being sent over the private network.  Enter &#x2018;CTRL-C&#x2019; to kill the <tt>tcpdump</tt> process once you are able to see that packets are being sent.</p>
+
+<div>
+<div>
+<pre class="source">sudo yum -y install tcpdump
+sudo tcpdump -i enp0s8
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="sink"></a><tt>sink</tt></h3>
+<p>Next validate that the <tt>sink</tt> is functioning properly. Run the following commands starting from the host operating system.</p>
+<p>First, ensure that the <tt>fastcapa</tt> service is running.</p>
+
+<div>
+<div>
+<pre class="source">vagrant ssh sink
+service fastcapa status
+</pre></div></div>
+
+<p>Ensure that the raw network packet data is being received by Kafka.</p>
+
+<div>
+<div>
+<pre class="source">/usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic pcap
+</pre></div></div>
+
+<p>Enter &#x2018;CTRL-C&#x2019; to kill the <tt>kafka-console-consumer</tt> process once you are able to see that packets are being sent.  These packets will appear to be gibberish in the console.  This is the raw binary network packet data after all.</p></div></div>
+<div class="section">
+<h2><a name="FAQ"></a>FAQ</h2>
+<div class="section">
+<h3><a name="Error_Message:_Timed_out_while_waiting_for_the_machine_to_boot"></a>Error Message: Timed out while waiting for the machine to boot</h3>
+
+<div>
+<div>
+<pre class="source">Timed out while waiting for the machine to boot. This means that
+Vagrant was unable to communicate with the guest machine within
+the configured (&quot;config.vm.boot_timeout&quot; value) time period.
+If you look above, you should be able to see the error(s) that
+Vagrant had when attempting to connect to the machine. These errors
+are usually good hints as to what may be wrong.
+If you're using a custom box, make sure that networking is properly
+working and you're able to connect to the machine. It is a common
+problem that networking isn't setup properly in these boxes.
+Verify that authentication configurations are also setup properly,
+as well.
+If the box appears to be booting properly, you may want to increase
+the timeout (&quot;config.vm.boot_timeout&quot;) value.
+&#x279c;  centos-7.4 git:(master) &#x2717; vagrant status
+Current machine states:
+source                    running (virtualbox)
+sink                      not created (virtualbox)
+</pre></div></div>
+
+<p>If you are unable to launch any of the Fastcapa test environments, which results in a message like the one above, then you may need to upgrade your version of Virtualbox.  Success has been reported with versions of VirtualBox 5.1.22+.</p></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>


[48/50] [abbrv] metron git commit: METRON-1533 Create KAFKA_FIND Stellar Function (nickwallen) closes apache/metron#1025

Posted by mm...@apache.org.
METRON-1533 Create KAFKA_FIND Stellar Function (nickwallen) closes apache/metron#1025


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/8202cd24
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/8202cd24
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/8202cd24

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: 8202cd2425b699afb5c6fb11cbb59b0fcbf4f82d
Parents: edec7b1
Author: nickwallen <ni...@nickallen.org>
Authored: Fri Jun 8 11:11:57 2018 -0400
Committer: nickallen <ni...@apache.org>
Committed: Fri Jun 8 11:11:57 2018 -0400

----------------------------------------------------------------------
 .../metron/management/KafkaFunctions.java       | 204 +++++++++++++++++--
 .../KafkaFunctionsIntegrationTest.java          |  98 +++++++++
 2 files changed, 289 insertions(+), 13 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/8202cd24/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java b/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
index a0c92eb..f256672 100644
--- a/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
+++ b/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
@@ -18,7 +18,9 @@
 
 package org.apache.metron.management;
 
+import org.apache.commons.lang3.ClassUtils;
 import org.apache.kafka.clients.consumer.ConsumerRecord;
+import org.apache.kafka.clients.consumer.ConsumerRecords;
 import org.apache.kafka.clients.consumer.KafkaConsumer;
 import org.apache.kafka.clients.producer.KafkaProducer;
 import org.apache.kafka.clients.producer.ProducerRecord;
@@ -28,7 +30,10 @@ import org.apache.kafka.common.TopicPartition;
 import org.apache.kafka.common.serialization.StringDeserializer;
 import org.apache.kafka.common.serialization.StringSerializer;
 import org.apache.metron.common.system.Clock;
+import org.apache.metron.profiler.client.stellar.Util;
+import org.apache.metron.stellar.common.LambdaExpression;
 import org.apache.metron.stellar.common.utils.ConversionUtils;
+import org.apache.metron.stellar.common.utils.JSONUtils;
 import org.apache.metron.stellar.dsl.Context;
 import org.apache.metron.stellar.dsl.ParseException;
 import org.apache.metron.stellar.dsl.Stellar;
@@ -36,6 +41,7 @@ import org.apache.metron.stellar.dsl.StellarFunction;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import java.io.IOException;
 import java.lang.invoke.MethodHandles;
 import java.util.ArrayList;
 import java.util.Collections;
@@ -150,18 +156,18 @@ public class KafkaFunctions {
     public Object apply(List<Object> args, Context context) throws ParseException {
 
       // required - name of the topic to retrieve messages from
-      String topic = ConversionUtils.convert(args.get(0), String.class);
+      String topic = getArg("topic", 0, String.class, args);
 
       // optional - how many messages should be retrieved?
       int count = 1;
       if(args.size() > 1) {
-        count = ConversionUtils.convert(args.get(1), Integer.class);
+        count = getArg("count", 1, Integer.class, args);
       }
 
       // optional - property overrides provided by the user
       Map<String, String> overrides = new HashMap<>();
       if(args.size() > 2) {
-        overrides = ConversionUtils.convert(args.get(2), Map.class);
+        overrides = getArg("overrides", 2, Map.class, args);
       }
 
       // build the properties for kafka
@@ -259,18 +265,18 @@ public class KafkaFunctions {
     public Object apply(List<Object> args, Context context) throws ParseException {
 
       // required - name of the topic to retrieve messages from
-      String topic = ConversionUtils.convert(args.get(0), String.class);
+      String topic = getArg("topic", 0, String.class, args);
 
       // optional - how many messages should be retrieved?
       int count = 1;
       if(args.size() > 1) {
-        count = ConversionUtils.convert(args.get(1), Integer.class);
+        count = getArg("count", 1, Integer.class, args);
       }
 
       // optional - property overrides provided by the user
       Map<String, String> overrides = new HashMap<>();
       if(args.size() > 2) {
-        overrides = ConversionUtils.convert(args.get(2), Map.class);
+        overrides = getArg("overrides", 2, Map.class, args);
       }
 
       Properties properties = buildKafkaProperties(overrides, context);
@@ -372,18 +378,18 @@ public class KafkaFunctions {
       List<String> messages;
       if(args.get(1) instanceof String) {
         // a single message needs sent
-        String msg = ConversionUtils.convert(args.get(1), String.class);
+        String msg = getArg("message(s)", 1, String.class, args);
         messages = Collections.singletonList(msg);
 
       } else {
         // a list of messages; all need sent
-        messages = ConversionUtils.convert(args.get(1), List.class);
+        messages = getArg("message(s)", 1, List.class, args);
       }
 
       // are there any overrides?
       Map<String, String> overrides = new HashMap<>();
       if(args.size() > 2) {
-        overrides = ConversionUtils.convert(args.get(2), Map.class);
+        overrides = getArg("overrides", 2, Map.class, args);
       }
 
       // send the messages
@@ -485,7 +491,7 @@ public class KafkaFunctions {
       // optional - did the user provide any overrides?
       Map<String, String> overrides = new HashMap<>();
       if(args.size() > 0) {
-        overrides = ConversionUtils.convert(args.get(0), Map.class);
+        overrides = getArg("overrides", 0, Map.class, args);
       }
 
       return buildKafkaProperties(overrides, context);
@@ -504,6 +510,163 @@ public class KafkaFunctions {
   }
 
   /**
+   * KAFKA_FIND
+   *
+   * <p>Finds messages that satisfy a given filter expression. Subsequent calls will continue retrieving messages
+   * sequentially from the original offset.
+   *
+   * <p>Example: Retrieve a 'bro' message.
+   * <pre>
+   * {@code
+   * KAFKA_FIND('topic', m -> MAP_GET('source.type', m) == 'bro')
+   * }
+   * </pre>
+   *
+   * <p>Example: Find 10 messages that contain geo-location data.
+   * <pre>
+   * {@code
+   * KAFKA_FIND('topic', m -> MAP_EXISTS('geo', m), 10)
+   * }
+   * </pre>
+   */
+  @Stellar(
+          namespace = "KAFKA",
+          name = "FIND",
+          description = "Find messages that satisfy a given filter expression. Messages are filtered starting from " +
+                  "the latest offset.",
+          params = {
+                  "topic - The name of the Kafka topic",
+                  "filter - A lambda expression that filters messages. Messages are presented as a map of fields to the expression.",
+                  "count - The number of Kafka messages to retrieve",
+                  "config - Optional map of key/values that override any global properties."
+          },
+          returns = "The messages as a list of strings"
+  )
+  public static class KafkaFind implements StellarFunction {
+
+    @Override
+    public Object apply(List<Object> args, Context context) throws ParseException {
+      // required - name of the topic to retrieve messages from
+      String topic = getArg("topic", 0, String.class, args);
+
+      // required - a lambda which filters the messages
+      LambdaExpression filter = getArg("filter", 1, LambdaExpression.class, args);
+
+      // optional - how many messages should be retrieved?
+      int count = 1;
+      if(args.size() > 2) {
+        count = getArg("count", 2, Integer.class, args);
+      }
+
+      // optional - property overrides provided by the user
+      Map<String, String> overrides = new HashMap<>();
+      if(args.size() > 3) {
+        overrides = getArg("overrides", 3, Map.class, args);
+      }
+
+      Properties properties = buildKafkaProperties(overrides, context);
+      properties.put("max.poll.records", 10 * count);
+
+      return findMessages(topic, filter, count, properties);
+    }
+
+    /**
+     * Find messages in Kafka that satisfy a filter expression.
+     *
+     * @param topic The kafka topic.
+     * @param filter The filter expression.
+     * @param count The maximum number of messages to find.
+     * @param properties Function configuration values.
+     * @return A list of messages that satisfy the filter expression.
+     */
+    private List<Object> findMessages(String topic, LambdaExpression filter, int count, Properties properties) {
+      final int pollTimeout = getPollTimeout(properties);
+      final int maxWait = getMaxWait(properties);
+
+      List<Object> messages = new ArrayList<>();
+      try (KafkaConsumer<String, String> consumer = new KafkaConsumer<>(properties)) {
+
+        // seek to the end of all topic/partitions
+        Set<TopicPartition> partitions = manualPartitionAssignment(topic, consumer);
+        consumer.seekToEnd(partitions);
+
+        // continue until we have enough messages or exceeded the max wait time
+        long wait = 0L;
+        final long start = clock.currentTimeMillis();
+        while(messages.size() < count && wait < maxWait) {
+
+          // poll kafka for messages
+          ConsumerRecords<String, String> records = consumer.poll(pollTimeout);
+          for(ConsumerRecord<String, String> record : records) {
+
+            // only keep the message if the filter expression is satisfied
+            if(isSatisfied(filter, record.value())) {
+              messages.add(record.value());
+
+              // do we have enough messages already?
+              if(messages.size() >= count) {
+                break;
+              }
+            }
+          }
+
+          // how long have we waited?
+          wait = clock.currentTimeMillis() - start;
+          consumer.commitSync();
+
+          LOG.debug("KAFKA_FIND polled for messages; topic={}, count={}, waitTime={} ms",
+                  topic, messages.size(), wait);
+        }
+      }
+
+      return messages;
+    }
+
+    /**
+     * Executes a given expression on a message.
+     *
+     * @param expr The filter expression to execute.
+     * @param message The message that the expression is executed on.
+     * @return Returns true, only if the expression returns true.  If the expression
+     * returns false or fails to execute, false is returned.
+     */
+    public boolean isSatisfied(LambdaExpression expr, String message) {
+      boolean result = false;
+      Map<String, Object> messageAsMap;
+      try {
+        // transform the message to a map of fields
+        messageAsMap = JSONUtils.INSTANCE.load(message, JSONUtils.MAP_SUPPLIER);
+
+        // apply the filter expression
+        Object out = expr.apply(Collections.singletonList(messageAsMap));
+        if(out instanceof Boolean) {
+          result = (Boolean) out;
+
+        } else {
+          LOG.error("Expected boolean from filter expression, got {}", ClassUtils.getShortClassName(out, "null"));
+        }
+
+      } catch(IOException e) {
+        LOG.error("Unable to parse message", e);
+      }
+
+      return result;
+    }
+
+
+    @Override
+    public void initialize(Context context) {
+      // no initialization required
+    }
+
+    @Override
+    public boolean isInitialized() {
+      // no initialization required
+      return true;
+    }
+  }
+
+  /**
    * Manually assigns all partitions in a topic to a consumer
    *
    * @param topic The topic whose partitions will be assigned.
@@ -511,7 +674,6 @@ public class KafkaFunctions {
    * @return A set of topic-partitions that were manually assigned to the consumer.
    */
   private static Set<TopicPartition> manualPartitionAssignment(String topic, KafkaConsumer<String, String> consumer) {
-
     // find all partitions for the topic
     Set<TopicPartition> partitions = new HashSet<>();
     for(PartitionInfo partition : consumer.partitionsFor(topic)) {
@@ -539,7 +701,6 @@ public class KafkaFunctions {
    * @param context The Stellar context.
    */
   private static Properties buildKafkaProperties(Map<String, String> overrides, Context context) {
-
     // start with minimal set of default properties
     Properties properties = new Properties();
     properties.putAll(defaultProperties);
@@ -599,7 +760,6 @@ public class KafkaFunctions {
    * via the global properties.
    */
   private static Properties defaultKafkaProperties() {
-
     Properties properties = new Properties();
     properties.put("bootstrap.servers", "localhost:9092");
     properties.put("group.id", "kafka-functions-stellar");
@@ -634,4 +794,22 @@ public class KafkaFunctions {
 
     return properties;
   }
+
+  /**
+   * Get an argument from a list of arguments.
+   *
+   * @param argName The name of the argument.
+   * @param index The index within the list of arguments.
+   * @param clazz The type expected.
+   * @param args All of the arguments.
+   * @param <T> The type of the argument expected.
+   */
+  public static <T> T getArg(String argName, int index, Class<T> clazz, List<Object> args) {
+    if(index >= args.size()) {
+      throw new IllegalArgumentException(format("missing '%s'; expected at least %d argument(s), found %d",
+              argName, index+1, args.size()));
+    }
+
+    return ConversionUtils.convert(args.get(index), clazz);
+  }
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/8202cd24/metron-platform/metron-management/src/test/java/org/apache/metron/management/KafkaFunctionsIntegrationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-management/src/test/java/org/apache/metron/management/KafkaFunctionsIntegrationTest.java b/metron-platform/metron-management/src/test/java/org/apache/metron/management/KafkaFunctionsIntegrationTest.java
index ad45b52..d82bb37 100644
--- a/metron-platform/metron-management/src/test/java/org/apache/metron/management/KafkaFunctionsIntegrationTest.java
+++ b/metron-platform/metron-management/src/test/java/org/apache/metron/management/KafkaFunctionsIntegrationTest.java
@@ -48,6 +48,7 @@ import java.util.concurrent.Future;
 import java.util.concurrent.TimeUnit;
 
 import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
 
 /**
  * Tests the KafkaFunctions class.
@@ -109,6 +110,7 @@ public class KafkaFunctionsIntegrationTest extends BaseIntegrationTest {
             .withClass(KafkaFunctions.KafkaPut.class)
             .withClass(KafkaFunctions.KafkaProps.class)
             .withClass(KafkaFunctions.KafkaTail.class)
+            .withClass(KafkaFunctions.KafkaFind.class)
             .withClass(MapFunctions.MapGet.class);
   }
 
@@ -339,6 +341,102 @@ public class KafkaFunctionsIntegrationTest extends BaseIntegrationTest {
   }
   
   /**
+   * KAFKA_FIND should only return messages that satisfy a filter expression.
+   */
+  @Test
+  public void testKafkaFind() throws Exception {
+
+    // use a unique topic name for this test
+    final String topicName = testName.getMethodName();
+    variables.put("topic", topicName);
+
+    // find all messages satisfying the filter expression
+    Future<Object> future = runAsync("KAFKA_FIND(topic, m -> MAP_GET('value', m) == 23)");
+
+    // put 10 messages onto the topic for KAFKA_TAIL to grab
+    runAsyncAndWait(Collections.nCopies(10, "KAFKA_PUT(topic, [message2])"));
+
+    // only expect `message2` where value == 23 to be returned
+    Object actual = future.get(10, TimeUnit.SECONDS);
+    List<String> expected = Collections.singletonList(message2);
+    assertEquals(expected, actual);
+  }
+
+  /**
+   * KAFKA_FIND should return no messages, if none match the filter expression.
+   */
+  @Test
+  public void testKafkaFindNone() throws Exception {
+
+    // use a unique topic name for this test
+    final String topicName = testName.getMethodName();
+    variables.put("topic", topicName);
+
+    // find all messages satisfying the filter expression
+    Future<Object> future = runAsync("KAFKA_FIND(topic, m -> false)");
+
+    // put 10 messages onto the topic for KAFKA_TAIL to grab
+    runAsyncAndWait(Collections.nCopies(10, "KAFKA_PUT(topic, [message1])"));
+
+    // no messages satisfy the filter expression
+    Object actual = future.get(10, TimeUnit.SECONDS);
+    List<String> expected = Collections.emptyList();
+    assertEquals(expected, actual);
+  }
+
+  /**
+   * KAFKA_FIND should return no more messages than its limit.
+   */
+  @Test
+  public void testKafkaFindMultiple() throws Exception {
+
+    // use a unique topic name for this test
+    final String topicName = testName.getMethodName();
+    variables.put("topic", topicName);
+
+    // find all messages satisfying the filter expression
+    Future<Object> future = runAsync("KAFKA_FIND(topic, m -> true, 2)");
+
+    // put 10 messages onto the topic for KAFKA_TAIL to grab
+    runAsyncAndWait(Collections.nCopies(10, "KAFKA_PUT(topic, [message2])"));
+
+    // all messages should satisfy the filter
+    List<String> expected = new ArrayList<String>() {{
+      add(message2);
+      add(message2);
+    }};
+    Object actual = future.get(10, TimeUnit.SECONDS);
+    assertEquals(expected, actual);
+  }
+
+  /**
+   * KAFKA_FIND should wait no more than a maximum time before returning, even if no matching
+   * messages are found.
+   */
+  @Test
+  public void testKafkaFindExceedsMaxWait() {
+
+    // use a unique topic name for this test
+    final String topicName = testName.getMethodName();
+    variables.put("topic", topicName);
+
+    // write all 3 messages to the topic
+    run("KAFKA_PUT(topic, [message1, message2, message3])");
+
+    // execute the test - none of the messages satisfy the filter
+    long before = System.currentTimeMillis();
+    Object actual = run("KAFKA_FIND(topic, m -> false, 10, { 'stellar.kafka.max.wait.millis': 1000 })");
+
+    // expect not to have waited more than roughly 1000 millis
+    long wait = System.currentTimeMillis() - before;
+    assertTrue("Expected wait not to exceed max wait; actual wait = " + wait, wait < 2 * 1000);
+
+    // expect no messages
+    List<String> expected = Collections.emptyList();
+    assertEquals(expected, actual);
+  }
+
+  /**
    * Runs a Stellar expression.
    * @param expression The expression to run.
    */


[05/50] [abbrv] metron git commit: METRON-1594: KafkaWriter is asynchronous and may lose data on node failure (mmiklavc via mmiklavc) closes apache/metron#1045

Posted by mm...@apache.org.
METRON-1594: KafkaWriter is asynchronous and may lose data on node failure (mmiklavc via mmiklavc) closes apache/metron#1045


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/523c38cf
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/523c38cf
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/523c38cf

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: 523c38cf6399e2e3974a51a2cd0fe47e096b0bdf
Parents: b6808f7
Author: mmiklavc <mi...@gmail.com>
Authored: Wed Jun 6 15:40:55 2018 -0600
Committer: Michael Miklavcic <mi...@gmail.com>
Committed: Wed Jun 6 15:40:55 2018 -0600

----------------------------------------------------------------------
 metron-analytics/metron-profiler/README.md      |  15 ++
 .../src/main/flux/profiler/remote.yaml          |   3 +-
 metron-platform/metron-common/README.md         |   5 +
 .../metron/common/bolt/ConfiguredBolt.java      |  14 +-
 .../common/bolt/ConfiguredEnrichmentBolt.java   |  11 +-
 .../common/bolt/ConfiguredIndexingBolt.java     |  13 +-
 .../common/bolt/ConfiguredParserBolt.java       |  13 +-
 .../common/bolt/ConfiguredProfilerBolt.java     |   9 +-
 .../common/configuration/Configurations.java    |   7 +-
 .../configuration/EnrichmentConfigurations.java |  26 +++
 .../configuration/IndexingConfigurations.java   |  12 +-
 .../configuration/ParserConfigurations.java     |   1 +
 .../profiler/ProfilerConfigurations.java        |  25 +++
 .../writer/ConfigurationStrategy.java           |  44 +++++
 .../writer/ConfigurationsStrategies.java        | 144 +++++++++++++++
 .../writer/EnrichmentWriterConfiguration.java   | 110 ++++++++++++
 .../writer/ParserWriterConfiguration.java       |   6 +-
 .../writer/ProfilerWriterConfiguration.java     | 109 ++++++++++++
 .../configuration/ParserConfigurationsTest.java | 120 +++++++++++++
 .../profiler/ProfilerConfigTest.java            |  13 +-
 .../writer/ConfigurationsStrategiesTest.java    |  79 +++++++++
 .../EnrichmentWriterConfigurationTest.java      |  54 ++++++
 .../writer/IndexingWriterConfigurationTest.java |  70 ++++++++
 .../writer/ParserWriterConfigurationTest.java   |  72 ++++++++
 .../writer/ProfilerWriterConfigurationTest.java |  54 ++++++
 .../writer/IndexingWriterConfigurationTest.java |  70 --------
 .../writer/ParserWriterConfigurationTest.java   |  38 ----
 metron-platform/metron-enrichment/README.md     |  15 +-
 .../main/flux/enrichment/remote-splitjoin.yaml  |   9 +-
 .../main/flux/enrichment/remote-unified.yaml    |   9 +-
 .../bolt/BulkMessageWriterBoltTest.java         |  64 ++++---
 .../src/main/flux/indexing/batch/remote.yaml    |   4 +-
 .../flux/indexing/random_access/remote.yaml     |   4 +-
 metron-platform/metron-parsers/README.md        |   6 +-
 .../apache/metron/parsers/bolt/ParserBolt.java  | 108 +++++++++++-
 .../metron/parsers/bolt/WriterHandler.java      |  63 +++++--
 .../metron/parsers/bolt/ParserBoltTest.java     | 176 +++++++++++++------
 .../integration/WriterBoltIntegrationTest.java  |   6 +-
 .../metron/writer/bolt/BatchTimeoutHelper.java  |   6 +-
 .../writer/bolt/BulkMessageWriterBolt.java      |  48 ++---
 .../apache/metron/writer/kafka/KafkaWriter.java |  92 +++++++---
 41 files changed, 1426 insertions(+), 321 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-analytics/metron-profiler/README.md
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler/README.md b/metron-analytics/metron-profiler/README.md
index 79cdd44..1a17e10 100644
--- a/metron-analytics/metron-profiler/README.md
+++ b/metron-analytics/metron-profiler/README.md
@@ -538,6 +538,8 @@ The Profiler runs as an independent Storm topology.  The configuration for the P
 | [`profiler.hbase.batch`](#profilerhbasebatch)                                 | The number of puts that are written to HBase in a single batch.
 | [`profiler.hbase.flush.interval.seconds`](#profilerhbaseflushintervalseconds) | The maximum number of seconds between batch writes to HBase.
 | [`topology.kryo.register`](#topologykryoregister)                             | Storm will use Kryo serialization for these classes.
+| [`profiler.writer.batchSize`](#profilerwriterbatchsize)                       | The number of records to batch when writing to Kakfa.
+| [`profiler.writer.batchTimeout`](#profilerwriterbatchtimeout)                 | The timeout in ms for batching when writing to Kakfa.
 
 
 ### `profiler.input.topic`
@@ -852,6 +854,19 @@ More information on accessing profile data can be found in the [Profiler Client]
 
 More information on using the [`STATS_*` functions in Stellar can be found here](../../metron-platform/metron-common).
 
+### `profiler.writer.batchSize`
+
+*Default*: 15
+
+The number of records to batch when writing to Kakfa. This is managed in the global configuration and does not require a topology restart.
+
+### `profiler.writer.batchTimeout`
+
+*Default*: 0
+
+The timeout after which a batch will be flushed even if batchSize has not been met.  Optional. If unspecified, or set to `0`, it defaults to a system-determined duration which is a fraction of the Storm parameter `topology.message.timeout.secs`.
+Ignored if batchSize is `1`, since this disables batching. This is managed in the global configuration and does not require a topology restart.
+
 ## Implementation
 
 ## Key Classes

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-analytics/metron-profiler/src/main/flux/profiler/remote.yaml
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler/src/main/flux/profiler/remote.yaml b/metron-analytics/metron-profiler/src/main/flux/profiler/remote.yaml
index 5e92c62..2f40554 100644
--- a/metron-analytics/metron-profiler/src/main/flux/profiler/remote.yaml
+++ b/metron-analytics/metron-profiler/src/main/flux/profiler/remote.yaml
@@ -183,8 +183,9 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "PROFILER"
         configMethods:
-            -   name: "withMessageWriter"
+            -   name: "withBulkMessageWriter"
                 args: [ref: "kafkaWriter"]
 
 streams:

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/README.md
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/README.md b/metron-platform/metron-common/README.md
index c741e72..dae2e22 100644
--- a/metron-platform/metron-common/README.md
+++ b/metron-platform/metron-common/README.md
@@ -92,9 +92,14 @@ but a convenient index is provided here:
 | [`stellar.function.resolver.excludes`](../../metron-stellar/stellar-common#stellarfunctionresolverincludesexcludes) | Stellar       | CSV String | N/A                        |
 | [`profiler.period.duration`](../../metron-analytics/metron-profiler#profilerperiodduration)                         | Profiler      | Integer    | `profiler_period_duration` |
 | [`profiler.period.duration.units`](../../metron-analytics/metron-profiler#profilerperioddurationunits)              | Profiler      | String     | `profiler_period_units`    |
+| [`profiler.writer.batchSize`](../../metron-analytics/metron-profiler/#profilerwriterbatchsize)                      | Profiler      | Integer    |  N/A                       |
+| [`profiler.writer.batchTimeout`](../../metron-analytics/metron-profiler/#profilerwriterbatchtimeout)                | Profiler      | Integer    |  N/A                       |
 | [`update.hbase.table`](../metron-indexing#updatehbasetable)                                                         | REST/Indexing | String     | `update_hbase_table`       |
 | [`update.hbase.cf`](../metron-indexing#updatehbasecf)                                                               | REST/Indexing | String     | `update_hbase_cf`          |
 | [`geo.hdfs.file`](../metron-enrichment#geohdfsfile)                                                                 | Enrichment    | String     | `geo_hdfs_file`            |
+| [`enrichment.writer.batchSize`](../metron-enrichment#enrichmentwriterbatchsize)                                     | Enrichment    | Integer    |  N/A                       |
+| [`enrichment.writer.batchTimeout`](../metron-enrichment#enrichmentwriterbatchtimeout)                               | Enrichment    | Integer    |  N/A                       |
+| [`geo.hdfs.file`](../metron-enrichment#geohdfsfile)                                                                 | Enrichment    | String     | `geo_hdfs_file`            |
 | [`source.type.field`](../../metron-interface/metron-alerts#sourcetypefield)                                         | UI            | String     |  N/A                       |
 
 ## Note Configs in Ambari

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredBolt.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredBolt.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredBolt.java
index 6f15746..ef3b2bf 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredBolt.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredBolt.java
@@ -28,6 +28,8 @@ import org.apache.metron.common.Constants;
 import org.apache.metron.common.configuration.ConfigurationType;
 import org.apache.metron.common.configuration.Configurations;
 import org.apache.metron.common.configuration.ConfigurationsUtils;
+import org.apache.metron.common.configuration.writer.ConfigurationStrategy;
+import org.apache.metron.common.configuration.writer.ConfigurationsStrategies;
 import org.apache.metron.zookeeper.SimpleEventListener;
 import org.apache.metron.common.zookeeper.configurations.ConfigurationsUpdater;
 import org.apache.metron.common.zookeeper.configurations.Reloadable;
@@ -43,12 +45,14 @@ public abstract class ConfiguredBolt<CONFIG_T extends Configurations> extends Ba
   private static final Logger LOG =  LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
 
   private String zookeeperUrl;
+  private String configurationStrategy;
 
   protected CuratorFramework client;
   protected ZKCache cache;
   private final CONFIG_T configurations;
-  public ConfiguredBolt(String zookeeperUrl) {
+  public ConfiguredBolt(String zookeeperUrl, String configurationStrategy) {
     this.zookeeperUrl = zookeeperUrl;
+    this.configurationStrategy = configurationStrategy;
     this.configurations = createUpdater().defaultConfigurations();
   }
 
@@ -68,7 +72,13 @@ public abstract class ConfiguredBolt<CONFIG_T extends Configurations> extends Ba
     return configurations;
   }
 
-  protected abstract ConfigurationsUpdater<CONFIG_T> createUpdater();
+  protected ConfigurationStrategy<CONFIG_T> getConfigurationStrategy() {
+    return ConfigurationsStrategies.valueOf(configurationStrategy);
+  }
+
+  protected ConfigurationsUpdater<CONFIG_T> createUpdater() {
+    return getConfigurationStrategy().createUpdater(this, this::getConfigurations);
+  }
 
 
   @Override

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredEnrichmentBolt.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredEnrichmentBolt.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredEnrichmentBolt.java
index 54fd7e8..c28ca7b 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredEnrichmentBolt.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredEnrichmentBolt.java
@@ -17,13 +17,8 @@
  */
 package org.apache.metron.common.bolt;
 
-import java.io.IOException;
 import java.lang.invoke.MethodHandles;
-import org.apache.metron.common.configuration.ConfigurationType;
-import org.apache.metron.common.configuration.ConfigurationsUtils;
 import org.apache.metron.common.configuration.EnrichmentConfigurations;
-import org.apache.metron.common.zookeeper.configurations.ConfigurationsUpdater;
-import org.apache.metron.common.zookeeper.configurations.EnrichmentUpdater;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -33,11 +28,7 @@ public abstract class ConfiguredEnrichmentBolt extends ConfiguredBolt<Enrichment
 
 
   public ConfiguredEnrichmentBolt(String zookeeperUrl) {
-    super(zookeeperUrl);
+    super(zookeeperUrl, "ENRICHMENT");
   }
 
-  @Override
-  protected ConfigurationsUpdater<EnrichmentConfigurations> createUpdater() {
-    return new EnrichmentUpdater(this, this::getConfigurations);
-  }
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredIndexingBolt.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredIndexingBolt.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredIndexingBolt.java
index 09300e4..27e081e 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredIndexingBolt.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredIndexingBolt.java
@@ -17,26 +17,17 @@
  */
 package org.apache.metron.common.bolt;
 
-import java.io.IOException;
 import java.lang.invoke.MethodHandles;
-import org.apache.metron.common.configuration.ConfigurationType;
-import org.apache.metron.common.configuration.ConfigurationsUtils;
 import org.apache.metron.common.configuration.IndexingConfigurations;
-import org.apache.metron.common.zookeeper.configurations.ConfigurationsUpdater;
-import org.apache.metron.common.zookeeper.configurations.IndexingUpdater;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+// TODO delete - no longer used after removing from BulkMessageWriterBolt?
 public abstract class ConfiguredIndexingBolt extends ConfiguredBolt<IndexingConfigurations> {
   private static final Logger LOG = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
 
   public ConfiguredIndexingBolt(String zookeeperUrl) {
-    super(zookeeperUrl);
-  }
-
-  @Override
-  protected ConfigurationsUpdater<IndexingConfigurations> createUpdater() {
-    return new IndexingUpdater(this, this::getConfigurations);
+    super(zookeeperUrl, "INDEXING");
   }
 
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredParserBolt.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredParserBolt.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredParserBolt.java
index 2f13658..1cb4e2e 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredParserBolt.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredParserBolt.java
@@ -17,14 +17,9 @@
  */
 package org.apache.metron.common.bolt;
 
-import java.io.IOException;
 import java.lang.invoke.MethodHandles;
-import org.apache.metron.common.configuration.ConfigurationType;
-import org.apache.metron.common.configuration.ConfigurationsUtils;
 import org.apache.metron.common.configuration.ParserConfigurations;
 import org.apache.metron.common.configuration.SensorParserConfig;
-import org.apache.metron.common.zookeeper.configurations.ConfigurationsUpdater;
-import org.apache.metron.common.zookeeper.configurations.ParserUpdater;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -35,7 +30,7 @@ public abstract class ConfiguredParserBolt extends ConfiguredBolt<ParserConfigur
   protected final ParserConfigurations configurations = new ParserConfigurations();
   private String sensorType;
   public ConfiguredParserBolt(String zookeeperUrl, String sensorType) {
-    super(zookeeperUrl);
+    super(zookeeperUrl, "PARSERS");
     this.sensorType = sensorType;
   }
 
@@ -47,10 +42,4 @@ public abstract class ConfiguredParserBolt extends ConfiguredBolt<ParserConfigur
     return sensorType;
   }
 
-
-  @Override
-  protected ConfigurationsUpdater<ParserConfigurations> createUpdater() {
-    return new ParserUpdater(this, this::getConfigurations);
-  }
-
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredProfilerBolt.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredProfilerBolt.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredProfilerBolt.java
index 90575d0..e4d9f7b 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredProfilerBolt.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/bolt/ConfiguredProfilerBolt.java
@@ -21,8 +21,6 @@ package org.apache.metron.common.bolt;
 import java.lang.invoke.MethodHandles;
 import org.apache.metron.common.configuration.profiler.ProfilerConfig;
 import org.apache.metron.common.configuration.profiler.ProfilerConfigurations;
-import org.apache.metron.common.zookeeper.configurations.ConfigurationsUpdater;
-import org.apache.metron.common.zookeeper.configurations.ProfilerUpdater;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -34,16 +32,11 @@ public abstract class ConfiguredProfilerBolt extends ConfiguredBolt<ProfilerConf
   private static final Logger LOG = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
 
   public ConfiguredProfilerBolt(String zookeeperUrl) {
-    super(zookeeperUrl);
+    super(zookeeperUrl, "PROFILER");
   }
 
   protected ProfilerConfig getProfilerConfig() {
     return getConfigurations().getProfilerConfig();
   }
 
-  @Override
-  protected ConfigurationsUpdater<ProfilerConfigurations> createUpdater() {
-    return new ProfilerUpdater(this, this::getConfigurations);
-  }
-
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/Configurations.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/Configurations.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/Configurations.java
index af421a9..62cf742 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/Configurations.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/Configurations.java
@@ -27,8 +27,8 @@ import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
 import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.ConcurrentMap;
 import org.apache.metron.common.utils.JSONUtils;
+import org.apache.metron.stellar.common.utils.ConversionUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -75,6 +75,11 @@ public class Configurations implements Serializable {
     getConfigurations().remove(ConfigurationType.GLOBAL.getTypeName());
   }
 
+  public static <T> T getAs(String key, Map<String, Object> map, T defaultValue, Class<T> clazz) {
+    return map == null ? defaultValue
+        : ConversionUtils.convert(map.getOrDefault(key, defaultValue), clazz);
+  }
+
   @Override
   public boolean equals(Object o) {
     if (this == o) return true;

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/EnrichmentConfigurations.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/EnrichmentConfigurations.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/EnrichmentConfigurations.java
index dfd7a65..1c723c7 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/EnrichmentConfigurations.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/EnrichmentConfigurations.java
@@ -27,6 +27,9 @@ import java.util.ArrayList;
 import java.util.List;
 
 public class EnrichmentConfigurations extends Configurations {
+  public static final Integer DEFAULT_KAFKA_BATCH_SIZE = 15;
+  public static final String BATCH_SIZE_CONF = "enrichment.writer.batchSize";
+  public static final String BATCH_TIMEOUT_CONF = "enrichment.writer.batchTimeout";
 
   public SensorEnrichmentConfig getSensorEnrichmentConfig(String sensorType) {
     return (SensorEnrichmentConfig) getConfigurations().get(getKey(sensorType));
@@ -49,6 +52,28 @@ public class EnrichmentConfigurations extends Configurations {
     getConfigurations().remove(getKey(sensorType));
   }
 
+  /**
+   * Pulled from global config.
+   * Note: enrichment writes out to 1 kafka topic, so it is not pulling this config by sensor.
+   *
+   * @return batch size for writing to kafka
+   * @see org.apache.metron.common.configuration.EnrichmentConfigurations#BATCH_SIZE_CONF
+   */
+  public int getBatchSize() {
+    return getAs(BATCH_SIZE_CONF, getGlobalConfig(true), DEFAULT_KAFKA_BATCH_SIZE, Integer.class);
+  }
+
+  /**
+   * Pulled from global config
+   * Note: enrichment writes out to 1 kafka topic, so it is not pulling this config by sensor.
+   *
+   * @return batch timeout for writing to kafka
+   * @see org.apache.metron.common.configuration.EnrichmentConfigurations#BATCH_TIMEOUT_CONF
+   */
+  public int getBatchTimeout() {
+    return getAs(BATCH_TIMEOUT_CONF, getGlobalConfig(true), 0, Integer.class);
+  }
+
   public List<String> getTypes() {
     List<String> ret = new ArrayList<>();
     for(String keyedSensor : getConfigurations().keySet()) {
@@ -62,4 +87,5 @@ public class EnrichmentConfigurations extends Configurations {
   public static String getKey(String sensorType) {
     return ConfigurationType.ENRICHMENT.getTypeName() + "." + sensorType;
   }
+
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/IndexingConfigurations.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/IndexingConfigurations.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/IndexingConfigurations.java
index 5b67aa5..7ecb606 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/IndexingConfigurations.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/IndexingConfigurations.java
@@ -17,13 +17,14 @@
  */
 package org.apache.metron.common.configuration;
 
-import org.apache.metron.stellar.common.utils.ConversionUtils;
-import org.apache.metron.common.utils.JSONUtils;
-
 import java.io.ByteArrayInputStream;
 import java.io.IOException;
 import java.io.InputStream;
-import java.util.*;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.metron.common.utils.JSONUtils;
 
 public class IndexingConfigurations extends Configurations {
   public static final String BATCH_SIZE_CONF = "batchSize";
@@ -226,7 +227,4 @@ public class IndexingConfigurations extends Configurations {
     return ret;
   }
 
-  public static <T> T getAs(String key, Map<String, Object> map, T defaultValue, Class<T> clazz) {
-    return map == null?defaultValue: ConversionUtils.convert(map.getOrDefault(key, defaultValue), clazz);
-  }
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ParserConfigurations.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ParserConfigurations.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ParserConfigurations.java
index 72af833..83daf0d 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ParserConfigurations.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ParserConfigurations.java
@@ -26,6 +26,7 @@ import java.util.ArrayList;
 import java.util.List;
 
 public class ParserConfigurations extends Configurations {
+  public static final Integer DEFAULT_KAFKA_BATCH_SIZE = 15;
 
   public SensorParserConfig getSensorParserConfig(String sensorType) {
     return (SensorParserConfig) getConfigurations().get(getKey(sensorType));

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/profiler/ProfilerConfigurations.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/profiler/ProfilerConfigurations.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/profiler/ProfilerConfigurations.java
index f50d770..1348068 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/profiler/ProfilerConfigurations.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/profiler/ProfilerConfigurations.java
@@ -29,6 +29,9 @@ import java.io.InputStream;
  * Used to manage configurations for the Profiler.
  */
 public class ProfilerConfigurations extends Configurations {
+  public static final Integer DEFAULT_KAFKA_BATCH_SIZE = 15;
+  public static final String BATCH_SIZE_CONF = "profiler.writer.batchSize";
+  public static final String BATCH_TIMEOUT_CONF = "profiler.writer.batchTimeout";
 
   public ProfilerConfig getProfilerConfig() {
     return (ProfilerConfig) getConfigurations().get(getKey());
@@ -55,4 +58,26 @@ public class ProfilerConfigurations extends Configurations {
     configurations.remove(getKey());
   }
 
+  /**
+   * Pulled from global config.
+   * Note: profiler writes out to 1 kafka topic, so it is not pulling this config by sensor.
+   *
+   * @return batch size for writing to kafka
+   * @see org.apache.metron.common.configuration.profiler.ProfilerConfigurations#BATCH_SIZE_CONF
+   */
+  public int getBatchSize() {
+    return getAs(BATCH_SIZE_CONF, getGlobalConfig(true), DEFAULT_KAFKA_BATCH_SIZE, Integer.class);
+  }
+
+  /**
+   * Pulled from global config
+   * Note: profiler writes out to 1 kafka topic, so it is not pulling this config by sensor.
+   *
+   * @return batch timeout for writing to kafka
+   * @see org.apache.metron.common.configuration.profiler.ProfilerConfigurations#BATCH_TIMEOUT_CONF
+   */
+  public int getBatchTimeout() {
+    return getAs(BATCH_TIMEOUT_CONF, getGlobalConfig(true), 0, Integer.class);
+  }
+
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ConfigurationStrategy.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ConfigurationStrategy.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ConfigurationStrategy.java
new file mode 100644
index 0000000..cd13dc7
--- /dev/null
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ConfigurationStrategy.java
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.common.configuration.writer;
+
+import java.util.function.Supplier;
+import org.apache.metron.common.configuration.Configurations;
+import org.apache.metron.common.writer.BulkMessageWriter;
+import org.apache.metron.common.zookeeper.configurations.ConfigurationsUpdater;
+import org.apache.metron.common.zookeeper.configurations.Reloadable;
+
+public interface ConfigurationStrategy<T extends Configurations> {
+
+  /**
+   * Create a specific writer configuration
+   * @param writer provided for the underlying creator to access metadata as needed
+   * @param configs a WriterConfiguration will typically access the pass configs
+   * @return
+   */
+  WriterConfiguration createWriterConfig(BulkMessageWriter writer, Configurations configs);
+
+  /**
+   * Create specific config updater for the type of config extending Configurations
+   * @param reloadable setup as a callback by the updater
+   * @param configSupplier supplies config to the updater
+   * @return updater
+   */
+  ConfigurationsUpdater<T> createUpdater(Reloadable reloadable, Supplier<T> configSupplier);
+
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ConfigurationsStrategies.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ConfigurationsStrategies.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ConfigurationsStrategies.java
new file mode 100644
index 0000000..f22dd01
--- /dev/null
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ConfigurationsStrategies.java
@@ -0,0 +1,144 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.common.configuration.writer;
+
+import java.util.function.Supplier;
+import org.apache.metron.common.configuration.Configurations;
+import org.apache.metron.common.configuration.EnrichmentConfigurations;
+import org.apache.metron.common.configuration.IndexingConfigurations;
+import org.apache.metron.common.configuration.ParserConfigurations;
+import org.apache.metron.common.configuration.profiler.ProfilerConfigurations;
+import org.apache.metron.common.writer.BulkMessageWriter;
+import org.apache.metron.common.zookeeper.configurations.ConfigurationsUpdater;
+import org.apache.metron.common.zookeeper.configurations.EnrichmentUpdater;
+import org.apache.metron.common.zookeeper.configurations.IndexingUpdater;
+import org.apache.metron.common.zookeeper.configurations.ParserUpdater;
+import org.apache.metron.common.zookeeper.configurations.ProfilerUpdater;
+import org.apache.metron.common.zookeeper.configurations.Reloadable;
+
+/**
+ * Strategy pattern implementation that couples factories for WriterConfiguration and
+ * ConfigurationsUpdater together for a particular type.
+ * <br>
+ * <strong>Note:</strong> The enum type definition does not specify generics because
+ * enums are disallowed from doing this in Java.
+ */
+public enum ConfigurationsStrategies implements ConfigurationStrategy {
+
+  PARSERS(new ConfigurationStrategy<ParserConfigurations>() {
+
+    @Override
+    public WriterConfiguration createWriterConfig(BulkMessageWriter writer,
+        Configurations configs) {
+      if (configs instanceof ParserConfigurations) {
+        return new ParserWriterConfiguration((ParserConfigurations) configs);
+      } else {
+        throw new IllegalArgumentException(
+            "Expected config of type ParserConfigurations but found " + configs.getClass());
+      }
+    }
+
+    @Override
+    public ConfigurationsUpdater<ParserConfigurations> createUpdater(Reloadable reloadable,
+        Supplier configSupplier) {
+      return new ParserUpdater(reloadable, configSupplier);
+    }
+  }),
+
+  ENRICHMENT(new ConfigurationStrategy() {
+
+    @Override
+    public WriterConfiguration createWriterConfig(BulkMessageWriter writer,
+        Configurations configs) {
+      if (configs instanceof EnrichmentConfigurations) {
+        return new EnrichmentWriterConfiguration((EnrichmentConfigurations) configs);
+      } else {
+        throw new IllegalArgumentException(
+            "Expected config of type EnrichmentConfigurations but found " + configs.getClass());
+      }
+    }
+
+    @Override
+    public ConfigurationsUpdater<EnrichmentConfigurations> createUpdater(Reloadable reloadable,
+        Supplier configSupplier) {
+      return new EnrichmentUpdater(reloadable, configSupplier);
+    }
+  }),
+
+  INDEXING(new ConfigurationStrategy() {
+
+    @Override
+    public WriterConfiguration createWriterConfig(BulkMessageWriter writer,
+        Configurations configs) {
+      if (configs instanceof IndexingConfigurations) {
+        return new IndexingWriterConfiguration(writer.getName(), (IndexingConfigurations) configs);
+      } else {
+        throw new IllegalArgumentException(
+            "Expected config of type IndexingConfigurations but found " + configs.getClass());
+      }
+    }
+
+    @Override
+    public ConfigurationsUpdater<IndexingConfigurations> createUpdater(Reloadable reloadable,
+        Supplier configSupplier) {
+      return new IndexingUpdater(reloadable, configSupplier);
+    }
+  }),
+
+  PROFILER(new ConfigurationStrategy() {
+
+    @Override
+    public WriterConfiguration createWriterConfig(BulkMessageWriter writer,
+        Configurations configs) {
+        if (configs instanceof ProfilerConfigurations) {
+          return new ProfilerWriterConfiguration((ProfilerConfigurations) configs);
+        } else {
+          throw new IllegalArgumentException(
+              "Expected config of type IndexingConfigurations but found " + configs.getClass());
+        }
+    }
+
+    @Override
+    public ConfigurationsUpdater createUpdater(Reloadable reloadable, Supplier configSupplier) {
+      return new ProfilerUpdater(reloadable, configSupplier);
+    }
+  });
+
+  private ConfigurationStrategy<? extends Configurations> strategy;
+
+  ConfigurationsStrategies(ConfigurationStrategy<? extends Configurations> strategy) {
+    this.strategy = strategy;
+  }
+
+  @Override
+  public WriterConfiguration createWriterConfig(BulkMessageWriter writer, Configurations configs) {
+    return strategy.createWriterConfig(writer, configs);
+  }
+
+  /**
+   * Config updater
+   * @param reloadable callback
+   * @param configSupplier Supplier provides config of type <? extends Configurations>
+   * @return Config updater
+   */
+  @Override
+  public ConfigurationsUpdater createUpdater(Reloadable reloadable, Supplier configSupplier) {
+    return strategy.createUpdater(reloadable, configSupplier);
+  }
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/EnrichmentWriterConfiguration.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/EnrichmentWriterConfiguration.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/EnrichmentWriterConfiguration.java
new file mode 100644
index 0000000..c275a76
--- /dev/null
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/EnrichmentWriterConfiguration.java
@@ -0,0 +1,110 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.common.configuration.writer;
+
+import static java.util.Arrays.asList;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.metron.common.configuration.EnrichmentConfigurations;
+
+/**
+ * Writer configuration for the enrichment topology. Batch size and batch timeout are a couple
+ * primary values of interest that are used for configuring the enrichment writer.
+ */
+public class EnrichmentWriterConfiguration implements WriterConfiguration {
+
+  private Optional<EnrichmentConfigurations> config;
+
+  public EnrichmentWriterConfiguration(EnrichmentConfigurations config) {
+    this.config = Optional.ofNullable(config);
+  }
+
+  /**
+   * Batch size for writing.
+   * @param sensorName n/a
+   * @return batch size in # messages
+   */
+  @Override
+  public int getBatchSize(String sensorName) {
+    return config.orElse(new EnrichmentConfigurations()).getBatchSize();
+  }
+
+  /**
+   * Timeout for this writer.
+   * @param sensorName n/a
+   * @return timeout in ms
+   */
+  @Override
+  public int getBatchTimeout(String sensorName) {
+    return config.orElse(new EnrichmentConfigurations()).getBatchTimeout();
+  }
+
+  /**
+   * Timeout for this writer.
+   * @return single item list with this writer's timeout
+   */
+  @Override
+  public List<Integer> getAllConfiguredTimeouts() {
+    return asList(getBatchTimeout(null));
+  }
+
+  /**
+   * n/a for enrichment
+   * @param sensorName n/a
+   * @return null
+   */
+  @Override
+  public String getIndex(String sensorName) {
+    return null;
+  }
+
+  /**
+   * Always enabled in enrichment
+   * @param sensorName n/a
+   * @return true
+   */
+  @Override
+  public boolean isEnabled(String sensorName) {
+    return true;
+  }
+
+  @Override
+  public Map<String, Object> getSensorConfig(String sensorName) {
+    return config.orElse(new EnrichmentConfigurations()).getSensorEnrichmentConfig(sensorName)
+        .getConfiguration();
+  }
+
+  @Override
+  public Map<String, Object> getGlobalConfig() {
+    return config.orElse(new EnrichmentConfigurations()).getGlobalConfig();
+  }
+
+  @Override
+  public boolean isDefault(String sensorName) {
+    return false;
+  }
+
+  @Override
+  public String getFieldNameConverter(String sensorName) {
+    // not applicable
+    return null;
+  }
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ParserWriterConfiguration.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ParserWriterConfiguration.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ParserWriterConfiguration.java
index 4603b32..a4f3f66 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ParserWriterConfiguration.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ParserWriterConfiguration.java
@@ -26,6 +26,10 @@ import java.util.ArrayList;
 import java.util.List;
 import java.util.Map;
 
+/*
+ * Note that parsers can also be used for streaming enrichments, which means broader scope than
+ * Kafka alone.
+ */
 public class ParserWriterConfiguration implements WriterConfiguration {
   private ParserConfigurations config;
   public ParserWriterConfiguration(ParserConfigurations config) {
@@ -38,7 +42,7 @@ public class ParserWriterConfiguration implements WriterConfiguration {
             && config.getSensorParserConfig(sensorName).getParserConfig() != null
             ) {
       Object batchObj = config.getSensorParserConfig(sensorName).getParserConfig().get(IndexingConfigurations.BATCH_SIZE_CONF);
-      return batchObj == null ? 1 : ConversionUtils.convert(batchObj, Integer.class);
+      return batchObj == null ? ParserConfigurations.DEFAULT_KAFKA_BATCH_SIZE : ConversionUtils.convert(batchObj, Integer.class);
     }
     return 1;
   }

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ProfilerWriterConfiguration.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ProfilerWriterConfiguration.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ProfilerWriterConfiguration.java
new file mode 100644
index 0000000..fe283bd
--- /dev/null
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/writer/ProfilerWriterConfiguration.java
@@ -0,0 +1,109 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.common.configuration.writer;
+
+import static java.util.Arrays.asList;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.metron.common.configuration.profiler.ProfilerConfigurations;
+
+/**
+ * Writer configuration for the profiler topology. Batch size and batch timeout are a couple
+ * primary values of interest that are used for configuring the profiler writer.
+ */
+public class ProfilerWriterConfiguration implements WriterConfiguration {
+  private Optional<ProfilerConfigurations> config;
+
+  public ProfilerWriterConfiguration(ProfilerConfigurations config) {
+    this.config = Optional.ofNullable(config);
+  }
+
+  /**
+   * Batch size for writing.
+   * @param sensorName n/a
+   * @return batch size in # messages
+   */
+  @Override
+  public int getBatchSize(String sensorName) {
+    return config.orElse(new ProfilerConfigurations()).getBatchSize();
+  }
+
+  /**
+   * Timeout for this writer.
+   * @param sensorName n/a
+   * @return timeout in ms
+   */
+  @Override
+  public int getBatchTimeout(String sensorName) {
+    return config.orElse(new ProfilerConfigurations()).getBatchTimeout();
+  }
+
+  /**
+   * Timeout for this writer.
+   * @return single item list with this writer's timeout
+   */
+  @Override
+  public List<Integer> getAllConfiguredTimeouts() {
+    return asList(getBatchTimeout(null));
+  }
+
+  /**
+   * n/a for profiler
+   * @param sensorName n/a
+   * @return null
+   */
+  @Override
+  public String getIndex(String sensorName) {
+    return null;
+  }
+
+  /**
+   * Always enabled in profiler
+   * @param sensorName n/a
+   * @return true
+   */
+  @Override
+  public boolean isEnabled(String sensorName) {
+    return true;
+  }
+
+  @Override
+  public Map<String, Object> getSensorConfig(String sensorName) {
+    throw new UnsupportedOperationException("Profiler does not have sensor configs");
+  }
+
+  @Override
+  public Map<String, Object> getGlobalConfig() {
+    return config.orElse(new ProfilerConfigurations()).getGlobalConfig();
+  }
+
+  @Override
+  public boolean isDefault(String sensorName) {
+    return false;
+  }
+
+  @Override
+  public String getFieldNameConverter(String sensorName) {
+    // not applicable
+    return null;
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/ParserConfigurationsTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/ParserConfigurationsTest.java b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/ParserConfigurationsTest.java
new file mode 100644
index 0000000..c099aec
--- /dev/null
+++ b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/ParserConfigurationsTest.java
@@ -0,0 +1,120 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.common.configuration;
+
+import static org.hamcrest.CoreMatchers.equalTo;
+import static org.hamcrest.CoreMatchers.not;
+import static org.hamcrest.CoreMatchers.nullValue;
+import static org.junit.Assert.assertThat;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import org.adrianwalker.multilinestring.Multiline;
+import org.junit.Test;
+
+public class ParserConfigurationsTest {
+
+  /**
+   * {
+   *  "parserClassName" : "parser-class",
+   *  "filterClassName" : "filter-class",
+   *  "sensorTopic" : "sensor-topic",
+   *  "outputTopic" : "output-topic",
+   *  "errorTopic" : "error-topic",
+   *  "writerClassName" : "writer-class",
+   *  "errorWriterClassName" : "error-writer-class",
+   *  "readMetadata" : true,
+   *  "mergeMetadata" : true,
+   *  "numWorkers" : 40,
+   *  "numAckers" : 40,
+   *  "spoutParallelism" : 40,
+   *  "spoutNumTasks" : 40,
+   *  "parserParallelism" : 40,
+   *  "parserNumTasks" : 40,
+   *  "errorWriterParallelism" : 40,
+   *  "errorWriterNumTasks" : 40,
+   *  "securityProtocol" : "security-protocol",
+   *  "spoutConfig" : {
+   *    "foo" : "bar"
+   *  },
+   *  "stormConfig" : {
+   *    "storm" : "config"
+   *  },
+   *  "cacheConfig" : {
+   *    "stellar.cache.maxSize" : 20000
+   *  },
+   *  "parserConfig" : {
+   *    "parser" : "config"
+   *  },
+   *  "fieldTransformations" : [
+   *    {
+   *      "input" : "input-field",
+   *      "transformation" : "REMOVE"
+   *    }
+   *  ]
+   * }
+   */
+  @Multiline
+  private static String parserConfig;
+
+  @Test
+  public void sensorParserConfig_properties_populated_by_JSON_configuration() throws IOException {
+    ParserConfigurations parserConfigs = new ParserConfigurations();
+    parserConfigs.updateSensorParserConfig("test-sensor", parserConfig.getBytes());
+    SensorParserConfig actualSensorConfig = parserConfigs.getSensorParserConfig("test-sensor");
+    assertThat(actualSensorConfig.getParserClassName(), equalTo("parser-class"));
+    assertThat(actualSensorConfig.getFilterClassName(), equalTo("filter-class"));
+    assertThat(actualSensorConfig.getSensorTopic(), equalTo("sensor-topic"));
+    assertThat(actualSensorConfig.getOutputTopic(), equalTo("output-topic"));
+    assertThat(actualSensorConfig.getErrorTopic(), equalTo("error-topic"));
+    assertThat(actualSensorConfig.getWriterClassName(), equalTo("writer-class"));
+    assertThat(actualSensorConfig.getErrorWriterClassName(), equalTo("error-writer-class"));
+    assertThat(actualSensorConfig.getReadMetadata(), equalTo(true));
+    assertThat(actualSensorConfig.getMergeMetadata(), equalTo(true));
+    assertThat(actualSensorConfig.getNumWorkers(), equalTo(40));
+    assertThat(actualSensorConfig.getNumAckers(), equalTo(40));
+    assertThat(actualSensorConfig.getSpoutParallelism(), equalTo(40));
+    assertThat(actualSensorConfig.getSpoutNumTasks(), equalTo(40));
+    assertThat(actualSensorConfig.getParserParallelism(), equalTo(40));
+    assertThat(actualSensorConfig.getParserNumTasks(), equalTo(40));
+    assertThat(actualSensorConfig.getErrorWriterParallelism(), equalTo(40));
+    assertThat(actualSensorConfig.getErrorWriterNumTasks(), equalTo(40));
+    assertThat(actualSensorConfig.getSecurityProtocol(), equalTo("security-protocol"));
+    assertThat(actualSensorConfig.getSpoutConfig(), not(new HashMap<>()));
+    assertThat(actualSensorConfig.getSpoutConfig().get("foo"), equalTo("bar"));
+    assertThat(actualSensorConfig.getStormConfig(), not(new HashMap<>()));
+    assertThat(actualSensorConfig.getStormConfig().get("storm"), equalTo("config"));
+    assertThat(actualSensorConfig.getCacheConfig(), not(new HashMap<>()));
+    assertThat(actualSensorConfig.getCacheConfig().get("stellar.cache.maxSize"), equalTo(20000));
+    assertThat(actualSensorConfig.getParserConfig(), not(new HashMap<>()));
+    assertThat(actualSensorConfig.getParserConfig().get("parser"), equalTo("config"));
+    assertThat(actualSensorConfig.getFieldTransformations(), not(new ArrayList<>()));
+    assertThat(actualSensorConfig.getFieldTransformations().get(0), not(nullValue()));
+    assertThat(
+        ((FieldTransformer) actualSensorConfig.getFieldTransformations().get(0)).getInput().size(),
+        equalTo(1));
+    assertThat(
+        ((FieldTransformer) actualSensorConfig.getFieldTransformations().get(0)).getInput().get(0),
+        equalTo("input-field"));
+    assertThat(((FieldTransformer) actualSensorConfig.getFieldTransformations().get(0))
+        .getTransformation(), equalTo("REMOVE"));
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/profiler/ProfilerConfigTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/profiler/ProfilerConfigTest.java b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/profiler/ProfilerConfigTest.java
index 2cbdfb9..4c70a8c 100644
--- a/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/profiler/ProfilerConfigTest.java
+++ b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/profiler/ProfilerConfigTest.java
@@ -19,19 +19,18 @@
  */
 package org.apache.metron.common.configuration.profiler;
 
-import org.adrianwalker.multilinestring.Multiline;
-import org.apache.metron.common.utils.SerDeUtils;
-import org.junit.Test;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
 
 import java.io.ByteArrayInputStream;
 import java.io.ByteArrayOutputStream;
 import java.io.IOException;
 import java.io.ObjectInputStream;
 import java.io.ObjectOutputStream;
-
-import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertFalse;
-import static org.junit.Assert.assertTrue;
+import org.adrianwalker.multilinestring.Multiline;
+import org.apache.metron.common.utils.SerDeUtils;
+import org.junit.Test;
 
 /**
  * Tests the {@link ProfilerConfig} class.

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ConfigurationsStrategiesTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ConfigurationsStrategiesTest.java b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ConfigurationsStrategiesTest.java
new file mode 100644
index 0000000..fd498ba
--- /dev/null
+++ b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ConfigurationsStrategiesTest.java
@@ -0,0 +1,79 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.common.configuration.writer;
+
+import static org.apache.metron.common.configuration.writer.ConfigurationsStrategies.ENRICHMENT;
+import static org.apache.metron.common.configuration.writer.ConfigurationsStrategies.INDEXING;
+import static org.apache.metron.common.configuration.writer.ConfigurationsStrategies.PARSERS;
+import static org.apache.metron.common.configuration.writer.ConfigurationsStrategies.PROFILER;
+import static org.hamcrest.CoreMatchers.instanceOf;
+import static org.junit.Assert.assertThat;
+
+import org.apache.metron.common.configuration.EnrichmentConfigurations;
+import org.apache.metron.common.configuration.IndexingConfigurations;
+import org.apache.metron.common.configuration.ParserConfigurations;
+import org.apache.metron.common.configuration.profiler.ProfilerConfigurations;
+import org.apache.metron.common.writer.BulkMessageWriter;
+import org.apache.metron.common.zookeeper.configurations.EnrichmentUpdater;
+import org.apache.metron.common.zookeeper.configurations.IndexingUpdater;
+import org.apache.metron.common.zookeeper.configurations.ParserUpdater;
+import org.apache.metron.common.zookeeper.configurations.ProfilerUpdater;
+import org.apache.metron.common.zookeeper.configurations.Reloadable;
+import org.junit.Before;
+import org.junit.Test;
+import org.mockito.Mock;
+import org.mockito.MockitoAnnotations;
+
+public class ConfigurationsStrategiesTest {
+
+  @Mock
+  private BulkMessageWriter writer;
+  @Mock
+  private Reloadable reloadable;
+
+  @Before
+  public void setup() {
+    MockitoAnnotations.initMocks(this);
+  }
+
+  @Test
+  public void strategies_build_writer_configs() {
+    assertThat(PARSERS.createWriterConfig(writer, new ParserConfigurations()),
+        instanceOf(ParserWriterConfiguration.class));
+    assertThat(ENRICHMENT.createWriterConfig(writer, new EnrichmentConfigurations()),
+        instanceOf(EnrichmentWriterConfiguration.class));
+    assertThat(INDEXING.createWriterConfig(writer, new IndexingConfigurations()),
+        instanceOf(IndexingWriterConfiguration.class));
+    assertThat(PROFILER.createWriterConfig(writer, new ProfilerConfigurations()),
+        instanceOf(ProfilerWriterConfiguration.class));
+  }
+
+  @Test
+  public void strategies_build_updaters() {
+    assertThat(PARSERS.createUpdater(reloadable, ParserConfigurations::new),
+        instanceOf(ParserUpdater.class));
+    assertThat(ENRICHMENT.createUpdater(reloadable, EnrichmentConfigurations::new),
+        instanceOf(EnrichmentUpdater.class));
+    assertThat(INDEXING.createUpdater(reloadable, IndexingConfigurations::new),
+        instanceOf(IndexingUpdater.class));
+    assertThat(PROFILER.createUpdater(reloadable, ProfilerConfigurations::new),
+        instanceOf(ProfilerUpdater.class));
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/EnrichmentWriterConfigurationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/EnrichmentWriterConfigurationTest.java b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/EnrichmentWriterConfigurationTest.java
new file mode 100644
index 0000000..b6aae19
--- /dev/null
+++ b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/EnrichmentWriterConfigurationTest.java
@@ -0,0 +1,54 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.common.configuration.writer;
+
+import static java.util.Arrays.asList;
+import static org.hamcrest.CoreMatchers.equalTo;
+import static org.junit.Assert.assertThat;
+
+import java.io.IOException;
+import org.adrianwalker.multilinestring.Multiline;
+import org.apache.metron.common.configuration.EnrichmentConfigurations;
+import org.junit.Test;
+
+public class EnrichmentWriterConfigurationTest {
+
+  /**
+   * {
+   *  "enrichment.writer.batchSize" : 12345,
+   *  "enrichment.writer.batchTimeout" : 555
+   * }
+   */
+  @Multiline
+  private static String globalJson;
+
+  @Test
+  public void gets_batch_size_and_timeout_from_global_config() throws IOException {
+    EnrichmentConfigurations configs = new EnrichmentConfigurations();
+    configs.updateGlobalConfig(globalJson.getBytes());
+    EnrichmentWriterConfiguration writerConfig = new EnrichmentWriterConfiguration(configs);
+    assertThat("batch timeout should match global config setting",
+        writerConfig.getBatchTimeout(null), equalTo(555));
+    assertThat("list should have single batch timeout matching global config setting",
+        writerConfig.getAllConfiguredTimeouts(), equalTo(asList(555)));
+    assertThat("batch size should match global config setting", writerConfig.getBatchSize(null),
+        equalTo(12345));
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/IndexingWriterConfigurationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/IndexingWriterConfigurationTest.java b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/IndexingWriterConfigurationTest.java
new file mode 100644
index 0000000..b2b5eca
--- /dev/null
+++ b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/IndexingWriterConfigurationTest.java
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.common.configuration.writer;
+
+import org.apache.metron.common.configuration.IndexingConfigurations;
+import org.apache.metron.common.configuration.writer.IndexingWriterConfiguration;
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+
+import static org.apache.metron.test.bolt.BaseEnrichmentBoltTest.sampleSensorIndexingConfigPath;
+import static org.apache.metron.test.bolt.BaseEnrichmentBoltTest.sensorType;
+
+public class IndexingWriterConfigurationTest {
+  @Test
+  public void testDefaultBatchSize() {
+    IndexingWriterConfiguration config = new IndexingWriterConfiguration("hdfs",
+           new IndexingConfigurations()
+    );
+    Assert.assertEquals(1, config.getBatchSize("foo"));
+  }
+  @Test
+  public void testDefaultBatchTimeout() {
+    IndexingWriterConfiguration config = new IndexingWriterConfiguration("hdfs",
+           new IndexingConfigurations()
+    );
+    Assert.assertEquals(0, config.getBatchTimeout("foo"));
+  }
+  @Test
+  public void testGetAllConfiguredTimeouts() throws FileNotFoundException, IOException {
+    //default
+    IndexingWriterConfiguration config = new IndexingWriterConfiguration("hdfs",
+            new IndexingConfigurations()
+    );
+    Assert.assertEquals(0, config.getAllConfiguredTimeouts().size());
+    //non-default
+    IndexingConfigurations iconfigs = new IndexingConfigurations();
+    iconfigs.updateSensorIndexingConfig(
+            sensorType, new FileInputStream(sampleSensorIndexingConfigPath));
+    config = new IndexingWriterConfiguration("elasticsearch", iconfigs);
+    Assert.assertEquals(1, config.getAllConfiguredTimeouts().size());
+    Assert.assertEquals(7, (long)config.getAllConfiguredTimeouts().get(0));
+  }
+  @Test
+  public void testDefaultIndex() {
+    IndexingWriterConfiguration config = new IndexingWriterConfiguration("hdfs",
+           new IndexingConfigurations()
+    );
+    Assert.assertEquals("foo", config.getIndex("foo"));
+  }
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ParserWriterConfigurationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ParserWriterConfigurationTest.java b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ParserWriterConfigurationTest.java
new file mode 100644
index 0000000..ebe3d9b
--- /dev/null
+++ b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ParserWriterConfigurationTest.java
@@ -0,0 +1,72 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.common.configuration.writer;
+
+import static org.hamcrest.CoreMatchers.equalTo;
+import static org.junit.Assert.assertThat;
+
+import java.io.IOException;
+import org.adrianwalker.multilinestring.Multiline;
+import org.apache.metron.common.configuration.ParserConfigurations;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class ParserWriterConfigurationTest {
+
+  @Test
+  public void testDefaultBatchSize() {
+    ParserWriterConfiguration config = new ParserWriterConfiguration( new ParserConfigurations() );
+    Assert.assertEquals(1, config.getBatchSize("foo"));
+  }
+
+  @Test
+  public void testDefaultIndex() {
+    ParserWriterConfiguration config = new ParserWriterConfiguration( new ParserConfigurations() );
+    Assert.assertEquals("foo", config.getIndex("foo"));
+  }
+
+  /**
+   * {
+   *   "parserClassName":"some-parser",
+   *   "sensorTopic":"testtopic",
+   *   "parserConfig": {
+   *     "batchSize" : 5,
+   *     "batchTimeout" : "10000",
+   *     "index" : "modified-index",
+   *     "enabled" : "false"
+   *   }
+   * }
+   */
+  @Multiline
+  private static String configJson;
+
+  @Test
+  public void pulls_writer_configuration_from_parserConfig() throws IOException {
+    ParserConfigurations parserConfigurations = new ParserConfigurations();
+    final String sensorName = "some-sensor";
+    parserConfigurations.updateSensorParserConfig("some-sensor", configJson.getBytes());
+    ParserWriterConfiguration writerConfiguration = new ParserWriterConfiguration(
+        parserConfigurations);
+    assertThat("batch size should match", writerConfiguration.getBatchSize(sensorName), equalTo(5));
+    assertThat("batch timeout should match", writerConfiguration.getBatchTimeout(sensorName), equalTo(10000));
+    assertThat("index should match", writerConfiguration.getIndex(sensorName), equalTo("modified-index"));
+    assertThat("enabled should match", writerConfiguration.isEnabled(sensorName), equalTo(false));
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ProfilerWriterConfigurationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ProfilerWriterConfigurationTest.java b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ProfilerWriterConfigurationTest.java
new file mode 100644
index 0000000..3639276
--- /dev/null
+++ b/metron-platform/metron-common/src/test/java/org/apache/metron/common/configuration/writer/ProfilerWriterConfigurationTest.java
@@ -0,0 +1,54 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.common.configuration.writer;
+
+import static java.util.Arrays.asList;
+import static org.hamcrest.CoreMatchers.equalTo;
+import static org.junit.Assert.assertThat;
+
+import java.io.IOException;
+import org.adrianwalker.multilinestring.Multiline;
+import org.apache.metron.common.configuration.profiler.ProfilerConfigurations;
+import org.junit.Test;
+
+public class ProfilerWriterConfigurationTest {
+
+  /**
+   * {
+   *  "profiler.writer.batchSize" : 12345,
+   *  "profiler.writer.batchTimeout" : 555
+   * }
+   */
+  @Multiline
+  private static String globalJson;
+
+  @Test
+  public void gets_batch_size_and_timeout_from_global_config() throws IOException {
+    ProfilerConfigurations configs = new ProfilerConfigurations();
+    configs.updateGlobalConfig(globalJson.getBytes());
+    ProfilerWriterConfiguration writerConfig = new ProfilerWriterConfiguration(configs);
+    assertThat("batch timeout should match global config setting",
+        writerConfig.getBatchTimeout(null), equalTo(555));
+    assertThat("list should have single batch timeout matching global config setting",
+        writerConfig.getAllConfiguredTimeouts(), equalTo(asList(555)));
+    assertThat("batch size should match global config setting", writerConfig.getBatchSize(null),
+        equalTo(12345));
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/test/java/org/apache/metron/common/writer/IndexingWriterConfigurationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/writer/IndexingWriterConfigurationTest.java b/metron-platform/metron-common/src/test/java/org/apache/metron/common/writer/IndexingWriterConfigurationTest.java
deleted file mode 100644
index aec215f..0000000
--- a/metron-platform/metron-common/src/test/java/org/apache/metron/common/writer/IndexingWriterConfigurationTest.java
+++ /dev/null
@@ -1,70 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *     http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.metron.common.writer;
-
-import org.apache.metron.common.configuration.IndexingConfigurations;
-import org.apache.metron.common.configuration.writer.IndexingWriterConfiguration;
-import org.junit.Assert;
-import org.junit.Test;
-
-import java.io.FileInputStream;
-import java.io.FileNotFoundException;
-import java.io.IOException;
-
-import static org.apache.metron.test.bolt.BaseEnrichmentBoltTest.sampleSensorIndexingConfigPath;
-import static org.apache.metron.test.bolt.BaseEnrichmentBoltTest.sensorType;
-
-public class IndexingWriterConfigurationTest {
-  @Test
-  public void testDefaultBatchSize() {
-    IndexingWriterConfiguration config = new IndexingWriterConfiguration("hdfs",
-           new IndexingConfigurations()
-    );
-    Assert.assertEquals(1, config.getBatchSize("foo"));
-  }
-  @Test
-  public void testDefaultBatchTimeout() {
-    IndexingWriterConfiguration config = new IndexingWriterConfiguration("hdfs",
-           new IndexingConfigurations()
-    );
-    Assert.assertEquals(0, config.getBatchTimeout("foo"));
-  }
-  @Test
-  public void testGetAllConfiguredTimeouts() throws FileNotFoundException, IOException {
-    //default
-    IndexingWriterConfiguration config = new IndexingWriterConfiguration("hdfs",
-            new IndexingConfigurations()
-    );
-    Assert.assertEquals(0, config.getAllConfiguredTimeouts().size());
-    //non-default
-    IndexingConfigurations iconfigs = new IndexingConfigurations();
-    iconfigs.updateSensorIndexingConfig(
-            sensorType, new FileInputStream(sampleSensorIndexingConfigPath));
-    config = new IndexingWriterConfiguration("elasticsearch", iconfigs);
-    Assert.assertEquals(1, config.getAllConfiguredTimeouts().size());
-    Assert.assertEquals(7, (long)config.getAllConfiguredTimeouts().get(0));
-  }
-  @Test
-  public void testDefaultIndex() {
-    IndexingWriterConfiguration config = new IndexingWriterConfiguration("hdfs",
-           new IndexingConfigurations()
-    );
-    Assert.assertEquals("foo", config.getIndex("foo"));
-  }
-}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-common/src/test/java/org/apache/metron/common/writer/ParserWriterConfigurationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/writer/ParserWriterConfigurationTest.java b/metron-platform/metron-common/src/test/java/org/apache/metron/common/writer/ParserWriterConfigurationTest.java
deleted file mode 100644
index e960e47..0000000
--- a/metron-platform/metron-common/src/test/java/org/apache/metron/common/writer/ParserWriterConfigurationTest.java
+++ /dev/null
@@ -1,38 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *     http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.metron.common.writer;
-
-import org.apache.metron.common.configuration.ParserConfigurations;
-import org.apache.metron.common.configuration.writer.ParserWriterConfiguration;
-import org.junit.Assert;
-import org.junit.Test;
-
-public class ParserWriterConfigurationTest {
-  @Test
-  public void testDefaultBatchSize() {
-    ParserWriterConfiguration config = new ParserWriterConfiguration( new ParserConfigurations() );
-    Assert.assertEquals(1, config.getBatchSize("foo"));
-  }
-
-  @Test
-  public void testDefaultIndex() {
-    ParserWriterConfiguration config = new ParserWriterConfiguration( new ParserConfigurations() );
-    Assert.assertEquals("foo", config.getIndex("foo"));
-  }
-}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-enrichment/README.md
----------------------------------------------------------------------
diff --git a/metron-platform/metron-enrichment/README.md b/metron-platform/metron-enrichment/README.md
index cbf8ee8..8a53e71 100644
--- a/metron-platform/metron-enrichment/README.md
+++ b/metron-platform/metron-enrichment/README.md
@@ -89,7 +89,8 @@ There are two types of configurations at the moment, `global` and
 ## Global Configuration 
 
 There are a few enrichments which have independent configurations, such
-as from the global config.
+as from the global config. You can also configure the enrichment topology's
+writer batching settings.
 
 Also, see the "[Global Configuration](../metron-common)" section for
 more discussion of the global config.
@@ -107,6 +108,18 @@ the topology and used from there. This is lazy, so if this property
 changes in a running topology, the file will be localized from HDFS upon first
 time the file is used via the geo enrichment. 
 
+### Writer Batching
+
+#### `enrichment.writer.batchSize`
+
+The size of the batch that is written to Kafka at once. Defaults to `15` (size of 1 disables batching).
+
+#### `enrichment.writer.batchTimeout`
+
+The timeout after which a batch will be flushed even if batchSize has not been met.  Optional.
+If unspecified, or set to `0`, it defaults to a system-determined duration which is a fraction of the Storm 
+parameter `topology.message.timeout.secs`.  Ignored if batchSize is `1`, since this disables batching.
+
 ## Sensor Enrichment Configuration
 
 The sensor specific configuration is intended to configure the

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-enrichment/src/main/flux/enrichment/remote-splitjoin.yaml
----------------------------------------------------------------------
diff --git a/metron-platform/metron-enrichment/src/main/flux/enrichment/remote-splitjoin.yaml b/metron-platform/metron-enrichment/src/main/flux/enrichment/remote-splitjoin.yaml
index fd7ceff..8a1bba1 100644
--- a/metron-platform/metron-enrichment/src/main/flux/enrichment/remote-splitjoin.yaml
+++ b/metron-platform/metron-enrichment/src/main/flux/enrichment/remote-splitjoin.yaml
@@ -328,8 +328,9 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "ENRICHMENT"
         configMethods:
-            -   name: "withMessageWriter"
+            -   name: "withBulkMessageWriter"
                 args:
                     - ref: "enrichmentErrorKafkaWriter"
 
@@ -388,8 +389,9 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "ENRICHMENT"
         configMethods:
-            -   name: "withMessageWriter"
+            -   name: "withBulkMessageWriter"
                 args:
                     - ref: "threatIntelErrorKafkaWriter"
 
@@ -398,8 +400,9 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "ENRICHMENT"
         configMethods:
-            -   name: "withMessageWriter"
+            -   name: "withBulkMessageWriter"
                 args:
                     - ref: "kafkaWriter"
         parallelism: ${kafka.writer.parallelism}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-enrichment/src/main/flux/enrichment/remote-unified.yaml
----------------------------------------------------------------------
diff --git a/metron-platform/metron-enrichment/src/main/flux/enrichment/remote-unified.yaml b/metron-platform/metron-enrichment/src/main/flux/enrichment/remote-unified.yaml
index d7107d9..45b05cf 100644
--- a/metron-platform/metron-enrichment/src/main/flux/enrichment/remote-unified.yaml
+++ b/metron-platform/metron-enrichment/src/main/flux/enrichment/remote-unified.yaml
@@ -295,8 +295,9 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "ENRICHMENT"
         configMethods:
-            -   name: "withMessageWriter"
+            -   name: "withBulkMessageWriter"
                 args:
                     - ref: "enrichmentErrorKafkaWriter"
 
@@ -329,8 +330,9 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "ENRICHMENT"
         configMethods:
-            -   name: "withMessageWriter"
+            -   name: "withBulkMessageWriter"
                 args:
                     - ref: "threatIntelErrorKafkaWriter"
 
@@ -339,8 +341,9 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "ENRICHMENT"
         configMethods:
-            -   name: "withMessageWriter"
+            -   name: "withBulkMessageWriter"
                 args:
                     - ref: "kafkaWriter"
         parallelism: ${kafka.writer.parallelism}

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/bolt/BulkMessageWriterBoltTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/bolt/BulkMessageWriterBoltTest.java b/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/bolt/BulkMessageWriterBoltTest.java
index 52516ac..588fc58 100644
--- a/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/bolt/BulkMessageWriterBoltTest.java
+++ b/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/bolt/BulkMessageWriterBoltTest.java
@@ -17,9 +17,27 @@
  */
 package org.apache.metron.enrichment.bolt;
 
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.argThat;
+import static org.mockito.Matchers.eq;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.reset;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.io.FileInputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
 import org.adrianwalker.multilinestring.Multiline;
 import org.apache.log4j.Level;
 import org.apache.metron.common.Constants;
+import org.apache.metron.common.configuration.IndexingConfigurations;
 import org.apache.metron.common.configuration.writer.WriterConfiguration;
 import org.apache.metron.common.message.MessageGetStrategy;
 import org.apache.metron.common.message.MessageGetters;
@@ -43,19 +61,6 @@ import org.mockito.ArgumentMatcher;
 import org.mockito.Matchers;
 import org.mockito.Mock;
 
-import java.io.FileInputStream;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.fail;
-import static org.mockito.Matchers.any;
-import static org.mockito.Matchers.argThat;
-import static org.mockito.Matchers.eq;
-import static org.mockito.Mockito.*;
-
 public class BulkMessageWriterBoltTest extends BaseEnrichmentBoltTest {
 
   protected class MessageListMatcher extends ArgumentMatcher<List<JSONObject>> {
@@ -119,9 +124,9 @@ public class BulkMessageWriterBoltTest extends BaseEnrichmentBoltTest {
 
   @Test
   public void testSourceTypeMissing() throws Exception {
-
     // setup the bolt
-    BulkMessageWriterBolt bulkMessageWriterBolt = new BulkMessageWriterBolt("zookeeperUrl")
+    BulkMessageWriterBolt<IndexingConfigurations> bulkMessageWriterBolt = new BulkMessageWriterBolt<IndexingConfigurations>(
+          "zookeeperUrl", "INDEXING")
             .withBulkMessageWriter(bulkMessageWriter)
             .withMessageGetter(MessageGetters.JSON_FROM_FIELD.name())
             .withMessageGetterField("message");
@@ -148,9 +153,11 @@ public class BulkMessageWriterBoltTest extends BaseEnrichmentBoltTest {
 
   @Test
   public void testFlushOnBatchSize() throws Exception {
-    BulkMessageWriterBolt bulkMessageWriterBolt = new BulkMessageWriterBolt("zookeeperUrl")
-            .withBulkMessageWriter(bulkMessageWriter).withMessageGetter(MessageGetters.JSON_FROM_FIELD.name())
-            .withMessageGetterField("message");
+    BulkMessageWriterBolt<IndexingConfigurations> bulkMessageWriterBolt = new BulkMessageWriterBolt<IndexingConfigurations>(
+        "zookeeperUrl", "INDEXING")
+        .withBulkMessageWriter(bulkMessageWriter)
+        .withMessageGetter(MessageGetters.JSON_FROM_FIELD.name())
+        .withMessageGetterField("message");
     bulkMessageWriterBolt.setCuratorFramework(client);
     bulkMessageWriterBolt.setZKCache(cache);
     bulkMessageWriterBolt.getConfigurations().updateSensorIndexingConfig(sensorType,
@@ -207,9 +214,12 @@ public class BulkMessageWriterBoltTest extends BaseEnrichmentBoltTest {
   @Test
   public void testFlushOnBatchTimeout() throws Exception {
     FakeClock clock = new FakeClock();
-    BulkMessageWriterBolt bulkMessageWriterBolt = new BulkMessageWriterBolt("zookeeperUrl")
-            .withBulkMessageWriter(bulkMessageWriter).withMessageGetter(MessageGetters.JSON_FROM_FIELD.name())
-            .withMessageGetterField("message").withBatchTimeoutDivisor(3);
+    BulkMessageWriterBolt<IndexingConfigurations> bulkMessageWriterBolt = new BulkMessageWriterBolt<IndexingConfigurations>(
+        "zookeeperUrl", "INDEXING")
+        .withBulkMessageWriter(bulkMessageWriter)
+        .withMessageGetter(MessageGetters.JSON_FROM_FIELD.name())
+        .withMessageGetterField("message")
+        .withBatchTimeoutDivisor(3);
     bulkMessageWriterBolt.setCuratorFramework(client);
     bulkMessageWriterBolt.setZKCache(cache);
     bulkMessageWriterBolt.getConfigurations().updateSensorIndexingConfig(sensorType,
@@ -251,9 +261,11 @@ public class BulkMessageWriterBoltTest extends BaseEnrichmentBoltTest {
   @Test
   public void testFlushOnTickTuple() throws Exception {
     FakeClock clock = new FakeClock();
-    BulkMessageWriterBolt bulkMessageWriterBolt = new BulkMessageWriterBolt("zookeeperUrl")
-            .withBulkMessageWriter(bulkMessageWriter).withMessageGetter(MessageGetters.JSON_FROM_FIELD.name())
-            .withMessageGetterField("message");
+    BulkMessageWriterBolt<IndexingConfigurations> bulkMessageWriterBolt = new BulkMessageWriterBolt<IndexingConfigurations>(
+        "zookeeperUrl", "INDEXING")
+        .withBulkMessageWriter(bulkMessageWriter)
+        .withMessageGetter(MessageGetters.JSON_FROM_FIELD.name())
+        .withMessageGetterField("message");
     bulkMessageWriterBolt.setCuratorFramework(client);
     bulkMessageWriterBolt.setZKCache(cache);
     bulkMessageWriterBolt.getConfigurations().updateSensorIndexingConfig(sensorType
@@ -309,7 +321,8 @@ public class BulkMessageWriterBoltTest extends BaseEnrichmentBoltTest {
     FakeClock clock = new FakeClock();
 
     // setup the bolt
-    BulkMessageWriterBolt bolt = new BulkMessageWriterBolt("zookeeperUrl")
+    BulkMessageWriterBolt<IndexingConfigurations> bolt = new BulkMessageWriterBolt<IndexingConfigurations>(
+        "zookeeperUrl", "INDEXING")
             .withBulkMessageWriter(bulkMessageWriter)
             .withMessageGetter(MessageGetters.JSON_FROM_POSITION.name())
             .withMessageGetterField("message");
@@ -331,4 +344,5 @@ public class BulkMessageWriterBoltTest extends BaseEnrichmentBoltTest {
     verify(outputCollector, times(1)).emit(eq(Constants.ERROR_STREAM), any());
     verify(outputCollector, times(1)).ack(tuple);
   }
+
 }


[40/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/js/apache-maven-fluido-1.3.0.min.js
----------------------------------------------------------------------
diff --git a/site/current-book/js/apache-maven-fluido-1.3.0.min.js b/site/current-book/js/apache-maven-fluido-1.3.0.min.js
deleted file mode 100644
index 677e611..0000000
--- a/site/current-book/js/apache-maven-fluido-1.3.0.min.js
+++ /dev/null
@@ -1,21 +0,0 @@
-/*!
- * jQuery JavaScript Library v1.8.0
- * http://jquery.com/
- *
- * Includes Sizzle.js
- * http://sizzlejs.com/
- *
- * Copyright 2012 jQuery Foundation and other contributors
- * Released under the MIT license
- * http://jquery.org/license
- *
- * Date: Thu Aug 09 2012 16:24:48 GMT-0400 (Eastern Daylight Time)
- */
-(function(a2,aB){var x,af,o=a2.document,aI=a2.location,d=a2.navigator,bg=a2.jQuery,I=a2.$,am=Array.prototype.push,a4=Array.prototype.slice,aK=Array.prototype.indexOf,A=Object.prototype.toString,V=Object.prototype.hasOwnProperty,aN=String.prototype.trim,bG=function(e,bZ){return new bG.fn.init(e,bZ,x)},bx=/[\-+]?(?:\d*\.|)\d+(?:[eE][\-+]?\d+|)/.source,aa=/\S/,aV=/\s+/,D=aa.test("\xA0")?(/^[\s\xA0]+|[\s\xA0]+$/g):/^\s+|\s+$/g,bo=/^(?:[^#<]*(<[\w\W]+>)[^>]*$|#([\w\-]*)$)/,a=/^<(\w+)\s*\/?>(?:<\/\1>|)$/,bf=/^[\],:{}\s]*$/,bi=/(?:^|:|,)(?:\s*\[)+/g,bD=/\\(?:["\\\/bfnrt]|u[\da-fA-F]{4})/g,a0=/"[^"\\\r\n]*"|true|false|null|-?(?:\d\d*\.|)\d+(?:[eE][\-+]?\d+|)/g,bP=/^-ms-/,aU=/-([\da-z])/gi,N=function(e,bZ){return(bZ+"").toUpperCase()},aF=function(){if(o.addEventListener){o.removeEventListener("DOMContentLoaded",aF,false);bG.ready()}else{if(o.readyState==="complete"){o.detachEvent("onreadystatechange",aF);bG.ready()}}},Z={};bG.fn=bG.prototype={constructor:bG,init:function(e,b2,b1){var b0,b3,b
 Z,b4;if(!e){return this}if(e.nodeType){this.context=this[0]=e;this.length=1;return this}if(typeof e==="string"){if(e.charAt(0)==="<"&&e.charAt(e.length-1)===">"&&e.length>=3){b0=[null,e,null]}else{b0=bo.exec(e)}if(b0&&(b0[1]||!b2)){if(b0[1]){b2=b2 instanceof bG?b2[0]:b2;b4=(b2&&b2.nodeType?b2.ownerDocument||b2:o);e=bG.parseHTML(b0[1],b4,true);if(a.test(b0[1])&&bG.isPlainObject(b2)){this.attr.call(e,b2,true)}return bG.merge(this,e)}else{b3=o.getElementById(b0[2]);if(b3&&b3.parentNode){if(b3.id!==b0[2]){return b1.find(e)}this.length=1;this[0]=b3}this.context=o;this.selector=e;return this}}else{if(!b2||b2.jquery){return(b2||b1).find(e)}else{return this.constructor(b2).find(e)}}}else{if(bG.isFunction(e)){return b1.ready(e)}}if(e.selector!==aB){this.selector=e.selector;this.context=e.context}return bG.makeArray(e,this)},selector:"",jquery:"1.8.0",length:0,size:function(){return this.length},toArray:function(){return a4.call(this)},get:function(e){return e==null?this.toArray():(e<0?this[t
 his.length+e]:this[e])},pushStack:function(bZ,b1,e){var b0=bG.merge(this.constructor(),bZ);b0.prevObject=this;b0.context=this.context;if(b1==="find"){b0.selector=this.selector+(this.selector?" ":"")+e}else{if(b1){b0.selector=this.selector+"."+b1+"("+e+")"}}return b0},each:function(bZ,e){return bG.each(this,bZ,e)},ready:function(e){bG.ready.promise().done(e);return this},eq:function(e){e=+e;return e===-1?this.slice(e):this.slice(e,e+1)},first:function(){return this.eq(0)},last:function(){return this.eq(-1)},slice:function(){return this.pushStack(a4.apply(this,arguments),"slice",a4.call(arguments).join(","))},map:function(e){return this.pushStack(bG.map(this,function(b0,bZ){return e.call(b0,bZ,b0)}))},end:function(){return this.prevObject||this.constructor(null)},push:am,sort:[].sort,splice:[].splice};bG.fn.init.prototype=bG.fn;bG.extend=bG.fn.extend=function(){var b7,b0,e,bZ,b4,b5,b3=arguments[0]||{},b2=1,b1=arguments.length,b6=false;if(typeof b3==="boolean"){b6=b3;b3=arguments[1]||{
 };b2=2}if(typeof b3!=="object"&&!bG.isFunction(b3)){b3={}}if(b1===b2){b3=this;--b2}for(;b2<b1;b2++){if((b7=arguments[b2])!=null){for(b0 in b7){e=b3[b0];bZ=b7[b0];if(b3===bZ){continue}if(b6&&bZ&&(bG.isPlainObject(bZ)||(b4=bG.isArray(bZ)))){if(b4){b4=false;b5=e&&bG.isArray(e)?e:[]}else{b5=e&&bG.isPlainObject(e)?e:{}}b3[b0]=bG.extend(b6,b5,bZ)}else{if(bZ!==aB){b3[b0]=bZ}}}}}return b3};bG.extend({noConflict:function(e){if(a2.$===bG){a2.$=I}if(e&&a2.jQuery===bG){a2.jQuery=bg}return bG},isReady:false,readyWait:1,holdReady:function(e){if(e){bG.readyWait++}else{bG.ready(true)}},ready:function(e){if(e===true?--bG.readyWait:bG.isReady){return}if(!o.body){return setTimeout(bG.ready,1)}bG.isReady=true;if(e!==true&&--bG.readyWait>0){return}af.resolveWith(o,[bG]);if(bG.fn.trigger){bG(o).trigger("ready").off("ready")}},isFunction:function(e){return bG.type(e)==="function"},isArray:Array.isArray||function(e){return bG.type(e)==="array"},isWindow:function(e){return e!=null&&e==e.window},isNumeric:fu
 nction(e){return !isNaN(parseFloat(e))&&isFinite(e)},type:function(e){return e==null?String(e):Z[A.call(e)]||"object"},isPlainObject:function(b1){if(!b1||bG.type(b1)!=="object"||b1.nodeType||bG.isWindow(b1)){return false}try{if(b1.constructor&&!V.call(b1,"constructor")&&!V.call(b1.constructor.prototype,"isPrototypeOf")){return false}}catch(b0){return false}var bZ;for(bZ in b1){}return bZ===aB||V.call(b1,bZ)},isEmptyObject:function(bZ){var e;for(e in bZ){return false}return true},error:function(e){throw new Error(e)},parseHTML:function(b1,b0,e){var bZ;if(!b1||typeof b1!=="string"){return null}if(typeof b0==="boolean"){e=b0;b0=0}b0=b0||o;if((bZ=a.exec(b1))){return[b0.createElement(bZ[1])]}bZ=bG.buildFragment([b1],b0,e?null:[]);return bG.merge([],(bZ.cacheable?bG.clone(bZ.fragment):bZ.fragment).childNodes)},parseJSON:function(e){if(!e||typeof e!=="string"){return null}e=bG.trim(e);if(a2.JSON&&a2.JSON.parse){return a2.JSON.parse(e)}if(bf.test(e.replace(bD,"@").replace(a0,"]").replace(bi
 ,""))){return(new Function("return "+e))()}bG.error("Invalid JSON: "+e)},parseXML:function(b1){var bZ,b0;if(!b1||typeof b1!=="string"){return null}try{if(a2.DOMParser){b0=new DOMParser();bZ=b0.parseFromString(b1,"text/xml")}else{bZ=new ActiveXObject("Microsoft.XMLDOM");bZ.async="false";bZ.loadXML(b1)}}catch(b2){bZ=aB}if(!bZ||!bZ.documentElement||bZ.getElementsByTagName("parsererror").length){bG.error("Invalid XML: "+b1)}return bZ},noop:function(){},globalEval:function(e){if(e&&aa.test(e)){(a2.execScript||function(bZ){a2["eval"].call(a2,bZ)})(e)}},camelCase:function(e){return e.replace(bP,"ms-").replace(aU,N)},nodeName:function(bZ,e){return bZ.nodeName&&bZ.nodeName.toUpperCase()===e.toUpperCase()},each:function(b3,b4,b0){var bZ,b1=0,b2=b3.length,e=b2===aB||bG.isFunction(b3);if(b0){if(e){for(bZ in b3){if(b4.apply(b3[bZ],b0)===false){break}}}else{for(;b1<b2;){if(b4.apply(b3[b1++],b0)===false){break}}}}else{if(e){for(bZ in b3){if(b4.call(b3[bZ],bZ,b3[bZ])===false){break}}}else{for(;b1<b
 2;){if(b4.call(b3[b1],b1,b3[b1++])===false){break}}}}return b3},trim:aN?function(e){return e==null?"":aN.call(e)}:function(e){return e==null?"":e.toString().replace(D,"")},makeArray:function(e,b0){var b1,bZ=b0||[];if(e!=null){b1=bG.type(e);if(e.length==null||b1==="string"||b1==="function"||b1==="regexp"||bG.isWindow(e)){am.call(bZ,e)}else{bG.merge(bZ,e)}}return bZ},inArray:function(b1,bZ,b0){var e;if(bZ){if(aK){return aK.call(bZ,b1,b0)}e=bZ.length;b0=b0?b0<0?Math.max(0,e+b0):b0:0;for(;b0<e;b0++){if(b0 in bZ&&bZ[b0]===b1){return b0}}}return -1},merge:function(b2,b0){var e=b0.length,b1=b2.length,bZ=0;if(typeof e==="number"){for(;bZ<e;bZ++){b2[b1++]=b0[bZ]}}else{while(b0[bZ]!==aB){b2[b1++]=b0[bZ++]}}b2.length=b1;return b2},grep:function(bZ,b4,e){var b3,b0=[],b1=0,b2=bZ.length;e=!!e;for(;b1<b2;b1++){b3=!!b4(bZ[b1],b1);if(e!==b3){b0.push(bZ[b1])}}return b0},map:function(e,b5,b6){var b3,b4,b2=[],b0=0,bZ=e.length,b1=e instanceof bG||bZ!==aB&&typeof bZ==="number"&&((bZ>0&&e[0]&&e[bZ-1])||bZ
 ===0||bG.isArray(e));if(b1){for(;b0<bZ;b0++){b3=b5(e[b0],b0,b6);if(b3!=null){b2[b2.length]=b3}}}else{for(b4 in e){b3=b5(e[b4],b4,b6);if(b3!=null){b2[b2.length]=b3}}}return b2.concat.apply([],b2)},guid:1,proxy:function(b2,b1){var b0,e,bZ;if(typeof b1==="string"){b0=b2[b1];b1=b2;b2=b0}if(!bG.isFunction(b2)){return aB}e=a4.call(arguments,2);bZ=function(){return b2.apply(b1,e.concat(a4.call(arguments)))};bZ.guid=b2.guid=b2.guid||bZ.guid||bG.guid++;return bZ},access:function(e,b4,b7,b5,b2,b8,b6){var b0,b3=b7==null,b1=0,bZ=e.length;if(b7&&typeof b7==="object"){for(b1 in b7){bG.access(e,b4,b1,b7[b1],1,b8,b5)}b2=1}else{if(b5!==aB){b0=b6===aB&&bG.isFunction(b5);if(b3){if(b0){b0=b4;b4=function(ca,b9,cb){return b0.call(bG(ca),cb)}}else{b4.call(e,b5);b4=null}}if(b4){for(;b1<bZ;b1++){b4(e[b1],b7,b0?b5.call(e[b1],b1,b4(e[b1],b7)):b5,b6)}}b2=1}}return b2?e:b3?b4.call(e):bZ?b4(e[0],b7):b8},now:function(){return(new Date()).getTime()}});bG.ready.promise=function(b2){if(!af){af=bG.Deferred();if(o.rea
 dyState==="complete"||(o.readyState!=="loading"&&o.addEventListener)){setTimeout(bG.ready,1)}else{if(o.addEventListener){o.addEventListener("DOMContentLoaded",aF,false);a2.addEventListener("load",bG.ready,false)}else{o.attachEvent("onreadystatechange",aF);a2.attachEvent("onload",bG.ready);var b1=false;try{b1=a2.frameElement==null&&o.documentElement}catch(b0){}if(b1&&b1.doScroll){(function bZ(){if(!bG.isReady){try{b1.doScroll("left")}catch(b3){return setTimeout(bZ,50)}bG.ready()}})()}}}}return af.promise(b2)};bG.each("Boolean Number String Function Array Date RegExp Object".split(" "),function(bZ,e){Z["[object "+e+"]"]=e.toLowerCase()});x=bG(o);var bU={};function ac(bZ){var e=bU[bZ]={};bG.each(bZ.split(aV),function(b1,b0){e[b0]=true});return e}bG.Callbacks=function(b8){b8=typeof b8==="string"?(bU[b8]||ac(b8)):bG.extend({},b8);var b1,e,b2,b0,b3,b4,b5=[],b6=!b8.once&&[],bZ=function(b9){b1=b8.memory&&b9;e=true;b4=b0||0;b0=0;b3=b5.length;b2=true;for(;b5&&b4<b3;b4++){if(b5[b4].apply(b9[0]
 ,b9[1])===false&&b8.stopOnFalse){b1=false;break}}b2=false;if(b5){if(b6){if(b6.length){bZ(b6.shift())}}else{if(b1){b5=[]}else{b7.disable()}}}},b7={add:function(){if(b5){var ca=b5.length;(function b9(cb){bG.each(cb,function(cd,cc){if(bG.isFunction(cc)&&(!b8.unique||!b7.has(cc))){b5.push(cc)}else{if(cc&&cc.length){b9(cc)}}})})(arguments);if(b2){b3=b5.length}else{if(b1){b0=ca;bZ(b1)}}}return this},remove:function(){if(b5){bG.each(arguments,function(cb,b9){var ca;while((ca=bG.inArray(b9,b5,ca))>-1){b5.splice(ca,1);if(b2){if(ca<=b3){b3--}if(ca<=b4){b4--}}}})}return this},has:function(b9){return bG.inArray(b9,b5)>-1},empty:function(){b5=[];return this},disable:function(){b5=b6=b1=aB;return this},disabled:function(){return !b5},lock:function(){b6=aB;if(!b1){b7.disable()}return this},locked:function(){return !b6},fireWith:function(ca,b9){b9=b9||[];b9=[ca,b9.slice?b9.slice():b9];if(b5&&(!e||b6)){if(b2){b6.push(b9)}else{bZ(b9)}}return this},fire:function(){b7.fireWith(this,arguments);return th
 is},fired:function(){return !!e}};return b7};bG.extend({Deferred:function(b0){var bZ=[["resolve","done",bG.Callbacks("once memory"),"resolved"],["reject","fail",bG.Callbacks("once memory"),"rejected"],["notify","progress",bG.Callbacks("memory")]],b1="pending",b2={state:function(){return b1},always:function(){e.done(arguments).fail(arguments);return this},then:function(){var b3=arguments;return bG.Deferred(function(b4){bG.each(bZ,function(b6,b5){var b8=b5[0],b7=b3[b6];e[b5[1]](bG.isFunction(b7)?function(){var b9=b7.apply(this,arguments);if(b9&&bG.isFunction(b9.promise)){b9.promise().done(b4.resolve).fail(b4.reject).progress(b4.notify)}else{b4[b8+"With"](this===e?b4:this,[b9])}}:b4[b8])});b3=null}).promise()},promise:function(b3){return typeof b3==="object"?bG.extend(b3,b2):b2}},e={};b2.pipe=b2.then;bG.each(bZ,function(b4,b3){var b6=b3[2],b5=b3[3];b2[b3[1]]=b6.add;if(b5){b6.add(function(){b1=b5},bZ[b4^1][2].disable,bZ[2][2].lock)}e[b3[0]]=b6.fire;e[b3[0]+"With"]=b6.fireWith});b2.promi
 se(e);if(b0){b0.call(e,e)}return e},when:function(b2){var b0=0,b4=a4.call(arguments),e=b4.length,bZ=e!==1||(b2&&bG.isFunction(b2.promise))?e:0,b7=bZ===1?b2:bG.Deferred(),b1=function(b9,ca,b8){return function(cb){ca[b9]=this;b8[b9]=arguments.length>1?a4.call(arguments):cb;if(b8===b6){b7.notifyWith(ca,b8)}else{if(!(--bZ)){b7.resolveWith(ca,b8)}}}},b6,b3,b5;if(e>1){b6=new Array(e);b3=new Array(e);b5=new Array(e);for(;b0<e;b0++){if(b4[b0]&&bG.isFunction(b4[b0].promise)){b4[b0].promise().done(b1(b0,b5,b4)).fail(b7.reject).progress(b1(b0,b3,b6))}else{--bZ}}}if(!bZ){b7.resolveWith(b5,b4)}return b7.promise()}});bG.support=(function(){var cb,ca,b8,b9,b2,b7,b6,b4,b3,b1,bZ,b0=o.createElement("div");b0.setAttribute("className","t");b0.innerHTML="  <link/><table></table><a href='/a'>a</a><input type='checkbox'/>";ca=b0.getElementsByTagName("*");b8=b0.getElementsByTagName("a")[0];b8.style.cssText="top:1px;float:left;opacity:.5";if(!ca||!ca.length||!b8){return{}}b9=o.createElement("select");b2=b9.
 appendChild(o.createElement("option"));b7=b0.getElementsByTagName("input")[0];cb={leadingWhitespace:(b0.firstChild.nodeType===3),tbody:!b0.getElementsByTagName("tbody").length,htmlSerialize:!!b0.getElementsByTagName("link").length,style:/top/.test(b8.getAttribute("style")),hrefNormalized:(b8.getAttribute("href")==="/a"),opacity:/^0.5/.test(b8.style.opacity),cssFloat:!!b8.style.cssFloat,checkOn:(b7.value==="on"),optSelected:b2.selected,getSetAttribute:b0.className!=="t",enctype:!!o.createElement("form").enctype,html5Clone:o.createElement("nav").cloneNode(true).outerHTML!=="<:nav></:nav>",boxModel:(o.compatMode==="CSS1Compat"),submitBubbles:true,changeBubbles:true,focusinBubbles:false,deleteExpando:true,noCloneEvent:true,inlineBlockNeedsLayout:false,shrinkWrapBlocks:false,reliableMarginRight:true,boxSizingReliable:true,pixelPosition:false};b7.checked=true;cb.noCloneChecked=b7.cloneNode(true).checked;b9.disabled=true;cb.optDisabled=!b2.disabled;try{delete b0.test}catch(b5){cb.deleteExp
 ando=false}if(!b0.addEventListener&&b0.attachEvent&&b0.fireEvent){b0.attachEvent("onclick",bZ=function(){cb.noCloneEvent=false});b0.cloneNode(true).fireEvent("onclick");b0.detachEvent("onclick",bZ)}b7=o.createElement("input");b7.value="t";b7.setAttribute("type","radio");cb.radioValue=b7.value==="t";b7.setAttribute("checked","checked");b7.setAttribute("name","t");b0.appendChild(b7);b6=o.createDocumentFragment();b6.appendChild(b0.lastChild);cb.checkClone=b6.cloneNode(true).cloneNode(true).lastChild.checked;cb.appendChecked=b7.checked;b6.removeChild(b7);b6.appendChild(b0);if(b0.attachEvent){for(b3 in {submit:true,change:true,focusin:true}){b4="on"+b3;b1=(b4 in b0);if(!b1){b0.setAttribute(b4,"return;");b1=(typeof b0[b4]==="function")}cb[b3+"Bubbles"]=b1}}bG(function(){var cc,cg,ce,cf,cd="padding:0;margin:0;border:0;display:block;overflow:hidden;",e=o.getElementsByTagName("body")[0];if(!e){return}cc=o.createElement("div");cc.style.cssText="visibility:hidden;border:0;width:0;height:0;posi
 tion:static;top:0;margin-top:1px";e.insertBefore(cc,e.firstChild);cg=o.createElement("div");cc.appendChild(cg);cg.innerHTML="<table><tr><td></td><td>t</td></tr></table>";ce=cg.getElementsByTagName("td");ce[0].style.cssText="padding:0;margin:0;border:0;display:none";b1=(ce[0].offsetHeight===0);ce[0].style.display="";ce[1].style.display="none";cb.reliableHiddenOffsets=b1&&(ce[0].offsetHeight===0);cg.innerHTML="";cg.style.cssText="box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box;padding:1px;border:1px;display:block;width:4px;margin-top:1%;position:absolute;top:1%;";cb.boxSizing=(cg.offsetWidth===4);cb.doesNotIncludeMarginInBodyOffset=(e.offsetTop!==1);if(a2.getComputedStyle){cb.pixelPosition=(a2.getComputedStyle(cg,null)||{}).top!=="1%";cb.boxSizingReliable=(a2.getComputedStyle(cg,null)||{width:"4px"}).width==="4px";cf=o.createElement("div");cf.style.cssText=cg.style.cssText=cd;cf.style.marginRight=cf.style.width="0";cg.style.width="1px";cg.appendChild(cf)
 ;cb.reliableMarginRight=!parseFloat((a2.getComputedStyle(cf,null)||{}).marginRight)}if(typeof cg.style.zoom!=="undefined"){cg.innerHTML="";cg.style.cssText=cd+"width:1px;padding:1px;display:inline;zoom:1";cb.inlineBlockNeedsLayout=(cg.offsetWidth===3);cg.style.display="block";cg.style.overflow="visible";cg.innerHTML="<div></div>";cg.firstChild.style.width="5px";cb.shrinkWrapBlocks=(cg.offsetWidth!==3);cc.style.zoom=1}e.removeChild(cc);cc=cg=ce=cf=null});b6.removeChild(b0);ca=b8=b9=b2=b7=b6=b0=null;return cb})();var bt=/^(?:\{.*\}|\[.*\])$/,aL=/([A-Z])/g;bG.extend({cache:{},deletedIds:[],uuid:0,expando:"jQuery"+(bG.fn.jquery+Math.random()).replace(/\D/g,""),noData:{embed:true,object:"clsid:D27CDB6E-AE6D-11cf-96B8-444553540000",applet:true},hasData:function(e){e=e.nodeType?bG.cache[e[bG.expando]]:e[bG.expando];return !!e&&!O(e)},data:function(b1,bZ,b3,b2){if(!bG.acceptData(b1)){return}var b4,b6,b7=bG.expando,b5=typeof bZ==="string",b8=b1.nodeType,e=b8?bG.cache:b1,b0=b8?b1[b7]:b1[b7]&&
 b7;if((!b0||!e[b0]||(!b2&&!e[b0].data))&&b5&&b3===aB){return}if(!b0){if(b8){b1[b7]=b0=bG.deletedIds.pop()||++bG.uuid}else{b0=b7}}if(!e[b0]){e[b0]={};if(!b8){e[b0].toJSON=bG.noop}}if(typeof bZ==="object"||typeof bZ==="function"){if(b2){e[b0]=bG.extend(e[b0],bZ)}else{e[b0].data=bG.extend(e[b0].data,bZ)}}b4=e[b0];if(!b2){if(!b4.data){b4.data={}}b4=b4.data}if(b3!==aB){b4[bG.camelCase(bZ)]=b3}if(b5){b6=b4[bZ];if(b6==null){b6=b4[bG.camelCase(bZ)]}}else{b6=b4}return b6},removeData:function(b1,bZ,b2){if(!bG.acceptData(b1)){return}var b5,b4,b3,b6=b1.nodeType,e=b6?bG.cache:b1,b0=b6?b1[bG.expando]:bG.expando;if(!e[b0]){return}if(bZ){b5=b2?e[b0]:e[b0].data;if(b5){if(!bG.isArray(bZ)){if(bZ in b5){bZ=[bZ]}else{bZ=bG.camelCase(bZ);if(bZ in b5){bZ=[bZ]}else{bZ=bZ.split(" ")}}}for(b4=0,b3=bZ.length;b4<b3;b4++){delete b5[bZ[b4]]}if(!(b2?O:bG.isEmptyObject)(b5)){return}}}if(!b2){delete e[b0].data;if(!O(e[b0])){return}}if(b6){bG.cleanData([b1],true)}else{if(bG.support.deleteExpando||e!=e.window){delete
  e[b0]}else{e[b0]=null}}},_data:function(bZ,e,b0){return bG.data(bZ,e,b0,true)},acceptData:function(bZ){var e=bZ.nodeName&&bG.noData[bZ.nodeName.toLowerCase()];return !e||e!==true&&bZ.getAttribute("classid")===e}});bG.fn.extend({data:function(b7,b6){var b2,bZ,b5,e,b1,b0=this[0],b4=0,b3=null;if(b7===aB){if(this.length){b3=bG.data(b0);if(b0.nodeType===1&&!bG._data(b0,"parsedAttrs")){b5=b0.attributes;for(b1=b5.length;b4<b1;b4++){e=b5[b4].name;if(e.indexOf("data-")===0){e=bG.camelCase(e.substring(5));bv(b0,e,b3[e])}}bG._data(b0,"parsedAttrs",true)}}return b3}if(typeof b7==="object"){return this.each(function(){bG.data(this,b7)})}b2=b7.split(".",2);b2[1]=b2[1]?"."+b2[1]:"";bZ=b2[1]+"!";return bG.access(this,function(b8){if(b8===aB){b3=this.triggerHandler("getData"+bZ,[b2[0]]);if(b3===aB&&b0){b3=bG.data(b0,b7);b3=bv(b0,b7,b3)}return b3===aB&&b2[1]?this.data(b2[0]):b3}b2[1]=b8;this.each(function(){var b9=bG(this);b9.triggerHandler("setData"+bZ,b2);bG.data(this,b7,b8);b9.triggerHandler("cha
 ngeData"+bZ,b2)})},null,b6,arguments.length>1,null,false)},removeData:function(e){return this.each(function(){bG.removeData(this,e)})}});function bv(b1,b0,b2){if(b2===aB&&b1.nodeType===1){var bZ="data-"+b0.replace(aL,"-$1").toLowerCase();b2=b1.getAttribute(bZ);if(typeof b2==="string"){try{b2=b2==="true"?true:b2==="false"?false:b2==="null"?null:+b2+""===b2?+b2:bt.test(b2)?bG.parseJSON(b2):b2}catch(b3){}bG.data(b1,b0,b2)}else{b2=aB}}return b2}function O(bZ){var e;for(e in bZ){if(e==="data"&&bG.isEmptyObject(bZ[e])){continue}if(e!=="toJSON"){return false}}return true}bG.extend({queue:function(b0,bZ,b1){var e;if(b0){bZ=(bZ||"fx")+"queue";e=bG._data(b0,bZ);if(b1){if(!e||bG.isArray(b1)){e=bG._data(b0,bZ,bG.makeArray(b1))}else{e.push(b1)}}return e||[]}},dequeue:function(b3,b2){b2=b2||"fx";var bZ=bG.queue(b3,b2),b1=bZ.shift(),e=bG._queueHooks(b3,b2),b0=function(){bG.dequeue(b3,b2)};if(b1==="inprogress"){b1=bZ.shift()}if(b1){if(b2==="fx"){bZ.unshift("inprogress")}delete e.stop;b1.call(b3,b0,
 e)}if(!bZ.length&&e){e.empty.fire()}},_queueHooks:function(b0,bZ){var e=bZ+"queueHooks";return bG._data(b0,e)||bG._data(b0,e,{empty:bG.Callbacks("once memory").add(function(){bG.removeData(b0,bZ+"queue",true);bG.removeData(b0,e,true)})})}});bG.fn.extend({queue:function(e,bZ){var b0=2;if(typeof e!=="string"){bZ=e;e="fx";b0--}if(arguments.length<b0){return bG.queue(this[0],e)}return bZ===aB?this:this.each(function(){var b1=bG.queue(this,e,bZ);bG._queueHooks(this,e);if(e==="fx"&&b1[0]!=="inprogress"){bG.dequeue(this,e)}})},dequeue:function(e){return this.each(function(){bG.dequeue(this,e)})},delay:function(bZ,e){bZ=bG.fx?bG.fx.speeds[bZ]||bZ:bZ;e=e||"fx";return this.queue(e,function(b1,b0){var b2=setTimeout(b1,bZ);b0.stop=function(){clearTimeout(b2)}})},clearQueue:function(e){return this.queue(e||"fx",[])},promise:function(b0,b4){var bZ,b1=1,b5=bG.Deferred(),b3=this,e=this.length,b2=function(){if(!(--b1)){b5.resolveWith(b3,[b3])}};if(typeof b0!=="string"){b4=b0;b0=aB}b0=b0||"fx";while(
 e--){if((bZ=bG._data(b3[e],b0+"queueHooks"))&&bZ.empty){b1++;bZ.empty.add(b2)}}b2();return b5.promise(b4)}});var a7,bV,n,bJ=/[\t\r\n]/g,ai=/\r/g,j=/^(?:button|input)$/i,aA=/^(?:button|input|object|select|textarea)$/i,E=/^a(?:rea|)$/i,M=/^(?:autofocus|autoplay|async|checked|controls|defer|disabled|hidden|loop|multiple|open|readonly|required|scoped|selected)$/i,bL=bG.support.getSetAttribute;bG.fn.extend({attr:function(e,bZ){return bG.access(this,bG.attr,e,bZ,arguments.length>1)},removeAttr:function(e){return this.each(function(){bG.removeAttr(this,e)})},prop:function(e,bZ){return bG.access(this,bG.prop,e,bZ,arguments.length>1)},removeProp:function(e){e=bG.propFix[e]||e;return this.each(function(){try{this[e]=aB;delete this[e]}catch(bZ){}})},addClass:function(b2){var b4,b0,bZ,b1,b3,b5,e;if(bG.isFunction(b2)){return this.each(function(b6){bG(this).addClass(b2.call(this,b6,this.className))})}if(b2&&typeof b2==="string"){b4=b2.split(aV);for(b0=0,bZ=this.length;b0<bZ;b0++){b1=this[b0];if(b
 1.nodeType===1){if(!b1.className&&b4.length===1){b1.className=b2}else{b3=" "+b1.className+" ";for(b5=0,e=b4.length;b5<e;b5++){if(!~b3.indexOf(" "+b4[b5]+" ")){b3+=b4[b5]+" "}}b1.className=bG.trim(b3)}}}}return this},removeClass:function(b4){var b1,b2,b3,b5,bZ,b0,e;if(bG.isFunction(b4)){return this.each(function(b6){bG(this).removeClass(b4.call(this,b6,this.className))})}if((b4&&typeof b4==="string")||b4===aB){b1=(b4||"").split(aV);for(b0=0,e=this.length;b0<e;b0++){b3=this[b0];if(b3.nodeType===1&&b3.className){b2=(" "+b3.className+" ").replace(bJ," ");for(b5=0,bZ=b1.length;b5<bZ;b5++){while(b2.indexOf(" "+b1[b5]+" ")>-1){b2=b2.replace(" "+b1[b5]+" "," ")}}b3.className=b4?bG.trim(b2):""}}}return this},toggleClass:function(b1,bZ){var b0=typeof b1,e=typeof bZ==="boolean";if(bG.isFunction(b1)){return this.each(function(b2){bG(this).toggleClass(b1.call(this,b2,this.className,bZ),bZ)})}return this.each(function(){if(b0==="string"){var b4,b3=0,b2=bG(this),b5=bZ,b6=b1.split(aV);while((b4=b6[
 b3++])){b5=e?b5:!b2.hasClass(b4);b2[b5?"addClass":"removeClass"](b4)}}else{if(b0==="undefined"||b0==="boolean"){if(this.className){bG._data(this,"__className__",this.className)}this.className=this.className||b1===false?"":bG._data(this,"__className__")||""}}})},hasClass:function(e){var b1=" "+e+" ",b0=0,bZ=this.length;for(;b0<bZ;b0++){if(this[b0].nodeType===1&&(" "+this[b0].className+" ").replace(bJ," ").indexOf(b1)>-1){return true}}return false},val:function(b1){var e,bZ,b2,b0=this[0];if(!arguments.length){if(b0){e=bG.valHooks[b0.type]||bG.valHooks[b0.nodeName.toLowerCase()];if(e&&"get" in e&&(bZ=e.get(b0,"value"))!==aB){return bZ}bZ=b0.value;return typeof bZ==="string"?bZ.replace(ai,""):bZ==null?"":bZ}return}b2=bG.isFunction(b1);return this.each(function(b4){var b5,b3=bG(this);if(this.nodeType!==1){return}if(b2){b5=b1.call(this,b4,b3.val())}else{b5=b1}if(b5==null){b5=""}else{if(typeof b5==="number"){b5+=""}else{if(bG.isArray(b5)){b5=bG.map(b5,function(b6){return b6==null?"":b6+""}
 )}}}e=bG.valHooks[this.type]||bG.valHooks[this.nodeName.toLowerCase()];if(!e||!("set" in e)||e.set(this,b5,"value")===aB){this.value=b5}})}});bG.extend({valHooks:{option:{get:function(e){var bZ=e.attributes.value;return !bZ||bZ.specified?e.value:e.text}},select:{get:function(e){var b4,bZ,b3,b1,b2=e.selectedIndex,b5=[],b6=e.options,b0=e.type==="select-one";if(b2<0){return null}bZ=b0?b2:0;b3=b0?b2+1:b6.length;for(;bZ<b3;bZ++){b1=b6[bZ];if(b1.selected&&(bG.support.optDisabled?!b1.disabled:b1.getAttribute("disabled")===null)&&(!b1.parentNode.disabled||!bG.nodeName(b1.parentNode,"optgroup"))){b4=bG(b1).val();if(b0){return b4}b5.push(b4)}}if(b0&&!b5.length&&b6.length){return bG(b6[b2]).val()}return b5},set:function(bZ,b0){var e=bG.makeArray(b0);bG(bZ).find("option").each(function(){this.selected=bG.inArray(bG(this).val(),e)>=0});if(!e.length){bZ.selectedIndex=-1}return e}}},attrFn:{},attr:function(b4,b1,b5,b3){var b0,e,b2,bZ=b4.nodeType;if(!b4||bZ===3||bZ===8||bZ===2){return}if(b3&&bG.isF
 unction(bG.fn[b1])){return bG(b4)[b1](b5)}if(typeof b4.getAttribute==="undefined"){return bG.prop(b4,b1,b5)}b2=bZ!==1||!bG.isXMLDoc(b4);if(b2){b1=b1.toLowerCase();e=bG.attrHooks[b1]||(M.test(b1)?bV:a7)}if(b5!==aB){if(b5===null){bG.removeAttr(b4,b1);return}else{if(e&&"set" in e&&b2&&(b0=e.set(b4,b5,b1))!==aB){return b0}else{b4.setAttribute(b1,""+b5);return b5}}}else{if(e&&"get" in e&&b2&&(b0=e.get(b4,b1))!==null){return b0}else{b0=b4.getAttribute(b1);return b0===null?aB:b0}}},removeAttr:function(b1,b3){var b2,b4,bZ,e,b0=0;if(b3&&b1.nodeType===1){b4=b3.split(aV);for(;b0<b4.length;b0++){bZ=b4[b0];if(bZ){b2=bG.propFix[bZ]||bZ;e=M.test(bZ);if(!e){bG.attr(b1,bZ,"")}b1.removeAttribute(bL?bZ:b2);if(e&&b2 in b1){b1[b2]=false}}}}},attrHooks:{type:{set:function(e,bZ){if(j.test(e.nodeName)&&e.parentNode){bG.error("type property can't be changed")}else{if(!bG.support.radioValue&&bZ==="radio"&&bG.nodeName(e,"input")){var b0=e.value;e.setAttribute("type",bZ);if(b0){e.value=b0}return bZ}}}},value:{
 get:function(bZ,e){if(a7&&bG.nodeName(bZ,"button")){return a7.get(bZ,e)}return e in bZ?bZ.value:null},set:function(bZ,b0,e){if(a7&&bG.nodeName(bZ,"button")){return a7.set(bZ,b0,e)}bZ.value=b0}}},propFix:{tabindex:"tabIndex",readonly:"readOnly","for":"htmlFor","class":"className",maxlength:"maxLength",cellspacing:"cellSpacing",cellpadding:"cellPadding",rowspan:"rowSpan",colspan:"colSpan",usemap:"useMap",frameborder:"frameBorder",contenteditable:"contentEditable"},prop:function(b3,b1,b4){var b0,e,b2,bZ=b3.nodeType;if(!b3||bZ===3||bZ===8||bZ===2){return}b2=bZ!==1||!bG.isXMLDoc(b3);if(b2){b1=bG.propFix[b1]||b1;e=bG.propHooks[b1]}if(b4!==aB){if(e&&"set" in e&&(b0=e.set(b3,b4,b1))!==aB){return b0}else{return(b3[b1]=b4)}}else{if(e&&"get" in e&&(b0=e.get(b3,b1))!==null){return b0}else{return b3[b1]}}},propHooks:{tabIndex:{get:function(bZ){var e=bZ.getAttributeNode("tabindex");return e&&e.specified?parseInt(e.value,10):aA.test(bZ.nodeName)||E.test(bZ.nodeName)&&bZ.href?0:aB}}}});bV={get:func
 tion(bZ,e){var b1,b0=bG.prop(bZ,e);return b0===true||typeof b0!=="boolean"&&(b1=bZ.getAttributeNode(e))&&b1.nodeValue!==false?e.toLowerCase():aB},set:function(bZ,b1,e){var b0;if(b1===false){bG.removeAttr(bZ,e)}else{b0=bG.propFix[e]||e;if(b0 in bZ){bZ[b0]=true}bZ.setAttribute(e,e.toLowerCase())}return e}};if(!bL){n={name:true,id:true,coords:true};a7=bG.valHooks.button={get:function(b0,bZ){var e;e=b0.getAttributeNode(bZ);return e&&(n[bZ]?e.value!=="":e.specified)?e.value:aB},set:function(b0,b1,bZ){var e=b0.getAttributeNode(bZ);if(!e){e=o.createAttribute(bZ);b0.setAttributeNode(e)}return(e.value=b1+"")}};bG.each(["width","height"],function(bZ,e){bG.attrHooks[e]=bG.extend(bG.attrHooks[e],{set:function(b0,b1){if(b1===""){b0.setAttribute(e,"auto");return b1}}})});bG.attrHooks.contenteditable={get:a7.get,set:function(bZ,b0,e){if(b0===""){b0="false"}a7.set(bZ,b0,e)}}}if(!bG.support.hrefNormalized){bG.each(["href","src","width","height"],function(bZ,e){bG.attrHooks[e]=bG.extend(bG.attrHooks[
 e],{get:function(b1){var b0=b1.getAttribute(e,2);return b0===null?aB:b0}})})}if(!bG.support.style){bG.attrHooks.style={get:function(e){return e.style.cssText.toLowerCase()||aB},set:function(e,bZ){return(e.style.cssText=""+bZ)}}}if(!bG.support.optSelected){bG.propHooks.selected=bG.extend(bG.propHooks.selected,{get:function(bZ){var e=bZ.parentNode;if(e){e.selectedIndex;if(e.parentNode){e.parentNode.selectedIndex}}return null}})}if(!bG.support.enctype){bG.propFix.enctype="encoding"}if(!bG.support.checkOn){bG.each(["radio","checkbox"],function(){bG.valHooks[this]={get:function(e){return e.getAttribute("value")===null?"on":e.value}}})}bG.each(["radio","checkbox"],function(){bG.valHooks[this]=bG.extend(bG.valHooks[this],{set:function(e,bZ){if(bG.isArray(bZ)){return(e.checked=bG.inArray(bG(e).val(),bZ)>=0)}}})});var bE=/^(?:textarea|input|select)$/i,br=/^([^\.]*|)(?:\.(.+)|)$/,ba=/(?:^|\s)hover(\.\S+|)\b/,a3=/^key/,bK=/^(?:mouse|contextmenu)|click/,by=/^(?:focusinfocus|focusoutblur)$/,aq=f
 unction(e){return bG.event.special.hover?e:e.replace(ba,"mouseenter$1 mouseleave$1")};bG.event={add:function(b1,b5,cc,b3,b2){var b6,b4,cd,cb,ca,b8,e,b9,bZ,b0,b7;if(b1.nodeType===3||b1.nodeType===8||!b5||!cc||!(b6=bG._data(b1))){return}if(cc.handler){bZ=cc;cc=bZ.handler;b2=bZ.selector}if(!cc.guid){cc.guid=bG.guid++}cd=b6.events;if(!cd){b6.events=cd={}}b4=b6.handle;if(!b4){b6.handle=b4=function(ce){return typeof bG!=="undefined"&&(!ce||bG.event.triggered!==ce.type)?bG.event.dispatch.apply(b4.elem,arguments):aB};b4.elem=b1}b5=bG.trim(aq(b5)).split(" ");for(cb=0;cb<b5.length;cb++){ca=br.exec(b5[cb])||[];b8=ca[1];e=(ca[2]||"").split(".").sort();b7=bG.event.special[b8]||{};b8=(b2?b7.delegateType:b7.bindType)||b8;b7=bG.event.special[b8]||{};b9=bG.extend({type:b8,origType:ca[1],data:b3,handler:cc,guid:cc.guid,selector:b2,namespace:e.join(".")},bZ);b0=cd[b8];if(!b0){b0=cd[b8]=[];b0.delegateCount=0;if(!b7.setup||b7.setup.call(b1,b3,e,b4)===false){if(b1.addEventListener){b1.addEventListener(b8
 ,b4,false)}else{if(b1.attachEvent){b1.attachEvent("on"+b8,b4)}}}}if(b7.add){b7.add.call(b1,b9);if(!b9.handler.guid){b9.handler.guid=cc.guid}}if(b2){b0.splice(b0.delegateCount++,0,b9)}else{b0.push(b9)}bG.event.global[b8]=true}b1=null},global:{},remove:function(b1,b6,cc,b2,b5){var cd,ce,b9,b0,bZ,b3,b4,cb,b8,e,ca,b7=bG.hasData(b1)&&bG._data(b1);if(!b7||!(cb=b7.events)){return}b6=bG.trim(aq(b6||"")).split(" ");for(cd=0;cd<b6.length;cd++){ce=br.exec(b6[cd])||[];b9=b0=ce[1];bZ=ce[2];if(!b9){for(b9 in cb){bG.event.remove(b1,b9+b6[cd],cc,b2,true)}continue}b8=bG.event.special[b9]||{};b9=(b2?b8.delegateType:b8.bindType)||b9;e=cb[b9]||[];b3=e.length;bZ=bZ?new RegExp("(^|\\.)"+bZ.split(".").sort().join("\\.(?:.*\\.|)")+"(\\.|$)"):null;for(b4=0;b4<e.length;b4++){ca=e[b4];if((b5||b0===ca.origType)&&(!cc||cc.guid===ca.guid)&&(!bZ||bZ.test(ca.namespace))&&(!b2||b2===ca.selector||b2==="**"&&ca.selector)){e.splice(b4--,1);if(ca.selector){e.delegateCount--}if(b8.remove){b8.remove.call(b1,ca)}}}if(e.le
 ngth===0&&b3!==e.length){if(!b8.teardown||b8.teardown.call(b1,bZ,b7.handle)===false){bG.removeEvent(b1,b9,b7.handle)}delete cb[b9]}}if(bG.isEmptyObject(cb)){delete b7.handle;bG.removeData(b1,"events",true)}},customEvent:{getData:true,setData:true,changeData:true},trigger:function(bZ,b6,b4,cd){if(b4&&(b4.nodeType===3||b4.nodeType===8)){return}var e,b1,b7,cb,b3,b2,b9,b8,b5,cc,ca=bZ.type||bZ,b0=[];if(by.test(ca+bG.event.triggered)){return}if(ca.indexOf("!")>=0){ca=ca.slice(0,-1);b1=true}if(ca.indexOf(".")>=0){b0=ca.split(".");ca=b0.shift();b0.sort()}if((!b4||bG.event.customEvent[ca])&&!bG.event.global[ca]){return}bZ=typeof bZ==="object"?bZ[bG.expando]?bZ:new bG.Event(ca,bZ):new bG.Event(ca);bZ.type=ca;bZ.isTrigger=true;bZ.exclusive=b1;bZ.namespace=b0.join(".");bZ.namespace_re=bZ.namespace?new RegExp("(^|\\.)"+b0.join("\\.(?:.*\\.|)")+"(\\.|$)"):null;b2=ca.indexOf(":")<0?"on"+ca:"";if(!b4){e=bG.cache;for(b7 in e){if(e[b7].events&&e[b7].events[ca]){bG.event.trigger(bZ,b6,e[b7].handle.ele
 m,true)}}return}bZ.result=aB;if(!bZ.target){bZ.target=b4}b6=b6!=null?bG.makeArray(b6):[];b6.unshift(bZ);b9=bG.event.special[ca]||{};if(b9.trigger&&b9.trigger.apply(b4,b6)===false){return}b5=[[b4,b9.bindType||ca]];if(!cd&&!b9.noBubble&&!bG.isWindow(b4)){cc=b9.delegateType||ca;cb=by.test(cc+ca)?b4:b4.parentNode;for(b3=b4;cb;cb=cb.parentNode){b5.push([cb,cc]);b3=cb}if(b3===(b4.ownerDocument||o)){b5.push([b3.defaultView||b3.parentWindow||a2,cc])}}for(b7=0;b7<b5.length&&!bZ.isPropagationStopped();b7++){cb=b5[b7][0];bZ.type=b5[b7][1];b8=(bG._data(cb,"events")||{})[bZ.type]&&bG._data(cb,"handle");if(b8){b8.apply(cb,b6)}b8=b2&&cb[b2];if(b8&&bG.acceptData(cb)&&b8.apply(cb,b6)===false){bZ.preventDefault()}}bZ.type=ca;if(!cd&&!bZ.isDefaultPrevented()){if((!b9._default||b9._default.apply(b4.ownerDocument,b6)===false)&&!(ca==="click"&&bG.nodeName(b4,"a"))&&bG.acceptData(b4)){if(b2&&b4[ca]&&((ca!=="focus"&&ca!=="blur")||bZ.target.offsetWidth!==0)&&!bG.isWindow(b4)){b3=b4[b2];if(b3){b4[b2]=null}bG
 .event.triggered=ca;b4[ca]();bG.event.triggered=aB;if(b3){b4[b2]=b3}}}}return bZ.result},dispatch:function(cb){cb=bG.event.fix(cb||a2.event);var cd,ca,b2,b4,ce,cc,b5,b0,e,b9,cf,b7=((bG._data(this,"events")||{})[cb.type]||[]),b6=b7.delegateCount,b1=[].slice.call(arguments),b8=!cb.exclusive&&!cb.namespace,b3=bG.event.special[cb.type]||{},bZ=[];b1[0]=cb;cb.delegateTarget=this;if(b3.preDispatch&&b3.preDispatch.call(this,cb)===false){return}if(b6&&!(cb.button&&cb.type==="click")){b4=bG(this);b4.context=this;for(b2=cb.target;b2!=this;b2=b2.parentNode||this){if(b2.disabled!==true||cb.type!=="click"){cc={};b0=[];b4[0]=b2;for(cd=0;cd<b6;cd++){e=b7[cd];b9=e.selector;if(cc[b9]===aB){cc[b9]=b4.is(b9)}if(cc[b9]){b0.push(e)}}if(b0.length){bZ.push({elem:b2,matches:b0})}}}}if(b7.length>b6){bZ.push({elem:this,matches:b7.slice(b6)})}for(cd=0;cd<bZ.length&&!cb.isPropagationStopped();cd++){b5=bZ[cd];cb.currentTarget=b5.elem;for(ca=0;ca<b5.matches.length&&!cb.isImmediatePropagationStopped();ca++){e=b5.m
 atches[ca];if(b8||(!cb.namespace&&!e.namespace)||cb.namespace_re&&cb.namespace_re.test(e.namespace)){cb.data=e.data;cb.handleObj=e;ce=((bG.event.special[e.origType]||{}).handle||e.handler).apply(b5.elem,b1);if(ce!==aB){cb.result=ce;if(ce===false){cb.preventDefault();cb.stopPropagation()}}}}}if(b3.postDispatch){b3.postDispatch.call(this,cb)}return cb.result},props:"attrChange attrName relatedNode srcElement altKey bubbles cancelable ctrlKey currentTarget eventPhase metaKey relatedTarget shiftKey target timeStamp view which".split(" "),fixHooks:{},keyHooks:{props:"char charCode key keyCode".split(" "),filter:function(bZ,e){if(bZ.which==null){bZ.which=e.charCode!=null?e.charCode:e.keyCode}return bZ}},mouseHooks:{props:"button buttons clientX clientY fromElement offsetX offsetY pageX pageY screenX screenY toElement".split(" "),filter:function(b1,b0){var b2,b3,e,bZ=b0.button,b4=b0.fromElement;if(b1.pageX==null&&b0.clientX!=null){b2=b1.target.ownerDocument||o;b3=b2.documentElement;e=b2.bo
 dy;b1.pageX=b0.clientX+(b3&&b3.scrollLeft||e&&e.scrollLeft||0)-(b3&&b3.clientLeft||e&&e.clientLeft||0);b1.pageY=b0.clientY+(b3&&b3.scrollTop||e&&e.scrollTop||0)-(b3&&b3.clientTop||e&&e.clientTop||0)}if(!b1.relatedTarget&&b4){b1.relatedTarget=b4===b1.target?b0.toElement:b4}if(!b1.which&&bZ!==aB){b1.which=(bZ&1?1:(bZ&2?3:(bZ&4?2:0)))}return b1}},fix:function(b0){if(b0[bG.expando]){return b0}var bZ,b3,e=b0,b1=bG.event.fixHooks[b0.type]||{},b2=b1.props?this.props.concat(b1.props):this.props;b0=bG.Event(e);for(bZ=b2.length;bZ;){b3=b2[--bZ];b0[b3]=e[b3]}if(!b0.target){b0.target=e.srcElement||o}if(b0.target.nodeType===3){b0.target=b0.target.parentNode}b0.metaKey=!!b0.metaKey;return b1.filter?b1.filter(b0,e):b0},special:{ready:{setup:bG.bindReady},load:{noBubble:true},focus:{delegateType:"focusin"},blur:{delegateType:"focusout"},beforeunload:{setup:function(b0,bZ,e){if(bG.isWindow(this)){this.onbeforeunload=e}},teardown:function(bZ,e){if(this.onbeforeunload===e){this.onbeforeunload=null}}}}
 ,simulate:function(b0,b2,b1,bZ){var b3=bG.extend(new bG.Event(),b1,{type:b0,isSimulated:true,originalEvent:{}});if(bZ){bG.event.trigger(b3,null,b2)}else{bG.event.dispatch.call(b2,b3)}if(b3.isDefaultPrevented()){b1.preventDefault()}}};bG.event.handle=bG.event.dispatch;bG.removeEvent=o.removeEventListener?function(bZ,e,b0){if(bZ.removeEventListener){bZ.removeEventListener(e,b0,false)}}:function(b0,bZ,b1){var e="on"+bZ;if(b0.detachEvent){if(typeof b0[e]==="undefined"){b0[e]=null}b0.detachEvent(e,b1)}};bG.Event=function(bZ,e){if(!(this instanceof bG.Event)){return new bG.Event(bZ,e)}if(bZ&&bZ.type){this.originalEvent=bZ;this.type=bZ.type;this.isDefaultPrevented=(bZ.defaultPrevented||bZ.returnValue===false||bZ.getPreventDefault&&bZ.getPreventDefault())?R:X}else{this.type=bZ}if(e){bG.extend(this,e)}this.timeStamp=bZ&&bZ.timeStamp||bG.now();this[bG.expando]=true};function X(){return false}function R(){return true}bG.Event.prototype={preventDefault:function(){this.isDefaultPrevented=R;var b
 Z=this.originalEvent;if(!bZ){return}if(bZ.preventDefault){bZ.preventDefault()}else{bZ.returnValue=false}},stopPropagation:function(){this.isPropagationStopped=R;var bZ=this.originalEvent;if(!bZ){return}if(bZ.stopPropagation){bZ.stopPropagation()}bZ.cancelBubble=true},stopImmediatePropagation:function(){this.isImmediatePropagationStopped=R;this.stopPropagation()},isDefaultPrevented:X,isPropagationStopped:X,isImmediatePropagationStopped:X};bG.each({mouseenter:"mouseover",mouseleave:"mouseout"},function(bZ,e){bG.event.special[bZ]={delegateType:e,bindType:e,handle:function(b3){var b1,b5=this,b4=b3.relatedTarget,b2=b3.handleObj,b0=b2.selector;if(!b4||(b4!==b5&&!bG.contains(b5,b4))){b3.type=b2.origType;b1=b2.handler.apply(this,arguments);b3.type=e}return b1}}});if(!bG.support.submitBubbles){bG.event.special.submit={setup:function(){if(bG.nodeName(this,"form")){return false}bG.event.add(this,"click._submit keypress._submit",function(b1){var b0=b1.target,bZ=bG.nodeName(b0,"input")||bG.nodeN
 ame(b0,"button")?b0.form:aB;if(bZ&&!bG._data(bZ,"_submit_attached")){bG.event.add(bZ,"submit._submit",function(e){e._submit_bubble=true});bG._data(bZ,"_submit_attached",true)}})},postDispatch:function(e){if(e._submit_bubble){delete e._submit_bubble;if(this.parentNode&&!e.isTrigger){bG.event.simulate("submit",this.parentNode,e,true)}}},teardown:function(){if(bG.nodeName(this,"form")){return false}bG.event.remove(this,"._submit")}}}if(!bG.support.changeBubbles){bG.event.special.change={setup:function(){if(bE.test(this.nodeName)){if(this.type==="checkbox"||this.type==="radio"){bG.event.add(this,"propertychange._change",function(e){if(e.originalEvent.propertyName==="checked"){this._just_changed=true}});bG.event.add(this,"click._change",function(e){if(this._just_changed&&!e.isTrigger){this._just_changed=false}bG.event.simulate("change",this,e,true)})}return false}bG.event.add(this,"beforeactivate._change",function(b0){var bZ=b0.target;if(bE.test(bZ.nodeName)&&!bG._data(bZ,"_change_attach
 ed")){bG.event.add(bZ,"change._change",function(e){if(this.parentNode&&!e.isSimulated&&!e.isTrigger){bG.event.simulate("change",this.parentNode,e,true)}});bG._data(bZ,"_change_attached",true)}})},handle:function(bZ){var e=bZ.target;if(this!==e||bZ.isSimulated||bZ.isTrigger||(e.type!=="radio"&&e.type!=="checkbox")){return bZ.handleObj.handler.apply(this,arguments)}},teardown:function(){bG.event.remove(this,"._change");return bE.test(this.nodeName)}}}if(!bG.support.focusinBubbles){bG.each({focus:"focusin",blur:"focusout"},function(b1,e){var bZ=0,b0=function(b2){bG.event.simulate(e,b2.target,bG.event.fix(b2),true)};bG.event.special[e]={setup:function(){if(bZ++===0){o.addEventListener(b1,b0,true)}},teardown:function(){if(--bZ===0){o.removeEventListener(b1,b0,true)}}}})}bG.fn.extend({on:function(b0,e,b3,b2,bZ){var b4,b1;if(typeof b0==="object"){if(typeof e!=="string"){b3=b3||e;e=aB}for(b1 in b0){this.on(b1,e,b3,b0[b1],bZ)}return this}if(b3==null&&b2==null){b2=e;b3=e=aB}else{if(b2==null){
 if(typeof e==="string"){b2=b3;b3=aB}else{b2=b3;b3=e;e=aB}}}if(b2===false){b2=X}else{if(!b2){return this}}if(bZ===1){b4=b2;b2=function(b5){bG().off(b5);return b4.apply(this,arguments)};b2.guid=b4.guid||(b4.guid=bG.guid++)}return this.each(function(){bG.event.add(this,b0,b2,b3,e)})},one:function(bZ,e,b1,b0){return this.on(bZ,e,b1,b0,1)},off:function(b0,e,b2){var bZ,b1;if(b0&&b0.preventDefault&&b0.handleObj){bZ=b0.handleObj;bG(b0.delegateTarget).off(bZ.namespace?bZ.origType+"."+bZ.namespace:bZ.origType,bZ.selector,bZ.handler);return this}if(typeof b0==="object"){for(b1 in b0){this.off(b1,e,b0[b1])}return this}if(e===false||typeof e==="function"){b2=e;e=aB}if(b2===false){b2=X}return this.each(function(){bG.event.remove(this,b0,b2,e)})},bind:function(e,b0,bZ){return this.on(e,null,b0,bZ)},unbind:function(e,bZ){return this.off(e,null,bZ)},live:function(e,b0,bZ){bG(this.context).on(e,this.selector,b0,bZ);return this},die:function(e,bZ){bG(this.context).off(e,this.selector||"**",bZ);return 
 this},delegate:function(e,bZ,b1,b0){return this.on(bZ,e,b1,b0)},undelegate:function(e,bZ,b0){return arguments.length==1?this.off(e,"**"):this.off(bZ,e||"**",b0)},trigger:function(e,bZ){return this.each(function(){bG.event.trigger(e,bZ,this)})},triggerHandler:function(e,bZ){if(this[0]){return bG.event.trigger(e,bZ,this[0],true)}},toggle:function(b1){var bZ=arguments,e=b1.guid||bG.guid++,b0=0,b2=function(b3){var b4=(bG._data(this,"lastToggle"+b1.guid)||0)%b0;bG._data(this,"lastToggle"+b1.guid,b4+1);b3.preventDefault();return bZ[b4].apply(this,arguments)||false};b2.guid=e;while(b0<bZ.length){bZ[b0++].guid=e}return this.click(b2)},hover:function(e,bZ){return this.mouseenter(e).mouseleave(bZ||e)}});bG.each(("blur focus focusin focusout load resize scroll unload click dblclick mousedown mouseup mousemove mouseover mouseout mouseenter mouseleave change select submit keydown keypress keyup error contextmenu").split(" "),function(bZ,e){bG.fn[e]=function(b1,b0){if(b0==null){b0=b1;b1=null}retu
 rn arguments.length>0?this.on(e,null,b1,b0):this.trigger(e)};if(a3.test(e)){bG.event.fixHooks[e]=bG.event.keyHooks}if(bK.test(e)){bG.event.fixHooks[e]=bG.event.mouseHooks}});
-/*!
- * Sizzle CSS Selector Engine
- *  Copyright 2012 jQuery Foundation and other contributors
- *  Released under the MIT license
- *  http://sizzlejs.com/
- */
-(function(cO,cg){var cT,co,cf,b2,b8,b6=cO.document,b9=b6.documentElement,cw="undefined",ca=false,b7=true,ce=0,cj=[].slice,cS=[].push,cW=("sizcache"+Math.random()).replace(".",""),cz="[\\x20\\t\\r\\n\\f]",ci="(?:\\\\.|[-\\w]|[^\\x00-\\xa0])+",ch=ci.replace("w","w#"),c1="([*^$|!~]?=)",cL="\\["+cz+"*("+ci+")"+cz+"*(?:"+c1+cz+"*(?:(['\"])((?:\\\\.|[^\\\\])*?)\\3|("+ch+")|)|)"+cz+"*\\]",c2=":("+ci+")(?:\\((?:(['\"])((?:\\\\.|[^\\\\])*?)\\2|((?:[^,]|\\\\,|(?:,(?=[^\\[]*\\]))|(?:,(?=[^\\(]*\\))))*))\\)|)",cB=":(nth|eq|gt|lt|first|last|even|odd)(?:\\((\\d*)\\)|)(?=[^-]|$)",cd=cz+"*([\\x20\\t\\r\\n\\f>+~])"+cz+"*",cc="(?=[^\\x20\\t\\r\\n\\f])(?:\\\\.|"+cL+"|"+c2.replace(2,7)+"|[^\\\\(),])+",cU=new RegExp("^"+cz+"+|((?:^|[^\\\\])(?:\\\\.)*)"+cz+"+$","g"),cF=new RegExp("^"+cd),ct=new RegExp(cc+"?(?="+cz+"*,|$)","g"),cJ=new RegExp("^(?:(?!,)(?:(?:^|,)"+cz+"*"+cc+")*?|"+cz+"*(.*?))(\\)|$)"),cZ=new RegExp(cc.slice(19,-6)+"\\x20\\t\\r\\n\\f>+~])+|"+cd,"g"),cK=/^(?:#([\w\-]+)|(\w+)|\.([\w\-]+))$/,c
 P=/[\x20\t\r\n\f]*[+~]/,cX=/:not\($/,cp=/h\d/i,cM=/input|select|textarea|button/i,cs=/\\(?!\\)/g,cE={ID:new RegExp("^#("+ci+")"),CLASS:new RegExp("^\\.("+ci+")"),NAME:new RegExp("^\\[name=['\"]?("+ci+")['\"]?\\]"),TAG:new RegExp("^("+ci.replace("[-","[-\\*")+")"),ATTR:new RegExp("^"+cL),PSEUDO:new RegExp("^"+c2),CHILD:new RegExp("^:(only|nth|last|first)-child(?:\\("+cz+"*(even|odd|(([+-]|)(\\d*)n|)"+cz+"*(?:([+-]|)"+cz+"*(\\d+)|))"+cz+"*\\)|)","i"),POS:new RegExp(cB,"ig"),needsContext:new RegExp("^"+cz+"*[>+~]|"+cB,"i")},cR={},cq=[],cl={},cu=[],cY=function(e){e.sizzleFilter=true;return e},b3=function(e){return function(c3){return c3.nodeName.toLowerCase()==="input"&&c3.type===e}},cr=function(e){return function(c4){var c3=c4.nodeName.toLowerCase();return(c3==="input"||c3==="button")&&c4.type===e}},cH=function(c3){var c4=false,c6=b6.createElement("div");try{c4=c3(c6)}catch(c5){}c6=null;return c4},cn=cH(function(c3){c3.innerHTML="<select></select>";var e=typeof c3.lastChild.getAttribut
 e("multiple");return e!=="boolean"&&e!=="string"}),b0=cH(function(c3){c3.id=cW+0;c3.innerHTML="<a name='"+cW+"'></a><div name='"+cW+"'></div>";b9.insertBefore(c3,b9.firstChild);var e=b6.getElementsByName&&b6.getElementsByName(cW).length===2+b6.getElementsByName(cW+0).length;b8=!b6.getElementById(cW);b9.removeChild(c3);return e}),b5=cH(function(e){e.appendChild(b6.createComment(""));return e.getElementsByTagName("*").length===0}),cD=cH(function(e){e.innerHTML="<a href='#'></a>";return e.firstChild&&typeof e.firstChild.getAttribute!==cw&&e.firstChild.getAttribute("href")==="#"}),cC=cH(function(e){e.innerHTML="<div class='hidden e'></div><div class='hidden'></div>";if(!e.getElementsByClassName||e.getElementsByClassName("e").length===0){return false}e.lastChild.className="e";return e.getElementsByClassName("e").length!==1});var cN=function(c5,e,c7,da){c7=c7||[];e=e||b6;var c8,c3,c9,c4,c6=e.nodeType;if(c6!==1&&c6!==9){return[]}if(!c5||typeof c5!=="string"){return c7}c9=ck(e);if(!c9&&!da)
 {if((c8=cK.exec(c5))){if((c4=c8[1])){if(c6===9){c3=e.getElementById(c4);if(c3&&c3.parentNode){if(c3.id===c4){c7.push(c3);return c7}}else{return c7}}else{if(e.ownerDocument&&(c3=e.ownerDocument.getElementById(c4))&&cA(e,c3)&&c3.id===c4){c7.push(c3);return c7}}}else{if(c8[2]){cS.apply(c7,cj.call(e.getElementsByTagName(c5),0));return c7}else{if((c4=c8[3])&&cC&&e.getElementsByClassName){cS.apply(c7,cj.call(e.getElementsByClassName(c4),0));return c7}}}}}return cV(c5,e,c7,da,c9)};var cG=cN.selectors={cacheLength:50,match:cE,order:["ID","TAG"],attrHandle:{},createPseudo:cY,find:{ID:b8?function(c5,c4,c3){if(typeof c4.getElementById!==cw&&!c3){var e=c4.getElementById(c5);return e&&e.parentNode?[e]:[]}}:function(c5,c4,c3){if(typeof c4.getElementById!==cw&&!c3){var e=c4.getElementById(c5);return e?e.id===c5||typeof e.getAttributeNode!==cw&&e.getAttributeNode("id").value===c5?[e]:cg:[]}},TAG:b5?function(e,c3){if(typeof c3.getElementsByTagName!==cw){return c3.getElementsByTagName(e)}}:function(e
 ,c6){var c5=c6.getElementsByTagName(e);if(e==="*"){var c7,c4=[],c3=0;for(;(c7=c5[c3]);c3++){if(c7.nodeType===1){c4.push(c7)}}return c4}return c5}},relative:{">":{dir:"parentNode",first:true}," ":{dir:"parentNode"},"+":{dir:"previousSibling",first:true},"~":{dir:"previousSibling"}},preFilter:{ATTR:function(e){e[1]=e[1].replace(cs,"");e[3]=(e[4]||e[5]||"").replace(cs,"");if(e[2]==="~="){e[3]=" "+e[3]+" "}return e.slice(0,4)},CHILD:function(e){e[1]=e[1].toLowerCase();if(e[1]==="nth"){if(!e[2]){cN.error(e[0])}e[3]=+(e[3]?e[4]+(e[5]||1):2*(e[2]==="even"||e[2]==="odd"));e[4]=+((e[6]+e[7])||e[2]==="odd")}else{if(e[2]){cN.error(e[0])}}return e},PSEUDO:function(e){var c3,c4=e[4];if(cE.CHILD.test(e[0])){return null}if(c4&&(c3=cJ.exec(c4))&&c3.pop()){e[0]=e[0].slice(0,c3[0].length-c4.length-1);c4=c3[0].slice(0,-1)}e.splice(2,3,c4||e[3]);return e}},filter:{ID:b8?function(e){e=e.replace(cs,"");return function(c3){return c3.getAttribute("id")===e}}:function(e){e=e.replace(cs,"");return function(c
 4){var c3=typeof c4.getAttributeNode!==cw&&c4.getAttributeNode("id");return c3&&c3.value===e}},TAG:function(e){if(e==="*"){return function(){return true}}e=e.replace(cs,"").toLowerCase();return function(c3){return c3.nodeName&&c3.nodeName.toLowerCase()===e}},CLASS:function(e){var c3=cR[e];if(!c3){c3=cR[e]=new RegExp("(^|"+cz+")"+e+"("+cz+"|$)");cq.push(e);if(cq.length>cG.cacheLength){delete cR[cq.shift()]}}return function(c4){return c3.test(c4.className||(typeof c4.getAttribute!==cw&&c4.getAttribute("class"))||"")}},ATTR:function(c4,c3,e){if(!c3){return function(c5){return cN.attr(c5,c4)!=null}}return function(c6){var c5=cN.attr(c6,c4),c7=c5+"";if(c5==null){return c3==="!="}switch(c3){case"=":return c7===e;case"!=":return c7!==e;case"^=":return e&&c7.indexOf(e)===0;case"*=":return e&&c7.indexOf(e)>-1;case"$=":return e&&c7.substr(c7.length-e.length)===e;case"~=":return(" "+c7+" ").indexOf(e)>-1;case"|=":return c7===e||c7.substr(0,e.length+1)===e+"-"}}},CHILD:function(c3,c5,c6,c4){if(
 c3==="nth"){var e=ce++;return function(da){var c7,db,c9=0,c8=da;if(c6===1&&c4===0){return true}c7=da.parentNode;if(c7&&(c7[cW]!==e||!da.sizset)){for(c8=c7.firstChild;c8;c8=c8.nextSibling){if(c8.nodeType===1){c8.sizset=++c9;if(c8===da){break}}}c7[cW]=e}db=da.sizset-c4;if(c6===0){return db===0}else{return(db%c6===0&&db/c6>=0)}}}return function(c8){var c7=c8;switch(c3){case"only":case"first":while((c7=c7.previousSibling)){if(c7.nodeType===1){return false}}if(c3==="first"){return true}c7=c8;case"last":while((c7=c7.nextSibling)){if(c7.nodeType===1){return false}}return true}}},PSEUDO:function(c6,c5,c3,e){var c4=cG.pseudos[c6]||cG.pseudos[c6.toLowerCase()];if(!c4){cN.error("unsupported pseudo: "+c6)}if(!c4.sizzleFilter){return c4}return c4(c5,c3,e)}},pseudos:{not:cY(function(e,c4,c3){var c5=cb(e.replace(cU,"$1"),c4,c3);return function(c6){return !c5(c6)}}),enabled:function(e){return e.disabled===false},disabled:function(e){return e.disabled===true},checked:function(e){var c3=e.nodeName.to
 LowerCase();return(c3==="input"&&!!e.checked)||(c3==="option"&&!!e.selected)},selected:function(e){if(e.parentNode){e.parentNode.selectedIndex}return e.selected===true},parent:function(e){return !cG.pseudos.empty(e)},empty:function(c3){var e;c3=c3.firstChild;while(c3){if(c3.nodeName>"@"||(e=c3.nodeType)===3||e===4){return false}c3=c3.nextSibling}return true},contains:cY(function(e){return function(c3){return(c3.textContent||c3.innerText||bZ(c3)).indexOf(e)>-1}}),has:cY(function(e){return function(c3){return cN(e,c3).length>0}}),header:function(e){return cp.test(e.nodeName)},text:function(c4){var c3,e;return c4.nodeName.toLowerCase()==="input"&&(c3=c4.type)==="text"&&((e=c4.getAttribute("type"))==null||e.toLowerCase()===c3)},radio:b3("radio"),checkbox:b3("checkbox"),file:b3("file"),password:b3("password"),image:b3("image"),submit:cr("submit"),reset:cr("reset"),button:function(c3){var e=c3.nodeName.toLowerCase();return e==="input"&&c3.type==="button"||e==="button"},input:function(e){r
 eturn cM.test(e.nodeName)},focus:function(e){var c3=e.ownerDocument;return e===c3.activeElement&&(!c3.hasFocus||c3.hasFocus())&&!!(e.type||e.href)},active:function(e){return e===e.ownerDocument.activeElement}},setFilters:{first:function(c4,c3,e){return e?c4.slice(1):[c4[0]]},last:function(c5,c4,c3){var e=c5.pop();return c3?c5:[e]},even:function(c7,c6,c5){var c4=[],c3=c5?1:0,e=c7.length;for(;c3<e;c3=c3+2){c4.push(c7[c3])}return c4},odd:function(c7,c6,c5){var c4=[],c3=c5?0:1,e=c7.length;for(;c3<e;c3=c3+2){c4.push(c7[c3])}return c4},lt:function(c4,c3,e){return e?c4.slice(+c3):c4.slice(0,+c3)},gt:function(c4,c3,e){return e?c4.slice(0,+c3+1):c4.slice(+c3+1)},eq:function(c5,c4,c3){var e=c5.splice(+c4,1);return c3?c5:e}}};cG.setFilters.nth=cG.setFilters.eq;cG.filters=cG.pseudos;if(!cD){cG.attrHandle={href:function(e){return e.getAttribute("href",2)},type:function(e){return e.getAttribute("type")}}}if(b0){cG.order.push("NAME");cG.find.NAME=function(e,c3){if(typeof c3.getElementsByName!==cw)
 {return c3.getElementsByName(e)}}}if(cC){cG.order.splice(1,0,"CLASS");cG.find.CLASS=function(c4,c3,e){if(typeof c3.getElementsByClassName!==cw&&!e){return c3.getElementsByClassName(c4)}}}try{cj.call(b9.childNodes,0)[0].nodeType}catch(c0){cj=function(c3){var c4,e=[];for(;(c4=this[c3]);c3++){e.push(c4)}return e}}var ck=cN.isXML=function(e){var c3=e&&(e.ownerDocument||e).documentElement;return c3?c3.nodeName!=="HTML":false};var cA=cN.contains=b9.compareDocumentPosition?function(c3,e){return !!(c3.compareDocumentPosition(e)&16)}:b9.contains?function(c3,e){var c5=c3.nodeType===9?c3.documentElement:c3,c4=e.parentNode;return c3===c4||!!(c4&&c4.nodeType===1&&c5.contains&&c5.contains(c4))}:function(c3,e){while((e=e.parentNode)){if(e===c3){return true}}return false};var bZ=cN.getText=function(c6){var c5,c3="",c4=0,e=c6.nodeType;if(e){if(e===1||e===9||e===11){if(typeof c6.textContent==="string"){return c6.textContent}else{for(c6=c6.firstChild;c6;c6=c6.nextSibling){c3+=bZ(c6)}}}else{if(e===3||e
 ===4){return c6.nodeValue}}}else{for(;(c5=c6[c4]);c4++){c3+=bZ(c5)}}return c3};cN.attr=function(c5,c4){var e,c3=ck(c5);if(!c3){c4=c4.toLowerCase()}if(cG.attrHandle[c4]){return cG.attrHandle[c4](c5)}if(cn||c3){return c5.getAttribute(c4)}e=c5.getAttributeNode(c4);return e?typeof c5[c4]==="boolean"?c5[c4]?c4:null:e.specified?e.value:null:null};cN.error=function(e){throw new Error("Syntax error, unrecognized expression: "+e)};[0,0].sort(function(){return(b7=0)});if(b9.compareDocumentPosition){cf=function(c3,e){if(c3===e){ca=true;return 0}return(!c3.compareDocumentPosition||!e.compareDocumentPosition?c3.compareDocumentPosition:c3.compareDocumentPosition(e)&4)?-1:1}}else{cf=function(da,c9){if(da===c9){ca=true;return 0}else{if(da.sourceIndex&&c9.sourceIndex){return da.sourceIndex-c9.sourceIndex}}var c7,c3,c4=[],e=[],c6=da.parentNode,c8=c9.parentNode,db=c6;if(c6===c8){return b2(da,c9)}else{if(!c6){return -1}else{if(!c8){return 1}}}while(db){c4.unshift(db);db=db.parentNode}db=c8;while(db){e.
 unshift(db);db=db.parentNode}c7=c4.length;c3=e.length;for(var c5=0;c5<c7&&c5<c3;c5++){if(c4[c5]!==e[c5]){return b2(c4[c5],e[c5])}}return c5===c7?b2(da,e[c5],-1):b2(c4[c5],c9,1)};b2=function(c3,e,c4){if(c3===e){return c4}var c5=c3.nextSibling;while(c5){if(c5===e){return -1}c5=c5.nextSibling}return 1}}cN.uniqueSort=function(c3){var c4,e=1;if(cf){ca=b7;c3.sort(cf);if(ca){for(;(c4=c3[e]);e++){if(c4===c3[e-1]){c3.splice(e--,1)}}}}return c3};function cm(c3,c7,c6,c4){var c5=0,e=c7.length;for(;c5<e;c5++){cN(c3,c7[c5],c6,c4)}}function cI(e,c4,c8,c9,c3,c7){var c5,c6=cG.setFilters[c4.toLowerCase()];if(!c6){cN.error(c4)}if(e||!(c5=c3)){cm(e||"*",c9,(c5=[]),c3)}return c5.length>0?c6(c5,c8,c7):[]}function cQ(dc,e,da,c4,dg){var c7,c3,c6,di,c9,dh,db,df,dd=0,de=dg.length,c5=cE.POS,c8=new RegExp("^"+c5.source+"(?!"+cz+")","i"),dj=function(){var dl=1,dk=arguments.length-2;for(;dl<dk;dl++){if(arguments[dl]===cg){c7[dl]=cg}}};for(;dd<de;dd++){c5.exec("");dc=dg[dd];di=[];c6=0;c9=c4;while((c7=c5.exec(dc))
 ){df=c5.lastIndex=c7.index+c7[0].length;if(df>c6){db=dc.slice(c6,c7.index);c6=df;dh=[e];if(cF.test(db)){if(c9){dh=c9}c9=c4}if((c3=cX.test(db))){db=db.slice(0,-5).replace(cF,"$&*")}if(c7.length>1){c7[0].replace(c8,dj)}c9=cI(db,c7[1],c7[2],dh,c9,c3)}}if(c9){di=di.concat(c9);if((db=dc.slice(c6))&&db!==")"){if(cF.test(db)){cm(db,di,da,c4)}else{cN(db,e,da,c4?c4.concat(c9):c9)}}else{cS.apply(da,di)}}else{cN(dc,e,da,c4)}}return de===1?da:cN.uniqueSort(da)}function b1(c8,c4,db){var dd,dc,de,c6=[],c9=0,da=cJ.exec(c8),c3=!da.pop()&&!da.pop(),df=c3&&c8.match(ct)||[""],e=cG.preFilter,c5=cG.filter,c7=!db&&c4!==b6;for(;(dc=df[c9])!=null&&c3;c9++){c6.push(dd=[]);if(c7){dc=" "+dc}while(dc){c3=false;if((da=cF.exec(dc))){dc=dc.slice(da[0].length);c3=dd.push({part:da.pop().replace(cU," "),captures:da})}for(de in c5){if((da=cE[de].exec(dc))&&(!e[de]||(da=e[de](da,c4,db)))){dc=dc.slice(da.shift().length);c3=dd.push({part:de,captures:da})}}if(!c3){break}}}if(!c3){cN.error(c8)}return c6}function cx(c6,c5,
 c4){var e=c5.dir,c3=ce++;if(!c6){c6=function(c7){return c7===c4}}return c5.first?function(c8,c7){while((c8=c8[e])){if(c8.nodeType===1){return c6(c8,c7)&&c8}}}:function(c9,c8){var c7,da=c3+"."+co,db=da+"."+cT;while((c9=c9[e])){if(c9.nodeType===1){if((c7=c9[cW])===db){return c9.sizset}else{if(typeof c7==="string"&&c7.indexOf(da)===0){if(c9.sizset){return c9}}else{c9[cW]=db;if(c6(c9,c8)){c9.sizset=true;return c9}c9.sizset=false}}}}}}function cv(e,c3){return e?function(c6,c5){var c4=c3(c6,c5);return c4&&e(c4===true?c6:c4,c5)}:c3}function cy(c7,c5,e){var c4,c6,c3=0;for(;(c4=c7[c3]);c3++){if(cG.relative[c4.part]){c6=cx(c6,cG.relative[c4.part],c5)}else{c4.captures.push(c5,e);c6=cv(c6,cG.filter[c4.part].apply(null,c4.captures))}}return c6}function b4(e){return function(c5,c4){var c6,c3=0;for(;(c6=e[c3]);c3++){if(c6(c5,c4)){return true}}return false}}var cb=cN.compile=function(e,c5,c3){var c8,c7,c4,c6=cl[e];if(c6&&c6.context===c5){return c6}c7=b1(e,c5,c3);for(c4=0;(c8=c7[c4]);c4++){c7[c4]=cy
 (c8,c5,c3)}c6=cl[e]=b4(c7);c6.context=c5;c6.runs=c6.dirruns=0;cu.push(e);if(cu.length>cG.cacheLength){delete cl[cu.shift()]}return c6};cN.matches=function(c3,e){return cN(c3,null,null,e)};cN.matchesSelector=function(e,c3){return cN(c3,null,null,[e]).length>0};var cV=function(c6,c3,c8,dc,db){c6=c6.replace(cU,"$1");var e,dd,c9,de,c4,c5,dg,dh,c7,da=c6.match(ct),df=c6.match(cZ),di=c3.nodeType;if(cE.POS.test(c6)){return cQ(c6,c3,c8,dc,da)}if(dc){e=cj.call(dc,0)}else{if(da&&da.length===1){if(df.length>1&&di===9&&!db&&(da=cE.ID.exec(df[0]))){c3=cG.find.ID(da[1],c3,db)[0];if(!c3){return c8}c6=c6.slice(df.shift().length)}dh=((da=cP.exec(df[0]))&&!da.index&&c3.parentNode)||c3;c7=df.pop();c5=c7.split(":not")[0];for(c9=0,de=cG.order.length;c9<de;c9++){dg=cG.order[c9];if((da=cE[dg].exec(c5))){e=cG.find[dg]((da[1]||"").replace(cs,""),dh,db);if(e==null){continue}if(c5===c7){c6=c6.slice(0,c6.length-c7.length)+c5.replace(cE[dg],"");if(!c6){cS.apply(c8,cj.call(e,0))}}break}}}}if(c6){dd=cb(c6,c3,db);c
 o=dd.dirruns++;if(e==null){e=cG.find.TAG("*",(cP.test(c6)&&c3.parentNode)||c3)}for(c9=0;(c4=e[c9]);c9++){cT=dd.runs++;if(dd(c4,c3)){c8.push(c4)}}}return c8};if(b6.querySelectorAll){(function(){var c7,c8=cV,c6=/'|\\/g,c4=/\=[\x20\t\r\n\f]*([^'"\]]*)[\x20\t\r\n\f]*\]/g,c3=[],e=[":active"],c5=b9.matchesSelector||b9.mozMatchesSelector||b9.webkitMatchesSelector||b9.oMatchesSelector||b9.msMatchesSelector;cH(function(c9){c9.innerHTML="<select><option selected></option></select>";if(!c9.querySelectorAll("[selected]").length){c3.push("\\["+cz+"*(?:checked|disabled|ismap|multiple|readonly|selected|value)")}if(!c9.querySelectorAll(":checked").length){c3.push(":checked")}});cH(function(c9){c9.innerHTML="<p test=''></p>";if(c9.querySelectorAll("[test^='']").length){c3.push("[*^$]="+cz+"*(?:\"\"|'')")}c9.innerHTML="<input type='hidden'>";if(!c9.querySelectorAll(":enabled").length){c3.push(":enabled",":disabled")}});c3=c3.length&&new RegExp(c3.join("|"));cV=function(de,da,df,dh,dg){if(!dh&&!dg&&(!
 c3||!c3.test(de))){if(da.nodeType===9){try{cS.apply(df,cj.call(da.querySelectorAll(de),0));return df}catch(dd){}}else{if(da.nodeType===1&&da.nodeName.toLowerCase()!=="object"){var dc=da.getAttribute("id"),c9=dc||cW,db=cP.test(de)&&da.parentNode||da;if(dc){c9=c9.replace(c6,"\\$&")}else{da.setAttribute("id",c9)}try{cS.apply(df,cj.call(db.querySelectorAll(de.replace(ct,"[id='"+c9+"'] $&")),0));return df}catch(dd){}finally{if(!dc){da.removeAttribute("id")}}}}}return c8(de,da,df,dh,dg)};if(c5){cH(function(da){c7=c5.call(da,"div");try{c5.call(da,"[test!='']:sizzle");e.push(cG.match.PSEUDO)}catch(c9){}});e=new RegExp(e.join("|"));cN.matchesSelector=function(da,dc){dc=dc.replace(c4,"='$1']");if(!ck(da)&&!e.test(dc)&&(!c3||!c3.test(dc))){try{var c9=c5.call(da,dc);if(c9||c7||da.document&&da.document.nodeType!==11){return c9}}catch(db){}}return cN(dc,null,null,[da]).length>0}}})()}cN.attr=bG.attr;bG.find=cN;bG.expr=cN.selectors;bG.expr[":"]=bG.expr.pseudos;bG.unique=cN.uniqueSort;bG.text=cN.ge
 tText;bG.isXMLDoc=cN.isXML;bG.contains=cN.contains})(a2);var ag=/Until$/,bq=/^(?:parents|prev(?:Until|All))/,al=/^.[^:#\[\.,]*$/,z=bG.expr.match.needsContext,bu={children:true,contents:true,next:true,prev:true};bG.fn.extend({find:function(e){var b2,bZ,b4,b5,b3,b1,b0=this;if(typeof e!=="string"){return bG(e).filter(function(){for(b2=0,bZ=b0.length;b2<bZ;b2++){if(bG.contains(b0[b2],this)){return true}}})}b1=this.pushStack("","find",e);for(b2=0,bZ=this.length;b2<bZ;b2++){b4=b1.length;bG.find(e,this[b2],b1);if(b2>0){for(b5=b4;b5<b1.length;b5++){for(b3=0;b3<b4;b3++){if(b1[b3]===b1[b5]){b1.splice(b5--,1);break}}}}}return b1},has:function(b1){var b0,bZ=bG(b1,this),e=bZ.length;return this.filter(function(){for(b0=0;b0<e;b0++){if(bG.contains(this,bZ[b0])){return true}}})},not:function(e){return this.pushStack(aM(this,e,false),"not",e)},filter:function(e){return this.pushStack(aM(this,e,true),"filter",e)},is:function(e){return !!e&&(typeof e==="string"?z.test(e)?bG(e,this.context).index(this[
 0])>=0:bG.filter(e,this).length>0:this.filter(e).length>0)},closest:function(b2,b1){var b3,b0=0,e=this.length,bZ=[],b4=z.test(b2)||typeof b2!=="string"?bG(b2,b1||this.context):0;for(;b0<e;b0++){b3=this[b0];while(b3&&b3.ownerDocument&&b3!==b1&&b3.nodeType!==11){if(b4?b4.index(b3)>-1:bG.find.matchesSelector(b3,b2)){bZ.push(b3);break}b3=b3.parentNode}}bZ=bZ.length>1?bG.unique(bZ):bZ;return this.pushStack(bZ,"closest",b2)},index:function(e){if(!e){return(this[0]&&this[0].parentNode)?this.prevAll().length:-1}if(typeof e==="string"){return bG.inArray(this[0],bG(e))}return bG.inArray(e.jquery?e[0]:e,this)},add:function(e,bZ){var b1=typeof e==="string"?bG(e,bZ):bG.makeArray(e&&e.nodeType?[e]:e),b0=bG.merge(this.get(),b1);return this.pushStack(aR(b1[0])||aR(b0[0])?b0:bG.unique(b0))},addBack:function(e){return this.add(e==null?this.prevObject:this.prevObject.filter(e))}});bG.fn.andSelf=bG.fn.addBack;function aR(e){return !e||!e.parentNode||e.parentNode.nodeType===11}function aY(bZ,e){do{bZ=bZ
 [e]}while(bZ&&bZ.nodeType!==1);return bZ}bG.each({parent:function(bZ){var e=bZ.parentNode;return e&&e.nodeType!==11?e:null},parents:function(e){return bG.dir(e,"parentNode")},parentsUntil:function(bZ,e,b0){return bG.dir(bZ,"parentNode",b0)},next:function(e){return aY(e,"nextSibling")},prev:function(e){return aY(e,"previousSibling")},nextAll:function(e){return bG.dir(e,"nextSibling")},prevAll:function(e){return bG.dir(e,"previousSibling")},nextUntil:function(bZ,e,b0){return bG.dir(bZ,"nextSibling",b0)},prevUntil:function(bZ,e,b0){return bG.dir(bZ,"previousSibling",b0)},siblings:function(e){return bG.sibling((e.parentNode||{}).firstChild,e)},children:function(e){return bG.sibling(e.firstChild)},contents:function(e){return bG.nodeName(e,"iframe")?e.contentDocument||e.contentWindow.document:bG.merge([],e.childNodes)}},function(e,bZ){bG.fn[e]=function(b2,b0){var b1=bG.map(this,bZ,b2);if(!ag.test(e)){b0=b2}if(b0&&typeof b0==="string"){b1=bG.filter(b0,b1)}b1=this.length>1&&!bu[e]?bG.unique
 (b1):b1;if(this.length>1&&bq.test(e)){b1=b1.reverse()}return this.pushStack(b1,e,a4.call(arguments).join(","))}});bG.extend({filter:function(b0,e,bZ){if(bZ){b0=":not("+b0+")"}return e.length===1?bG.find.matchesSelector(e[0],b0)?[e[0]]:[]:bG.find.matches(b0,e)},dir:function(b0,bZ,b2){var e=[],b1=b0[bZ];while(b1&&b1.nodeType!==9&&(b2===aB||b1.nodeType!==1||!bG(b1).is(b2))){if(b1.nodeType===1){e.push(b1)}b1=b1[bZ]}return e},sibling:function(b0,bZ){var e=[];for(;b0;b0=b0.nextSibling){if(b0.nodeType===1&&b0!==bZ){e.push(b0)}}return e}});function aM(b1,b0,e){b0=b0||0;if(bG.isFunction(b0)){return bG.grep(b1,function(b3,b2){var b4=!!b0.call(b3,b2,b3);return b4===e})}else{if(b0.nodeType){return bG.grep(b1,function(b3,b2){return(b3===b0)===e})}else{if(typeof b0==="string"){var bZ=bG.grep(b1,function(b2){return b2.nodeType===1});if(al.test(b0)){return bG.filter(b0,bZ,!e)}else{b0=bG.filter(b0,bZ)}}}}return bG.grep(b1,function(b3,b2){return(bG.inArray(b3,b0)>=0)===e})}function B(e){var b0=c.spli
 t("|"),bZ=e.createDocumentFragment();if(bZ.createElement){while(b0.length){bZ.createElement(b0.pop())}}return bZ}var c="abbr|article|aside|audio|bdi|canvas|data|datalist|details|figcaption|figure|footer|header|hgroup|mark|meter|nav|output|progress|section|summary|time|video",av=/ jQuery\d+="(?:null|\d+)"/g,bY=/^\s+/,ay=/<(?!area|br|col|embed|hr|img|input|link|meta|param)(([\w:]+)[^>]*)\/>/gi,p=/<([\w:]+)/,bT=/<tbody/i,J=/<|&#?\w+;/,aj=/<(?:script|style|link)/i,ap=/<(?:script|object|embed|option|style)/i,K=new RegExp("<(?:"+c+")[\\s/>]","i"),aE=/^(?:checkbox|radio)$/,bR=/checked\s*(?:[^=]|=\s*.checked.)/i,bw=/\/(java|ecma)script/i,aH=/^\s*<!(?:\[CDATA\[|\-\-)|[\]\-]{2}>\s*$/g,T={option:[1,"<select multiple='multiple'>","</select>"],legend:[1,"<fieldset>","</fieldset>"],thead:[1,"<table>","</table>"],tr:[2,"<table><tbody>","</tbody></table>"],td:[3,"<table><tbody><tr>","</tr></tbody></table>"],col:[2,"<table><tbody></tbody><colgroup>","</colgroup></table>"],area:[1,"<map>","</map>"],_
 default:[0,"",""]},aQ=B(o),l=aQ.appendChild(o.createElement("div"));T.optgroup=T.option;T.tbody=T.tfoot=T.colgroup=T.caption=T.thead;T.th=T.td;if(!bG.support.htmlSerialize){T._default=[1,"X<div>","</div>"]}bG.fn.extend({text:function(e){return bG.access(this,function(bZ){return bZ===aB?bG.text(this):this.empty().append((this[0]&&this[0].ownerDocument||o).createTextNode(bZ))},null,e,arguments.length)},wrapAll:function(e){if(bG.isFunction(e)){return this.each(function(b0){bG(this).wrapAll(e.call(this,b0))})}if(this[0]){var bZ=bG(e,this[0].ownerDocument).eq(0).clone(true);if(this[0].parentNode){bZ.insertBefore(this[0])}bZ.map(function(){var b0=this;while(b0.firstChild&&b0.firstChild.nodeType===1){b0=b0.firstChild}return b0}).append(this)}return this},wrapInner:function(e){if(bG.isFunction(e)){return this.each(function(bZ){bG(this).wrapInner(e.call(this,bZ))})}return this.each(function(){var bZ=bG(this),b0=bZ.contents();if(b0.length){b0.wrapAll(e)}else{bZ.append(e)}})},wrap:function(e){
 var bZ=bG.isFunction(e);return this.each(function(b0){bG(this).wrapAll(bZ?e.call(this,b0):e)})},unwrap:function(){return this.parent().each(function(){if(!bG.nodeName(this,"body")){bG(this).replaceWith(this.childNodes)}}).end()},append:function(){return this.domManip(arguments,true,function(e){if(this.nodeType===1||this.nodeType===11){this.appendChild(e)}})},prepend:function(){return this.domManip(arguments,true,function(e){if(this.nodeType===1||this.nodeType===11){this.insertBefore(e,this.firstChild)}})},before:function(){if(!aR(this[0])){return this.domManip(arguments,false,function(bZ){this.parentNode.insertBefore(bZ,this)})}if(arguments.length){var e=bG.clean(arguments);return this.pushStack(bG.merge(e,this),"before",this.selector)}},after:function(){if(!aR(this[0])){return this.domManip(arguments,false,function(bZ){this.parentNode.insertBefore(bZ,this.nextSibling)})}if(arguments.length){var e=bG.clean(arguments);return this.pushStack(bG.merge(this,e),"after",this.selector)}},re
 move:function(e,b1){var b0,bZ=0;for(;(b0=this[bZ])!=null;bZ++){if(!e||bG.filter(e,[b0]).length){if(!b1&&b0.nodeType===1){bG.cleanData(b0.getElementsByTagName("*"));bG.cleanData([b0])}if(b0.parentNode){b0.parentNode.removeChild(b0)}}}return this},empty:function(){var bZ,e=0;for(;(bZ=this[e])!=null;e++){if(bZ.nodeType===1){bG.cleanData(bZ.getElementsByTagName("*"))}while(bZ.firstChild){bZ.removeChild(bZ.firstChild)}}return this},clone:function(bZ,e){bZ=bZ==null?false:bZ;e=e==null?bZ:e;return this.map(function(){return bG.clone(this,bZ,e)})},html:function(e){return bG.access(this,function(b2){var b1=this[0]||{},b0=0,bZ=this.length;if(b2===aB){return b1.nodeType===1?b1.innerHTML.replace(av,""):aB}if(typeof b2==="string"&&!aj.test(b2)&&(bG.support.htmlSerialize||!K.test(b2))&&(bG.support.leadingWhitespace||!bY.test(b2))&&!T[(p.exec(b2)||["",""])[1].toLowerCase()]){b2=b2.replace(ay,"<$1></$2>");try{for(;b0<bZ;b0++){b1=this[b0]||{};if(b1.nodeType===1){bG.cleanData(b1.getElementsByTagName("
 *"));b1.innerHTML=b2}}b1=0}catch(b3){}}if(b1){this.empty().append(b2)}},null,e,arguments.length)},replaceWith:function(e){if(!aR(this[0])){if(bG.isFunction(e)){return this.each(function(b1){var b0=bG(this),bZ=b0.html();b0.replaceWith(e.call(this,b1,bZ))})}if(typeof e!=="string"){e=bG(e).detach()}return this.each(function(){var b0=this.nextSibling,bZ=this.parentNode;bG(this).remove();if(b0){bG(b0).before(e)}else{bG(bZ).append(e)}})}return this.length?this.pushStack(bG(bG.isFunction(e)?e():e),"replaceWith",e):this},detach:function(e){return this.remove(e,true)},domManip:function(b4,b8,b7){b4=[].concat.apply([],b4);var b0,b2,b3,b6,b1=0,b5=b4[0],bZ=[],e=this.length;if(!bG.support.checkClone&&e>1&&typeof b5==="string"&&bR.test(b5)){return this.each(function(){bG(this).domManip(b4,b8,b7)})}if(bG.isFunction(b5)){return this.each(function(ca){var b9=bG(this);b4[0]=b5.call(this,ca,b8?b9.html():aB);b9.domManip(b4,b8,b7)})}if(this[0]){b0=bG.buildFragment(b4,this,bZ);b3=b0.fragment;b2=b3.firstC
 hild;if(b3.childNodes.length===1){b3=b2}if(b2){b8=b8&&bG.nodeName(b2,"tr");for(b6=b0.cacheable||e-1;b1<e;b1++){b7.call(b8&&bG.nodeName(this[b1],"table")?y(this[b1],"tbody"):this[b1],b1===b6?b3:bG.clone(b3,true,true))}}b3=b2=null;if(bZ.length){bG.each(bZ,function(b9,ca){if(ca.src){if(bG.ajax){bG.ajax({url:ca.src,type:"GET",dataType:"script",async:false,global:false,"throws":true})}else{bG.error("no ajax")}}else{bG.globalEval((ca.text||ca.textContent||ca.innerHTML||"").replace(aH,""))}if(ca.parentNode){ca.parentNode.removeChild(ca)}})}}return this}});function y(bZ,e){return bZ.getElementsByTagName(e)[0]||bZ.appendChild(bZ.ownerDocument.createElement(e))}function ao(b5,bZ){if(bZ.nodeType!==1||!bG.hasData(b5)){return}var b2,b1,e,b4=bG._data(b5),b3=bG._data(bZ,b4),b0=b4.events;if(b0){delete b3.handle;b3.events={};for(b2 in b0){for(b1=0,e=b0[b2].length;b1<e;b1++){bG.event.add(bZ,b2,b0[b2][b1])}}}if(b3.data){b3.data=bG.extend({},b3.data)}}function G(bZ,e){var b0;if(e.nodeType!==1){return}i
 f(e.clearAttributes){e.clearAttributes()}if(e.mergeAttributes){e.mergeAttributes(bZ)}b0=e.nodeName.toLowerCase();if(b0==="object"){if(e.parentNode){e.outerHTML=bZ.outerHTML}if(bG.support.html5Clone&&(bZ.innerHTML&&!bG.trim(e.innerHTML))){e.innerHTML=bZ.innerHTML}}else{if(b0==="input"&&aE.test(bZ.type)){e.defaultChecked=e.checked=bZ.checked;if(e.value!==bZ.value){e.value=bZ.value}}else{if(b0==="option"){e.selected=bZ.defaultSelected}else{if(b0==="input"||b0==="textarea"){e.defaultValue=bZ.defaultValue}else{if(b0==="script"&&e.text!==bZ.text){e.text=bZ.text}}}}}e.removeAttribute(bG.expando)}bG.buildFragment=function(b1,b2,bZ){var b0,e,b3,b4=b1[0];b2=b2||o;b2=(b2[0]||b2).ownerDocument||b2[0]||b2;if(typeof b2.createDocumentFragment==="undefined"){b2=o}if(b1.length===1&&typeof b4==="string"&&b4.length<512&&b2===o&&b4.charAt(0)==="<"&&!ap.test(b4)&&(bG.support.checkClone||!bR.test(b4))&&(bG.support.html5Clone||!K.test(b4))){e=true;b0=bG.fragments[b4];b3=b0!==aB}if(!b0){b0=b2.createDocumen
 tFragment();bG.clean(b1,b2,b0,bZ);if(e){bG.fragments[b4]=b3&&b0}}return{fragment:b0,cacheable:e}};bG.fragments={};bG.each({appendTo:"append",prependTo:"prepend",insertBefore:"before",insertAfter:"after",replaceAll:"replaceWith"},function(e,bZ){bG.fn[e]=function(b0){var b2,b4=0,b3=[],b6=bG(b0),b1=b6.length,b5=this.length===1&&this[0].parentNode;if((b5==null||b5&&b5.nodeType===11&&b5.childNodes.length===1)&&b1===1){b6[bZ](this[0]);return this}else{for(;b4<b1;b4++){b2=(b4>0?this.clone(true):this).get();bG(b6[b4])[bZ](b2);b3=b3.concat(b2)}return this.pushStack(b3,e,b6.selector)}}});function m(e){if(typeof e.getElementsByTagName!=="undefined"){return e.getElementsByTagName("*")}else{if(typeof e.querySelectorAll!=="undefined"){return e.querySelectorAll("*")}else{return[]}}}function bS(e){if(aE.test(e.type)){e.defaultChecked=e.checked}}bG.extend({clone:function(b2,b4,b0){var e,bZ,b1,b3;if(bG.support.html5Clone||bG.isXMLDoc(b2)||!K.test("<"+b2.nodeName+">")){b3=b2.cloneNode(true)}else{l.inn
 erHTML=b2.outerHTML;l.removeChild(b3=l.firstChild)}if((!bG.support.noCloneEvent||!bG.support.noCloneChecked)&&(b2.nodeType===1||b2.nodeType===11)&&!bG.isXMLDoc(b2)){G(b2,b3);e=m(b2);bZ=m(b3);for(b1=0;e[b1];++b1){if(bZ[b1]){G(e[b1],bZ[b1])}}}if(b4){ao(b2,b3);if(b0){e=m(b2);bZ=m(b3);for(b1=0;e[b1];++b1){ao(e[b1],bZ[b1])}}}e=bZ=null;return b3},clean:function(cb,b0,e,b1){var b7,b3,ca,cf,b4,ce,b5,b2,bZ,b9,cd,b6,b8=0,cc=[];if(!b0||typeof b0.createDocumentFragment==="undefined"){b0=o}for(b3=b0===o&&aQ;(ca=cb[b8])!=null;b8++){if(typeof ca==="number"){ca+=""}if(!ca){continue}if(typeof ca==="string"){if(!J.test(ca)){ca=b0.createTextNode(ca)}else{b3=b3||B(b0);b5=b5||b3.appendChild(b0.createElement("div"));ca=ca.replace(ay,"<$1></$2>");cf=(p.exec(ca)||["",""])[1].toLowerCase();b4=T[cf]||T._default;ce=b4[0];b5.innerHTML=b4[1]+ca+b4[2];while(ce--){b5=b5.lastChild}if(!bG.support.tbody){b2=bT.test(ca);bZ=cf==="table"&&!b2?b5.firstChild&&b5.firstChild.childNodes:b4[1]==="<table>"&&!b2?b5.childNodes:
 [];for(b7=bZ.length-1;b7>=0;--b7){if(bG.nodeName(bZ[b7],"tbody")&&!bZ[b7].childNodes.length){bZ[b7].parentNode.removeChild(bZ[b7])}}}if(!bG.support.leadingWhitespace&&bY.test(ca)){b5.insertBefore(b0.createTextNode(bY.exec(ca)[0]),b5.firstChild)}ca=b5.childNodes;b5=b3.lastChild}}if(ca.nodeType){cc.push(ca)}else{cc=bG.merge(cc,ca)}}if(b5){b3.removeChild(b5);ca=b5=b3=null}if(!bG.support.appendChecked){for(b8=0;(ca=cc[b8])!=null;b8++){if(bG.nodeName(ca,"input")){bS(ca)}else{if(typeof ca.getElementsByTagName!=="undefined"){bG.grep(ca.getElementsByTagName("input"),bS)}}}}if(e){cd=function(cg){if(!cg.type||bw.test(cg.type)){return b1?b1.push(cg.parentNode?cg.parentNode.removeChild(cg):cg):e.appendChild(cg)}};for(b8=0;(ca=cc[b8])!=null;b8++){if(!(bG.nodeName(ca,"script")&&cd(ca))){e.appendChild(ca);if(typeof ca.getElementsByTagName!=="undefined"){b6=bG.grep(bG.merge([],ca.getElementsByTagName("script")),cd);cc.splice.apply(cc,[b8+1,0].concat(b6));b8+=b6.length}}}}return cc},cleanData:functi
 on(bZ,b7){var b2,b0,b1,b6,b3=0,b8=bG.expando,e=bG.cache,b4=bG.support.deleteExpando,b5=bG.event.special;for(;(b1=bZ[b3])!=null;b3++){if(b7||bG.acceptData(b1)){b0=b1[b8];b2=b0&&e[b0];if(b2){if(b2.events){for(b6 in b2.events){if(b5[b6]){bG.event.remove(b1,b6)}else{bG.removeEvent(b1,b6,b2.handle)}}}if(e[b0]){delete e[b0];if(b4){delete b1[b8]}else{if(b1.removeAttribute){b1.removeAttribute(b8)}else{b1[b8]=null}}bG.deletedIds.push(b0)}}}}}});(function(){var e,bZ;bG.uaMatch=function(b1){b1=b1.toLowerCase();var b0=/(chrome)[ \/]([\w.]+)/.exec(b1)||/(webkit)[ \/]([\w.]+)/.exec(b1)||/(opera)(?:.*version|)[ \/]([\w.]+)/.exec(b1)||/(msie) ([\w.]+)/.exec(b1)||b1.indexOf("compatible")<0&&/(mozilla)(?:.*? rv:([\w.]+)|)/.exec(b1)||[];return{browser:b0[1]||"",version:b0[2]||"0"}};e=bG.uaMatch(d.userAgent);bZ={};if(e.browser){bZ[e.browser]=true;bZ.version=e.version}if(bZ.webkit){bZ.safari=true}bG.browser=bZ;bG.sub=function(){function b0(b3,b4){return new b0.fn.init(b3,b4)}bG.extend(true,b0,this);b0.s
 uperclass=this;b0.fn=b0.prototype=this();b0.fn.constructor=b0;b0.sub=this.sub;b0.fn.init=function b2(b3,b4){if(b4&&b4 instanceof bG&&!(b4 instanceof b0)){b4=b0(b4)}return bG.fn.init.call(this,b3,b4,b1)};b0.fn.init.prototype=b0.fn;var b1=b0(o);return b0}})();var F,az,aW,be=/alpha\([^)]*\)/i,aS=/opacity=([^)]*)/,bk=/^(top|right|bottom|left)$/,aZ=/^margin/,a8=new RegExp("^("+bx+")(.*)$","i"),W=new RegExp("^("+bx+")(?!px)[a-z%]+$","i"),S=new RegExp("^([-+])=("+bx+")","i"),bh={},a9={position:"absolute",visibility:"hidden",display:"block"},bA={letterSpacing:0,fontWeight:400,lineHeight:1},bQ=["Top","Right","Bottom","Left"],ar=["Webkit","O","Moz","ms"],aJ=bG.fn.toggle;function b(b1,bZ){if(bZ in b1){return bZ}var b2=bZ.charAt(0).toUpperCase()+bZ.slice(1),e=bZ,b0=ar.length;while(b0--){bZ=ar[b0]+b2;if(bZ in b1){return bZ}}return e}function Q(bZ,e){bZ=e||bZ;return bG.css(bZ,"display")==="none"||!bG.contains(bZ.ownerDocument,bZ)}function t(b3,e){var b2,b4,bZ=[],b0=0,b1=b3.length;for(;b0<b1;b0++)
 {b2=b3[b0];if(!b2.style){continue}bZ[b0]=bG._data(b2,"olddisplay");if(e){if(!bZ[b0]&&b2.style.display==="none"){b2.style.display=""}if(b2.style.display===""&&Q(b2)){bZ[b0]=bG._data(b2,"olddisplay",bC(b2.nodeName))}}else{b4=F(b2,"display");if(!bZ[b0]&&b4!=="none"){bG._data(b2,"olddisplay",b4)}}}for(b0=0;b0<b1;b0++){b2=b3[b0];if(!b2.style){continue}if(!e||b2.style.display==="none"||b2.style.display===""){b2.style.display=e?bZ[b0]||"":"none"}}return b3}bG.fn.extend({css:function(e,bZ){return bG.access(this,function(b1,b0,b2){return b2!==aB?bG.style(b1,b0,b2):bG.css(b1,b0)},e,bZ,arguments.length>1)},show:function(){return t(this,true)},hide:function(){return t(this)},toggle:function(b0,bZ){var e=typeof b0==="boolean";if(bG.isFunction(b0)&&bG.isFunction(bZ)){return aJ.apply(this,arguments)}return this.each(function(){if(e?b0:Q(this)){bG(this).show()}else{bG(this).hide()}})}});bG.extend({cssHooks:{opacity:{get:function(b0,bZ){if(bZ){var e=F(b0,"opacity");return e===""?"1":e}}}},cssNumber:
 {fillOpacity:true,fontWeight:true,lineHeight:true,opacity:true,orphans:true,widows:true,zIndex:true,zoom:true},cssProps:{"float":bG.support.cssFloat?"cssFloat":"styleFloat"},style:function(b1,b0,b7,b2){if(!b1||b1.nodeType===3||b1.nodeType===8||!b1.style){return}var b5,b6,b8,b3=bG.camelCase(b0),bZ=b1.style;b0=bG.cssProps[b3]||(bG.cssProps[b3]=b(bZ,b3));b8=bG.cssHooks[b0]||bG.cssHooks[b3];if(b7!==aB){b6=typeof b7;if(b6==="string"&&(b5=S.exec(b7))){b7=(b5[1]+1)*b5[2]+parseFloat(bG.css(b1,b0));b6="number"}if(b7==null||b6==="number"&&isNaN(b7)){return}if(b6==="number"&&!bG.cssNumber[b3]){b7+="px"}if(!b8||!("set" in b8)||(b7=b8.set(b1,b7,b2))!==aB){try{bZ[b0]=b7}catch(b4){}}}else{if(b8&&"get" in b8&&(b5=b8.get(b1,false,b2))!==aB){return b5}return bZ[b0]}},css:function(b4,b2,b3,bZ){var b5,b1,e,b0=bG.camelCase(b2);b2=bG.cssProps[b0]||(bG.cssProps[b0]=b(b4.style,b0));e=bG.cssHooks[b2]||bG.cssHooks[b0];if(e&&"get" in e){b5=e.get(b4,true,bZ)}if(b5===aB){b5=F(b4,b2)}if(b5==="normal"&&b2 in bA){
 b5=bA[b2]}if(b3||bZ!==aB){b1=parseFloat(b5);return b3||bG.isNumeric(b1)?b1||0:b5}return b5},swap:function(b2,b1,b3){var b0,bZ,e={};for(bZ in b1){e[bZ]=b2.style[bZ];b2.style[bZ]=b1[bZ]}b0=b3.call(b2);for(bZ in b1){b2.style[bZ]=e[bZ]}return b0}});if(a2.getComputedStyle){F=function(b5,bZ){var e,b2,b1,b4,b3=getComputedStyle(b5,null),b0=b5.style;if(b3){e=b3[bZ];if(e===""&&!bG.contains(b5.ownerDocument.documentElement,b5)){e=bG.style(b5,bZ)}if(W.test(e)&&aZ.test(bZ)){b2=b0.width;b1=b0.minWidth;b4=b0.maxWidth;b0.minWidth=b0.maxWidth=b0.width=e;e=b3.width;b0.width=b2;b0.minWidth=b1;b0.maxWidth=b4}}return e}}else{if(o.documentElement.currentStyle){F=function(b2,b0){var b3,e,bZ=b2.currentStyle&&b2.currentStyle[b0],b1=b2.style;if(bZ==null&&b1&&b1[b0]){bZ=b1[b0]}if(W.test(bZ)&&!bk.test(b0)){b3=b1.left;e=b2.runtimeStyle&&b2.runtimeStyle.left;if(e){b2.runtimeStyle.left=b2.currentStyle.left}b1.left=b0==="fontSize"?"1em":bZ;bZ=b1.pixelLeft+"px";b1.left=b3;if(e){b2.runtimeStyle.left=e}}return bZ==="
 "?"auto":bZ}}}function aG(e,b0,b1){var bZ=a8.exec(b0);return bZ?Math.max(0,bZ[1]-(b1||0))+(bZ[2]||"px"):b0}function at(b1,bZ,e,b3){var b0=e===(b3?"border":"content")?4:bZ==="width"?1:0,b2=0;for(;b0<4;b0+=2){if(e==="margin"){b2+=bG.css(b1,e+bQ[b0],true)}if(b3){if(e==="content"){b2-=parseFloat(F(b1,"padding"+bQ[b0]))||0}if(e!=="margin"){b2-=parseFloat(F(b1,"border"+bQ[b0]+"Width"))||0}}else{b2+=parseFloat(F(b1,"padding"+bQ[b0]))||0;if(e!=="padding"){b2+=parseFloat(F(b1,"border"+bQ[b0]+"Width"))||0}}}return b2}function v(b1,bZ,e){var b2=bZ==="width"?b1.offsetWidth:b1.offsetHeight,b0=true,b3=bG.support.boxSizing&&bG.css(b1,"boxSizing")==="border-box";if(b2<=0){b2=F(b1,bZ);if(b2<0||b2==null){b2=b1.style[bZ]}if(W.test(b2)){return b2}b0=b3&&(bG.support.boxSizingReliable||b2===b1.style[bZ]);b2=parseFloat(b2)||0}return(b2+at(b1,bZ,e||(b3?"border":"content"),b0))+"px"}function bC(b0){if(bh[b0]){return bh[b0]}var e=bG("<"+b0+">").appendTo(o.body),bZ=e.css("display");e.remove();if(bZ==="none"||
 bZ===""){az=o.body.appendChild(az||bG.extend(o.createElement("iframe"),{frameBorder:0,width:0,height:0}));if(!aW||!az.createElement){aW=(az.contentWindow||az.contentDocument).document;aW.write("<!doctype html><html><body>");aW.close()}e=aW.body.appendChild(aW.createElement(b0));bZ=F(e,"display");o.body.removeChild(az)}bh[b0]=bZ;return bZ}bG.each(["height","width"],function(bZ,e){bG.cssHooks[e]={get:function(b2,b1,b0){if(b1){if(b2.offsetWidth!==0||F(b2,"display")!=="none"){return v(b2,e,b0)}else{return bG.swap(b2,a9,function(){return v(b2,e,b0)})}}},set:function(b1,b2,b0){return aG(b1,b2,b0?at(b1,e,b0,bG.support.boxSizing&&bG.css(b1,"boxSizing")==="border-box"):0)}}});if(!bG.support.opacity){bG.cssHooks.opacity={get:function(bZ,e){return aS.test((e&&bZ.currentStyle?bZ.currentStyle.filter:bZ.style.filter)||"")?(0.01*parseFloat(RegExp.$1))+"":e?"1":""},set:function(b2,b3){var b1=b2.style,bZ=b2.currentStyle,e=bG.isNumeric(b3)?"alpha(opacity="+b3*100+")":"",b0=bZ&&bZ.filter||b1.filter||"
 ";b1.zoom=1;if(b3>=1&&bG.trim(b0.replace(be,""))===""&&b1.removeAttribute){b1.removeAttribute("filter");if(bZ&&!bZ.filter){return}}b1.filter=be.test(b0)?b0.replace(be,e):b0+" "+e}}}bG(function(){if(!bG.support.reliableMarginRight){bG.cssHooks.marginRight={get:function(bZ,e){return bG.swap(bZ,{display:"inline-block"},function(){if(e){return F(bZ,"marginRight")}})}}}if(!bG.support.pixelPosition&&bG.fn.position){bG.each(["top","left"],function(e,bZ){bG.cssHooks[bZ]={get:function(b2,b1){if(b1){var b0=F(b2,bZ);return W.test(b0)?bG(b2).position()[bZ]+"px":b0}}}})}});if(bG.expr&&bG.expr.filters){bG.expr.filters.hidden=function(e){return(e.offsetWidth===0&&e.offsetHeight===0)||(!bG.support.reliableHiddenOffsets&&((e.style&&e.style.display)||F(e,"display"))==="none")};bG.expr.filters.visible=function(e){return !bG.expr.filters.hidden(e)}}bG.each({margin:"",padding:"",border:"Width"},function(e,bZ){bG.cssHooks[e+bZ]={expand:function(b2){var b1,b3=typeof b2==="string"?b2.split(" "):[b2],b0={};
 for(b1=0;b1<4;b1++){b0[e+bQ[b1]+bZ]=b3[b1]||b3[b1-2]||b3[0]}return b0}};if(!aZ.test(e)){bG.cssHooks[e+bZ].set=aG}});var bs=/%20/g,aP=/\[\]$/,U=/\r?\n/g,bz=/^(?:color|date|datetime|datetime-local|email|hidden|month|number|password|range|search|tel|text|time|url|week)$/i,aD=/^(?:select|textarea)/i;bG.fn.extend({serialize:function(){return bG.param(this.serializeArray())},serializeArray:function(){return this.map(function(){return this.elements?bG.makeArray(this.elements):this}).filter(function(){return this.name&&!this.disabled&&(this.checked||aD.test(this.nodeName)||bz.test(this.type))}).map(function(e,bZ){var b0=bG(this).val();return b0==null?null:bG.isArray(b0)?bG.map(b0,function(b2,b1){return{name:bZ.name,value:b2.replace(U,"\r\n")}}):{name:bZ.name,value:b0.replace(U,"\r\n")}}).get()}});bG.param=function(e,b0){var b1,bZ=[],b2=function(b3,b4){b4=bG.isFunction(b4)?b4():(b4==null?"":b4);bZ[bZ.length]=encodeURIComponent(b3)+"="+encodeURIComponent(b4)};if(b0===aB){b0=bG.ajaxSettings&&b
 G.ajaxSettings.traditional}if(bG.isArray(e)||(e.jquery&&!bG.isPlainObject(e))){bG.each(e,function(){b2(this.name,this.value)})}else{for(b1 in e){k(b1,e[b1],b0,b2)}}return bZ.join("&").replace(bs,"+")};function k(b0,b2,bZ,b1){var e;if(bG.isArray(b2)){bG.each(b2,function(b4,b3){if(bZ||aP.test(b0)){b1(b0,b3)}else{k(b0+"["+(typeof b3==="object"?b4:"")+"]",b3,bZ,b1)}})}else{if(!bZ&&bG.type(b2)==="object"){for(e in b2){k(b0+"["+e+"]",b2[e],bZ,b1)}}else{b1(b0,b2)}}}var Y,bX,an=/#.*$/,ad=/^(.*?):[ \t]*([^\r\n]*)\r?$/mg,C=/^(?:about|app|app\-storage|.+\-extension|file|res|widget):$/,s=/^(?:GET|HEAD)$/,aC=/^\/\//,bN=/\?/,g=/<script\b[^<]*(?:(?!<\/script>)<[^<]*)*<\/script>/gi,P=/([?&])_=[^&]*/,aT=/^([\w\+\.\-]+:)(?:\/\/([^\/?#:]*)(?::(\d+)|)|)/,bW=bG.fn.load,w={},a6={},aX=["*/"]+["*"];try{Y=aI.href}catch(bd){Y=o.createElement("a");Y.href="";Y=Y.href}bX=aT.exec(Y.toLowerCase())||[];function bI(e){return function(b2,b4){if(typeof b2!=="string"){b4=b2;b2="*"}var bZ,b5,b6,b1=b2.toLowerCase().spli
 t(aV),b0=0,b3=b1.length;if(bG.isFunction(b4)){for(;b0<b3;b0++){bZ=b1[b0];b6=/^\+/.test(bZ);if(b6){bZ=bZ.substr(1)||"*"}b5=e[bZ]=e[bZ]||[];b5[b6?"unshift":"push"](b4)}}}}function r(bZ,b8,b3,b6,b5,b1){b5=b5||b8.dataTypes[0];b1=b1||{};b1[b5]=true;var b7,b4=bZ[b5],b0=0,e=b4?b4.length:0,b2=(bZ===w);for(;b0<e&&(b2||!b7);b0++){b7=b4[b0](b8,b3,b6);if(typeof b7==="string"){if(!b2||b1[b7]){b7=aB}else{b8.dataTypes.unshift(b7);b7=r(bZ,b8,b3,b6,b7,b1)}}}if((b2||!b7)&&!b1["*"]){b7=r(bZ,b8,b3,b6,"*",b1)}return b7}function u(b0,b1){var bZ,e,b2=bG.ajaxSettings.flatOptions||{};for(bZ in b1){if(b1[bZ]!==aB){(b2[bZ]?b0:(e||(e={})))[bZ]=b1[bZ]}}if(e){bG.extend(true,b0,e)}}bG.fn.load=function(b1,b4,b5){if(typeof b1!=="string"&&bW){return bW.apply(this,arguments)}if(!this.length){return this}var e,b2,b0,bZ=this,b3=b1.indexOf(" ");if(b3>=0){e=b1.slice(b3,b1.length);b1=b1.slice(0,b3)}if(bG.isFunction(b4)){b5=b4;b4=aB}else{if(typeof b4==="object"){b2="POST"}}bG.ajax({url:b1,type:b2,dataType:"html",data:b4,co
 mplete:function(b7,b6){if(b5){bZ.each(b5,b0||[b7.responseText,b6,b7])}}}).done(function(b6){b0=arguments;bZ.html(e?bG("<div>").append(b6.replace(g,"")).find(e):b6)});return this};bG.each("ajaxStart ajaxStop ajaxComplete ajaxError ajaxSuccess ajaxSend".split(" "),function(e,bZ){bG.fn[bZ]=function(b0){return this.on(bZ,b0)}});bG.each(["get","post"],function(e,bZ){bG[bZ]=function(b0,b2,b3,b1){if(bG.isFunction(b2)){b1=b1||b3;b3=b2;b2=aB}return bG.ajax({type:bZ,url:b0,data:b2,success:b3,dataType:b1})}});bG.extend({getScript:function(e,bZ){return bG.get(e,aB,bZ,"script")},getJSON:function(e,bZ,b0){return bG.get(e,bZ,b0,"json")},ajaxSetup:function(bZ,e){if(e){u(bZ,bG.ajaxSettings)}else{e=bZ;bZ=bG.ajaxSettings}u(bZ,e);return bZ},ajaxSettings:{url:Y,isLocal:C.test(bX[1]),global:true,type:"GET",contentType:"application/x-www-form-urlencoded; charset=UTF-8",processData:true,async:true,accepts:{xml:"application/xml, text/xml",html:"text/html",text:"text/plain",json:"application/json, text/javas
 cript","*":aX},contents:{xml:/xml/,html:/html/,json:/json/},responseFields:{xml:"responseXML",text:"responseText"},converters:{"* text":a2.String,"text html":true,"text json":bG.parseJSON,"text xml":bG.parseXML},flatOptions:{context:true,url:true}},ajaxPrefilter:bI(w),ajaxTransport:bI(a6),ajax:function(b4,b1){if(typeof b4==="object"){b1=b4;b4=aB}b1=b1||{};var b7,cl,b2,cg,b9,cd,b0,cf,b8=bG.ajaxSetup({},b1),cn=b8.context||b8,cb=cn!==b8&&(cn.nodeType||cn instanceof bG)?bG(cn):bG.event,cm=bG.Deferred(),ci=bG.Callbacks("once memory"),b5=b8.statusCode||{},cc={},cj={},b3=0,b6="canceled",ce={readyState:0,setRequestHeader:function(co,cp){if(!b3){var e=co.toLowerCase();co=cj[e]=cj[e]||co;cc[co]=cp}return this},getAllResponseHeaders:function(){return b3===2?cl:null},getResponseHeader:function(co){var e;if(b3===2){if(!b2){b2={};while((e=ad.exec(cl))){b2[e[1].toLowerCase()]=e[2]}}e=b2[co.toLowerCase()]}return e===aB?null:e},overrideMimeType:function(e){if(!b3){b8.mimeType=e}return this},abort:fu
 nction(e){e=e||b6;if(cg){cg.abort(e)}ca(0,e);return this}};function ca(cs,co,ct,cq){var e,cw,cu,cr,cv,cp=co;if(b3===2){return}b3=2;if(b9){clearTimeout(b9)}cg=aB;cl=cq||"";ce.readyState=cs>0?4:0;if(ct){cr=h(b8,ce,ct)}if(cs>=200&&cs<300||cs===304){if(b8.ifModified){cv=ce.getResponseHeader("Last-Modified");if(cv){bG.lastModified[b7]=cv}cv=ce.getResponseHeader("Etag");if(cv){bG.etag[b7]=cv}}if(cs===304){cp="notmodified";e=true}else{e=ae(b8,cr);cp=e.state;cw=e.data;cu=e.error;e=!cu}}else{cu=cp;if(!cp||cs){cp="error";if(cs<0){cs=0}}}ce.status=cs;ce.statusText=""+(co||cp);if(e){cm.resolveWith(cn,[cw,cp,ce])}else{cm.rejectWith(cn,[ce,cp,cu])}ce.statusCode(b5);b5=aB;if(b0){cb.trigger("ajax"+(e?"Success":"Error"),[ce,b8,e?cw:cu])}ci.fireWith(cn,[ce,cp]);if(b0){cb.trigger("ajaxComplete",[ce,b8]);if(!(--bG.active)){bG.event.trigger("ajaxStop")}}}cm.promise(ce);ce.success=ce.done;ce.error=ce.fail;ce.complete=ci.add;ce.statusCode=function(co){if(co){var e;if(b3<2){for(e in co){b5[e]=[b5[e],co[e]]
 }}else{e=co[ce.status];ce.always(e)}}return this};b8.url=((b4||b8.url)+"").replace(an,"").replace(aC,bX[1]+"//");b8.dataTypes=bG.trim(b8.dataType||"*").toLowerCase().split(aV);if(b8.crossDomain==null){cd=aT.exec(b8.url.toLowerCase());b8.crossDomain=!!(cd&&(cd[1]!=bX[1]||cd[2]!=bX[2]||(cd[3]||(cd[1]==="http:"?80:443))!=(bX[3]||(bX[1]==="http:"?80:443))))}if(b8.data&&b8.processData&&typeof b8.data!=="string"){b8.data=bG.param(b8.data,b8.traditional)}r(w,b8,b1,ce);if(b3===2){return ce}b0=b8.global;b8.type=b8.type.toUpperCase();b8.hasContent=!s.test(b8.type);if(b0&&bG.active++===0){bG.event.trigger("ajaxStart")}if(!b8.hasContent){if(b8.data){b8.url+=(bN.test(b8.url)?"&":"?")+b8.data;delete b8.data}b7=b8.url;if(b8.cache===false){var bZ=bG.now(),ck=b8.url.replace(P,"$1_="+bZ);b8.url=ck+((ck===b8.url)?(bN.test(b8.url)?"&":"?")+"_="+bZ:"")}}if(b8.data&&b8.hasContent&&b8.contentType!==false||b1.contentType){ce.setRequestHeader("Content-Type",b8.contentType)}if(b8.ifModified){b7=b7||b8.url;if
 (bG.lastModified[b7]){ce.setRequestHeader("If-Modified-Since",bG.lastModified[b7])}if(bG.etag[b7]){ce.setRequestHeader("If-None-Match",bG.etag[b7])}}ce.setRequestHeader("Accept",b8.dataTypes[0]&&b8.accepts[b8.dataTypes[0]]?b8.accepts[b8.dataTypes[0]]+(b8.dataTypes[0]!=="*"?", "+aX+"; q=0.01":""):b8.accepts["*"]);for(cf in b8.headers){ce.setRequestHeader(cf,b8.headers[cf])}if(b8.beforeSend&&(b8.beforeSend.call(cn,ce,b8)===false||b3===2)){return ce.abort()}b6="abort";for(cf in {success:1,error:1,complete:1}){ce[cf](b8[cf])}cg=r(a6,b8,b1,ce);if(!cg){ca(-1,"No Transport")}else{ce.readyState=1;if(b0){cb.trigger("ajaxSend",[ce,b8])}if(b8.async&&b8.timeout>0){b9=setTimeout(function(){ce.abort("timeout")},b8.timeout)}try{b3=1;cg.send(cc,ca)}catch(ch){if(b3<2){ca(-1,ch)}else{throw ch}}}return ce},active:0,lastModified:{},etag:{}});function h(b7,b6,b3){var b2,b4,b1,e,bZ=b7.contents,b5=b7.dataTypes,b0=b7.responseFields;for(b4 in b0){if(b4 in b3){b6[b0[b4]]=b3[b4]}}while(b5[0]==="*"){b5.shift()
 ;if(b2===aB){b2=b7.mimeType||b6.getResponseHeader("content-type")}}if(b2){for(b4 in bZ){if(bZ[b4]&&bZ[b4].test(b2)){b5.unshift(b4);break}}}if(b5[0] in b3){b1=b5[0]}else{for(b4 in b3){if(!b5[0]||b7.converters[b4+" "+b5[0]]){b1=b4;break}if(!e){e=b4}}b1=b1||e}if(b1){if(b1!==b5[0]){b5.unshift(b1)}return b3[b1]}}function ae(b9,b1){var b7,bZ,b5,b3,b6=b9.dataTypes.slice(),b0=b6[0],b8={},b2=0;if(b9.dataFilter){b1=b9.dataFilter(b1,b9.dataType)}if(b6[1]){for(b7 in b9.converters){b8[b7.toLowerCase()]=b9.converters[b7]}}for(;(b5=b6[++b2]);){if(b5!=="*"){if(b0!=="*"&&b0!==b5){b7=b8[b0+" "+b5]||b8["* "+b5];if(!b7){for(bZ in b8){b3=bZ.split(" ");if

<TRUNCATED>

[43/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/css/apache-maven-fluido-1.7.min.css
----------------------------------------------------------------------
diff --git a/site/current-book/css/apache-maven-fluido-1.7.min.css b/site/current-book/css/apache-maven-fluido-1.7.min.css
new file mode 100644
index 0000000..b436baf
--- /dev/null
+++ b/site/current-book/css/apache-maven-fluido-1.7.min.css
@@ -0,0 +1,17 @@
+/*!
+ * Bootstrap v2.3.2
+ *
+ * Copyright 2013 Twitter, Inc
+ * Licensed under the Apache License v2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Designed and built with all the love in the world by @mdo and @fat.
+ */.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;line-height:0;content:""}.clearfix:after{clear:both}.hide-text{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.input-block-level{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}article,aside,details,figcaption,figure,footer,header,hgroup,nav,section{display:block}audio,canvas,video{display:inline-block;*display:inline;*zoom:1}audio:not([controls]){display:none}html{font-size:100%;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}a:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}a:hover,a:active{outline:0}sub,sup{position:relative;font-size:75%;line-height:0;vertical-align:baseline}sup{top:-0.5em}sub{bottom:-0.25em}img{width:auto\9;height:auto;max-width:100%;vertical-align:middle;border:0;-ms-interpolation-mode:bicubic}#map_canvas img,.google-maps img{max-widt
 h:none}button,input,select,textarea{margin:0;font-size:100%;vertical-align:middle}button,input{*overflow:visible;line-height:normal}button::-moz-focus-inner,input::-moz-focus-inner{padding:0;border:0}button,html input[type="button"],input[type="reset"],input[type="submit"]{cursor:pointer;-webkit-appearance:button}label,select,button,input[type="button"],input[type="reset"],input[type="submit"],input[type="radio"],input[type="checkbox"]{cursor:pointer}input[type="search"]{-webkit-box-sizing:content-box;-moz-box-sizing:content-box;box-sizing:content-box;-webkit-appearance:textfield}input[type="search"]::-webkit-search-decoration,input[type="search"]::-webkit-search-cancel-button{-webkit-appearance:none}textarea{overflow:auto;vertical-align:top}@media print{*{color:#000!important;text-shadow:none!important;background:transparent!important;box-shadow:none!important}a,a:visited{text-decoration:underline}a[href]:after{content:" (" attr(href) ")"}abbr[title]:after{content:" (" attr(title) 
 ")"}.ir a:after,a[href^="javascript:"]:after,a[href^="#"]:after{content:""}pre,blockquote{border:1px solid #999;page-break-inside:avoid}thead{display:table-header-group}tr,img{page-break-inside:avoid}img{max-width:100%!important}@page{margin:.5cm}p,h2,h3{orphans:3;widows:3}h2,h3{page-break-after:avoid}}body{margin:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:14px;line-height:20px;color:#333;background-color:#fff}a{color:#08c;text-decoration:none}a:hover,a:focus{color:#005580;text-decoration:underline}.img-rounded{-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.img-polaroid{padding:4px;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);-webkit-box-shadow:0 1px 3px rgba(0,0,0,0.1);-moz-box-shadow:0 1px 3px rgba(0,0,0,0.1);box-shadow:0 1px 3px rgba(0,0,0,0.1)}.img-circle{-webkit-border-radius:500px;-moz-border-radius:500px;border-radius:500px}.row{margin-left:-20px;*zoom:1}.row:before,.row:after{display:table;line-hei
 ght:0;content:""}.row:after{clear:both}[class*="span"]{float:left;min-height:1px;margin-left:20px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:940px}.span12{width:940px}.span11{width:860px}.span10{width:780px}.span9{width:700px}.span8{width:620px}.span7{width:540px}.span6{width:460px}.span5{width:380px}.span4{width:300px}.span3{width:220px}.span2{width:140px}.span1{width:60px}.offset12{margin-left:980px}.offset11{margin-left:900px}.offset10{margin-left:820px}.offset9{margin-left:740px}.offset8{margin-left:660px}.offset7{margin-left:580px}.offset6{margin-left:500px}.offset5{margin-left:420px}.offset4{margin-left:340px}.offset3{margin-left:260px}.offset2{margin-left:180px}.offset1{margin-left:100px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid [class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.127
 659574468085%;*margin-left:2.074468085106383%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .controls-row [class*="span"]+[class*="span"]{margin-left:2.127659574468085%}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.48936170212765%;*width:91.43617021276594%}.row-fluid .span10{width:82.97872340425532%;*width:82.92553191489361%}.row-fluid .span9{width:74.46808510638297%;*width:74.41489361702126%}.row-fluid .span8{width:65.95744680851064%;*width:65.90425531914893%}.row-fluid .span7{width:57.44680851063829%;*width:57.39361702127659%}.row-fluid .span6{width:48.93617021276595%;*width:48.88297872340425%}.row-fluid .span5{width:40.42553191489362%;*width:40.37234042553192%}.row-fluid .span4{width:31.914893617021278%;*width:31.861702127659576%}.row-fluid .span3{width:23.404255319148934%;*width:23.351063829787233%}.row-fluid .span2{width:14.893617021276595%;*wid
 th:14.840425531914894%}.row-fluid .span1{width:6.382978723404255%;*width:6.329787234042553%}.row-fluid .offset12{margin-left:104.25531914893617%;*margin-left:104.14893617021275%}.row-fluid .offset12:first-child{margin-left:102.12765957446808%;*margin-left:102.02127659574467%}.row-fluid .offset11{margin-left:95.74468085106382%;*margin-left:95.6382978723404%}.row-fluid .offset11:first-child{margin-left:93.61702127659574%;*margin-left:93.51063829787232%}.row-fluid .offset10{margin-left:87.23404255319149%;*margin-left:87.12765957446807%}.row-fluid .offset10:first-child{margin-left:85.1063829787234%;*margin-left:84.99999999999999%}.row-fluid .offset9{margin-left:78.72340425531914%;*margin-left:78.61702127659572%}.row-fluid .offset9:first-child{margin-left:76.59574468085106%;*margin-left:76.48936170212764%}.row-fluid .offset8{margin-left:70.2127659574468%;*margin-left:70.10638297872339%}.row-fluid .offset8:first-child{margin-left:68.08510638297872%;*margin-left:67.9787234042553%}.row-flui
 d .offset7{margin-left:61.70212765957446%;*margin-left:61.59574468085106%}.row-fluid .offset7:first-child{margin-left:59.574468085106375%;*margin-left:59.46808510638297%}.row-fluid .offset6{margin-left:53.191489361702125%;*margin-left:53.085106382978715%}.row-fluid .offset6:first-child{margin-left:51.063829787234035%;*margin-left:50.95744680851063%}.row-fluid .offset5{margin-left:44.68085106382979%;*margin-left:44.57446808510638%}.row-fluid .offset5:first-child{margin-left:42.5531914893617%;*margin-left:42.4468085106383%}.row-fluid .offset4{margin-left:36.170212765957444%;*margin-left:36.06382978723405%}.row-fluid .offset4:first-child{margin-left:34.04255319148936%;*margin-left:33.93617021276596%}.row-fluid .offset3{margin-left:27.659574468085104%;*margin-left:27.5531914893617%}.row-fluid .offset3:first-child{margin-left:25.53191489361702%;*margin-left:25.425531914893618%}.row-fluid .offset2{margin-left:19.148936170212764%;*margin-left:19.04255319148936%}.row-fluid .offset2:first-ch
 ild{margin-left:17.02127659574468%;*margin-left:16.914893617021278%}.row-fluid .offset1{margin-left:10.638297872340425%;*margin-left:10.53191489361702%}.row-fluid .offset1:first-child{margin-left:8.51063829787234%;*margin-left:8.404255319148938%}[class*="span"].hide,.row-fluid [class*="span"].hide{display:none}[class*="span"].pull-right,.row-fluid [class*="span"].pull-right{float:right}.container{margin-right:auto;margin-left:auto;*zoom:1}.container:before,.container:after{display:table;line-height:0;content:""}.container:after{clear:both}.container-fluid{padding-right:20px;padding-left:20px;*zoom:1}.container-fluid:before,.container-fluid:after{display:table;line-height:0;content:""}.container-fluid:after{clear:both}p{margin:0 0 10px}.lead{margin-bottom:20px;font-size:21px;font-weight:200;line-height:30px}small{font-size:85%}strong{font-weight:bold}em{font-style:italic}cite{font-style:normal}.muted{color:#999}a.muted:hover,a.muted:focus{color:#808080}.text-warning{color:#c09853}a.t
 ext-warning:hover,a.text-warning:focus{color:#a47e3c}.text-error{color:#b94a48}a.text-error:hover,a.text-error:focus{color:#953b39}.text-info{color:#3a87ad}a.text-info:hover,a.text-info:focus{color:#2d6987}.text-success{color:#468847}a.text-success:hover,a.text-success:focus{color:#356635}.text-left{text-align:left}.text-right{text-align:right}.text-center{text-align:center}h1,h2,h3,h4,h5,h6{margin:10px 0;font-family:inherit;font-weight:bold;line-height:20px;color:inherit;text-rendering:optimizelegibility}h1 small,h2 small,h3 small,h4 small,h5 small,h6 small{font-weight:normal;line-height:1;color:#999}h1,h2,h3{line-height:40px}h1{font-size:38.5px}h2{font-size:31.5px}h3{font-size:24.5px}h4{font-size:17.5px}h5{font-size:14px}h6{font-size:11.9px}h1 small{font-size:24.5px}h2 small{font-size:17.5px}h3 small{font-size:14px}h4 small{font-size:14px}.page-header{padding-bottom:9px;margin:20px 0 30px;border-bottom:1px solid #eee}ul,ol{padding:0;margin:0 0 10px 25px}ul ul,ul ol,ol ol,ol ul{mar
 gin-bottom:0}li{line-height:20px}ul.unstyled,ol.unstyled{margin-left:0;list-style:none}ul.inline,ol.inline{margin-left:0;list-style:none}ul.inline>li,ol.inline>li{display:inline-block;*display:inline;padding-right:5px;padding-left:5px;*zoom:1}dl{margin-bottom:20px}dt,dd{line-height:20px}dt{font-weight:bold}dd{margin-left:10px}.dl-horizontal{*zoom:1}.dl-horizontal:before,.dl-horizontal:after{display:table;line-height:0;content:""}.dl-horizontal:after{clear:both}.dl-horizontal dt{float:left;width:160px;overflow:hidden;clear:left;text-align:right;text-overflow:ellipsis;white-space:nowrap}.dl-horizontal dd{margin-left:180px}hr{margin:20px 0;border:0;border-top:1px solid #eee;border-bottom:1px solid #fff}abbr[title],abbr[data-original-title]{cursor:help;border-bottom:1px dotted #999}abbr.initialism{font-size:90%;text-transform:uppercase}blockquote{padding:0 0 0 15px;margin:0 0 20px;border-left:5px solid #eee}blockquote p{margin-bottom:0;font-size:17.5px;font-weight:300;line-height:1.25}b
 lockquote small{display:block;line-height:20px;color:#999}blockquote small:before{content:'\2014 \00A0'}blockquote.pull-right{float:right;padding-right:15px;padding-left:0;border-right:5px solid #eee;border-left:0}blockquote.pull-right p,blockquote.pull-right small{text-align:right}blockquote.pull-right small:before{content:''}blockquote.pull-right small:after{content:'\00A0 \2014'}q:before,q:after,blockquote:before,blockquote:after{content:""}address{display:block;margin-bottom:20px;font-style:normal;line-height:20px}code,pre{padding:0 3px 2px;font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:12px;color:#333;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}code{padding:2px 4px;color:#d14;white-space:nowrap;background-color:#f7f7f9;border:1px solid #e1e1e8}pre{display:block;padding:9.5px;margin:0 0 10px;font-size:13px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre;white-space:pre-wrap;background-color:#f5f5f5;border:1px
  solid #ccc;border:1px solid rgba(0,0,0,0.15);-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}pre.prettyprint{margin-bottom:20px}pre code{padding:0;color:inherit;white-space:pre;white-space:pre-wrap;background-color:transparent;border:0}.pre-scrollable{max-height:340px;overflow-y:scroll}form{margin:0 0 20px}fieldset{padding:0;margin:0;border:0}legend{display:block;width:100%;padding:0;margin-bottom:20px;font-size:21px;line-height:40px;color:#333;border:0;border-bottom:1px solid #e5e5e5}legend small{font-size:15px;color:#999}label,input,button,select,textarea{font-size:14px;font-weight:normal;line-height:20px}input,button,select,textarea{font-family:"Helvetica Neue",Helvetica,Arial,sans-serif}label{display:block;margin-bottom:5px}select,textarea,input[type="text"],input[type="password"],input[type="datetime"],input[type="datetime-local"],input[type="date"],input[type="month"],input[type="time"],input[type="week"],input[type="number"],input[type="email"],input[type=
 "url"],input[type="search"],input[type="tel"],input[type="color"],.uneditable-input{display:inline-block;height:20px;padding:4px 6px;margin-bottom:10px;font-size:14px;line-height:20px;color:#555;vertical-align:middle;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}input,textarea,.uneditable-input{width:206px}textarea{height:auto}textarea,input[type="text"],input[type="password"],input[type="datetime"],input[type="datetime-local"],input[type="date"],input[type="month"],input[type="time"],input[type="week"],input[type="number"],input[type="email"],input[type="url"],input[type="search"],input[type="tel"],input[type="color"],.uneditable-input{background-color:#fff;border:1px solid #ccc;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-webkit-transition:border linear .2s,box-shadow linear .2s;-moz-transition:border linear .2s,box-shadow linear .2s;-o-transition:border lin
 ear .2s,box-shadow linear .2s;transition:border linear .2s,box-shadow linear .2s}textarea:focus,input[type="text"]:focus,input[type="password"]:focus,input[type="datetime"]:focus,input[type="datetime-local"]:focus,input[type="date"]:focus,input[type="month"]:focus,input[type="time"]:focus,input[type="week"]:focus,input[type="number"]:focus,input[type="email"]:focus,input[type="url"]:focus,input[type="search"]:focus,input[type="tel"]:focus,input[type="color"]:focus,.uneditable-input:focus{border-color:rgba(82,168,236,0.8);outline:0;outline:thin dotted \9;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6)}input[type="radio"],input[type="checkbox"]{margin:4px 0 0;margin-top:1px \9;*margin-top:0;line-height:normal}input[type="file"],input[type="image"],input[type="submit"],input[type="reset"],input[type="b
 utton"],input[type="radio"],input[type="checkbox"]{width:auto}select,input[type="file"]{height:30px;*margin-top:4px;line-height:30px}select{width:220px;background-color:#fff;border:1px solid #ccc}select[multiple],select[size]{height:auto}select:focus,input[type="file"]:focus,input[type="radio"]:focus,input[type="checkbox"]:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}.uneditable-input,.uneditable-textarea{color:#999;cursor:not-allowed;background-color:#fcfcfc;border-color:#ccc;-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.025);-moz-box-shadow:inset 0 1px 2px rgba(0,0,0,0.025);box-shadow:inset 0 1px 2px rgba(0,0,0,0.025)}.uneditable-input{overflow:hidden;white-space:nowrap}.uneditable-textarea{width:auto;height:auto}input:-moz-placeholder,textarea:-moz-placeholder{color:#999}input:-ms-input-placeholder,textarea:-ms-input-placeholder{color:#999}input::-webkit-input-placeholder,textarea::-webkit-input-placeholder{color:#999}.radio,.chec
 kbox{min-height:20px;padding-left:20px}.radio input[type="radio"],.checkbox input[type="checkbox"]{float:left;margin-left:-20px}.controls>.radio:first-child,.controls>.checkbox:first-child{padding-top:5px}.radio.inline,.checkbox.inline{display:inline-block;padding-top:5px;margin-bottom:0;vertical-align:middle}.radio.inline+.radio.inline,.checkbox.inline+.checkbox.inline{margin-left:10px}.input-mini{width:60px}.input-small{width:90px}.input-medium{width:150px}.input-large{width:210px}.input-xlarge{width:270px}.input-xxlarge{width:530px}input[class*="span"],select[class*="span"],textarea[class*="span"],.uneditable-input[class*="span"],.row-fluid input[class*="span"],.row-fluid select[class*="span"],.row-fluid textarea[class*="span"],.row-fluid .uneditable-input[class*="span"]{float:none;margin-left:0}.input-append input[class*="span"],.input-append .uneditable-input[class*="span"],.input-prepend input[class*="span"],.input-prepend .uneditable-input[class*="span"],.row-fluid input[clas
 s*="span"],.row-fluid select[class*="span"],.row-fluid textarea[class*="span"],.row-fluid .uneditable-input[class*="span"],.row-fluid .input-prepend [class*="span"],.row-fluid .input-append [class*="span"]{display:inline-block}input,textarea,.uneditable-input{margin-left:0}.controls-row [class*="span"]+[class*="span"]{margin-left:20px}input.span12,textarea.span12,.uneditable-input.span12{width:926px}input.span11,textarea.span11,.uneditable-input.span11{width:846px}input.span10,textarea.span10,.uneditable-input.span10{width:766px}input.span9,textarea.span9,.uneditable-input.span9{width:686px}input.span8,textarea.span8,.uneditable-input.span8{width:606px}input.span7,textarea.span7,.uneditable-input.span7{width:526px}input.span6,textarea.span6,.uneditable-input.span6{width:446px}input.span5,textarea.span5,.uneditable-input.span5{width:366px}input.span4,textarea.span4,.uneditable-input.span4{width:286px}input.span3,textarea.span3,.uneditable-input.span3{width:206px}input.span2,textarea.
 span2,.uneditable-input.span2{width:126px}input.span1,textarea.span1,.uneditable-input.span1{width:46px}.controls-row{*zoom:1}.controls-row:before,.controls-row:after{display:table;line-height:0;content:""}.controls-row:after{clear:both}.controls-row [class*="span"],.row-fluid .controls-row [class*="span"]{float:left}.controls-row .checkbox[class*="span"],.controls-row .radio[class*="span"]{padding-top:5px}input[disabled],select[disabled],textarea[disabled],input[readonly],select[readonly],textarea[readonly]{cursor:not-allowed;background-color:#eee}input[type="radio"][disabled],input[type="checkbox"][disabled],input[type="radio"][readonly],input[type="checkbox"][readonly]{background-color:transparent}.control-group.warning .control-label,.control-group.warning .help-block,.control-group.warning .help-inline{color:#c09853}.control-group.warning .checkbox,.control-group.warning .radio,.control-group.warning input,.control-group.warning select,.control-group.warning textarea{color:#c09
 853}.control-group.warning input,.control-group.warning select,.control-group.warning textarea{border-color:#c09853;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.warning input:focus,.control-group.warning select:focus,.control-group.warning textarea:focus{border-color:#a47e3c;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e}.control-group.warning .input-prepend .add-on,.control-group.warning .input-append .add-on{color:#c09853;background-color:#fcf8e3;border-color:#c09853}.control-group.error .control-label,.control-group.error .help-block,.control-group.error .help-inline{color:#b94a48}.control-group.error .checkbox,.control-group.error .radio,.control-group.error input,.control-group.error select,.control-group.error text
 area{color:#b94a48}.control-group.error input,.control-group.error select,.control-group.error textarea{border-color:#b94a48;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.error input:focus,.control-group.error select:focus,.control-group.error textarea:focus{border-color:#953b39;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392}.control-group.error .input-prepend .add-on,.control-group.error .input-append .add-on{color:#b94a48;background-color:#f2dede;border-color:#b94a48}.control-group.success .control-label,.control-group.success .help-block,.control-group.success .help-inline{color:#468847}.control-group.success .checkbox,.control-group.success .radio,.control-group.success input,.control-group.success select,.control-gro
 up.success textarea{color:#468847}.control-group.success input,.control-group.success select,.control-group.success textarea{border-color:#468847;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.success input:focus,.control-group.success select:focus,.control-group.success textarea:focus{border-color:#356635;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b}.control-group.success .input-prepend .add-on,.control-group.success .input-append .add-on{color:#468847;background-color:#dff0d8;border-color:#468847}.control-group.info .control-label,.control-group.info .help-block,.control-group.info .help-inline{color:#3a87ad}.control-group.info .checkbox,.control-group.info .radio,.control-group.info input,.control-group.info select,.c
 ontrol-group.info textarea{color:#3a87ad}.control-group.info input,.control-group.info select,.control-group.info textarea{border-color:#3a87ad;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.info input:focus,.control-group.info select:focus,.control-group.info textarea:focus{border-color:#2d6987;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7ab5d3;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7ab5d3;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7ab5d3}.control-group.info .input-prepend .add-on,.control-group.info .input-append .add-on{color:#3a87ad;background-color:#d9edf7;border-color:#3a87ad}input:focus:invalid,textarea:focus:invalid,select:focus:invalid{color:#b94a48;border-color:#ee5f5b}input:focus:invalid:focus,textarea:focus:invalid:focus,select:focus:invalid:focus{border-color:#e9322d;-webkit-box-shadow:0 0 6px #f8b9b7;-moz-b
 ox-shadow:0 0 6px #f8b9b7;box-shadow:0 0 6px #f8b9b7}.form-actions{padding:19px 20px 20px;margin-top:20px;margin-bottom:20px;background-color:#f5f5f5;border-top:1px solid #e5e5e5;*zoom:1}.form-actions:before,.form-actions:after{display:table;line-height:0;content:""}.form-actions:after{clear:both}.help-block,.help-inline{color:#595959}.help-block{display:block;margin-bottom:10px}.help-inline{display:inline-block;*display:inline;padding-left:5px;vertical-align:middle;*zoom:1}.input-append,.input-prepend{display:inline-block;margin-bottom:10px;font-size:0;white-space:nowrap;vertical-align:middle}.input-append input,.input-prepend input,.input-append select,.input-prepend select,.input-append .uneditable-input,.input-prepend .uneditable-input,.input-append .dropdown-menu,.input-prepend .dropdown-menu,.input-append .popover,.input-prepend .popover{font-size:14px}.input-append input,.input-prepend input,.input-append select,.input-prepend select,.input-append .uneditable-input,.input-pre
 pend .uneditable-input{position:relative;margin-bottom:0;*margin-left:0;vertical-align:top;-webkit-border-radius:0 4px 4px 0;-moz-border-radius:0 4px 4px 0;border-radius:0 4px 4px 0}.input-append input:focus,.input-prepend input:focus,.input-append select:focus,.input-prepend select:focus,.input-append .uneditable-input:focus,.input-prepend .uneditable-input:focus{z-index:2}.input-append .add-on,.input-prepend .add-on{display:inline-block;width:auto;height:20px;min-width:16px;padding:4px 5px;font-size:14px;font-weight:normal;line-height:20px;text-align:center;text-shadow:0 1px 0 #fff;background-color:#eee;border:1px solid #ccc}.input-append .add-on,.input-prepend .add-on,.input-append .btn,.input-prepend .btn,.input-append .btn-group>.dropdown-toggle,.input-prepend .btn-group>.dropdown-toggle{vertical-align:top;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.input-append .active,.input-prepend .active{background-color:#a9dba9;border-color:#46a546}.input-prepend .add-on
 ,.input-prepend .btn{margin-right:-1px}.input-prepend .add-on:first-child,.input-prepend .btn:first-child{-webkit-border-radius:4px 0 0 4px;-moz-border-radius:4px 0 0 4px;border-radius:4px 0 0 4px}.input-append input,.input-append select,.input-append .uneditable-input{-webkit-border-radius:4px 0 0 4px;-moz-border-radius:4px 0 0 4px;border-radius:4px 0 0 4px}.input-append input+.btn-group .btn:last-child,.input-append select+.btn-group .btn:last-child,.input-append .uneditable-input+.btn-group .btn:last-child{-webkit-border-radius:0 4px 4px 0;-moz-border-radius:0 4px 4px 0;border-radius:0 4px 4px 0}.input-append .add-on,.input-append .btn,.input-append .btn-group{margin-left:-1px}.input-append .add-on:last-child,.input-append .btn:last-child,.input-append .btn-group:last-child>.dropdown-toggle{-webkit-border-radius:0 4px 4px 0;-moz-border-radius:0 4px 4px 0;border-radius:0 4px 4px 0}.input-prepend.input-append input,.input-prepend.input-append select,.input-prepend.input-append .une
 ditable-input{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.input-prepend.input-append input+.btn-group .btn,.input-prepend.input-append select+.btn-group .btn,.input-prepend.input-append .uneditable-input+.btn-group .btn{-webkit-border-radius:0 4px 4px 0;-moz-border-radius:0 4px 4px 0;border-radius:0 4px 4px 0}.input-prepend.input-append .add-on:first-child,.input-prepend.input-append .btn:first-child{margin-right:-1px;-webkit-border-radius:4px 0 0 4px;-moz-border-radius:4px 0 0 4px;border-radius:4px 0 0 4px}.input-prepend.input-append .add-on:last-child,.input-prepend.input-append .btn:last-child{margin-left:-1px;-webkit-border-radius:0 4px 4px 0;-moz-border-radius:0 4px 4px 0;border-radius:0 4px 4px 0}.input-prepend.input-append .btn-group:first-child{margin-left:0}input.search-query{padding-right:14px;padding-right:4px \9;padding-left:14px;padding-left:4px \9;margin-bottom:0;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.form-search .input
 -append .search-query,.form-search .input-prepend .search-query{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.form-search .input-append .search-query{-webkit-border-radius:14px 0 0 14px;-moz-border-radius:14px 0 0 14px;border-radius:14px 0 0 14px}.form-search .input-append .btn{-webkit-border-radius:0 14px 14px 0;-moz-border-radius:0 14px 14px 0;border-radius:0 14px 14px 0}.form-search .input-prepend .search-query{-webkit-border-radius:0 14px 14px 0;-moz-border-radius:0 14px 14px 0;border-radius:0 14px 14px 0}.form-search .input-prepend .btn{-webkit-border-radius:14px 0 0 14px;-moz-border-radius:14px 0 0 14px;border-radius:14px 0 0 14px}.form-search input,.form-inline input,.form-horizontal input,.form-search textarea,.form-inline textarea,.form-horizontal textarea,.form-search select,.form-inline select,.form-horizontal select,.form-search .help-inline,.form-inline .help-inline,.form-horizontal .help-inline,.form-search .uneditable-input,.form-inline .uneditable-inp
 ut,.form-horizontal .uneditable-input,.form-search .input-prepend,.form-inline .input-prepend,.form-horizontal .input-prepend,.form-search .input-append,.form-inline .input-append,.form-horizontal .input-append{display:inline-block;*display:inline;margin-bottom:0;vertical-align:middle;*zoom:1}.form-search .hide,.form-inline .hide,.form-horizontal .hide{display:none}.form-search label,.form-inline label,.form-search .btn-group,.form-inline .btn-group{display:inline-block}.form-search .input-append,.form-inline .input-append,.form-search .input-prepend,.form-inline .input-prepend{margin-bottom:0}.form-search .radio,.form-search .checkbox,.form-inline .radio,.form-inline .checkbox{padding-left:0;margin-bottom:0;vertical-align:middle}.form-search .radio input[type="radio"],.form-search .checkbox input[type="checkbox"],.form-inline .radio input[type="radio"],.form-inline .checkbox input[type="checkbox"]{float:left;margin-right:3px;margin-left:0}.control-group{margin-bottom:10px}legend+.c
 ontrol-group{margin-top:20px;-webkit-margin-top-collapse:separate}.form-horizontal .control-group{margin-bottom:20px;*zoom:1}.form-horizontal .control-group:before,.form-horizontal .control-group:after{display:table;line-height:0;content:""}.form-horizontal .control-group:after{clear:both}.form-horizontal .control-label{float:left;width:160px;padding-top:5px;text-align:right}.form-horizontal .controls{*display:inline-block;*padding-left:20px;margin-left:180px;*margin-left:0}.form-horizontal .controls:first-child{*padding-left:180px}.form-horizontal .help-block{margin-bottom:0}.form-horizontal input+.help-block,.form-horizontal select+.help-block,.form-horizontal textarea+.help-block,.form-horizontal .uneditable-input+.help-block,.form-horizontal .input-prepend+.help-block,.form-horizontal .input-append+.help-block{margin-top:10px}.form-horizontal .form-actions{padding-left:180px}table{max-width:100%;background-color:transparent;border-collapse:collapse;border-spacing:0}.table{width:
 100%;margin-bottom:20px}.table th,.table td{padding:8px;line-height:20px;text-align:left;vertical-align:top;border-top:1px solid #ddd}.table th{font-weight:bold}.table thead th{vertical-align:bottom}.table caption+thead tr:first-child th,.table caption+thead tr:first-child td,.table colgroup+thead tr:first-child th,.table colgroup+thead tr:first-child td,.table thead:first-child tr:first-child th,.table thead:first-child tr:first-child td{border-top:0}.table tbody+tbody{border-top:2px solid #ddd}.table .table{background-color:#fff}.table-condensed th,.table-condensed td{padding:4px 5px}.table-bordered{border:1px solid #ddd;border-collapse:separate;*border-collapse:collapse;border-left:0;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.table-bordered th,.table-bordered td{border-left:1px solid #ddd}.table-bordered caption+thead tr:first-child th,.table-bordered caption+tbody tr:first-child th,.table-bordered caption+tbody tr:first-child td,.table-bordered colgroup+
 thead tr:first-child th,.table-bordered colgroup+tbody tr:first-child th,.table-bordered colgroup+tbody tr:first-child td,.table-bordered thead:first-child tr:first-child th,.table-bordered tbody:first-child tr:first-child th,.table-bordered tbody:first-child tr:first-child td{border-top:0}.table-bordered thead:first-child tr:first-child>th:first-child,.table-bordered tbody:first-child tr:first-child>td:first-child,.table-bordered tbody:first-child tr:first-child>th:first-child{-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-topleft:4px}.table-bordered thead:first-child tr:first-child>th:last-child,.table-bordered tbody:first-child tr:first-child>td:last-child,.table-bordered tbody:first-child tr:first-child>th:last-child{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-moz-border-radius-topright:4px}.table-bordered thead:last-child tr:last-child>th:first-child,.table-bordered tbody:last-child tr:last-child>td:first-child,.table-borde
 red tbody:last-child tr:last-child>th:first-child,.table-bordered tfoot:last-child tr:last-child>td:first-child,.table-bordered tfoot:last-child tr:last-child>th:first-child{-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-moz-border-radius-bottomleft:4px}.table-bordered thead:last-child tr:last-child>th:last-child,.table-bordered tbody:last-child tr:last-child>td:last-child,.table-bordered tbody:last-child tr:last-child>th:last-child,.table-bordered tfoot:last-child tr:last-child>td:last-child,.table-bordered tfoot:last-child tr:last-child>th:last-child{-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-moz-border-radius-bottomright:4px}.table-bordered tfoot+tbody:last-child tr:last-child td:first-child{-webkit-border-bottom-left-radius:0;border-bottom-left-radius:0;-moz-border-radius-bottomleft:0}.table-bordered tfoot+tbody:last-child tr:last-child td:last-child{-webkit-border-bottom-right-radius:0;border-bottom-right-radius:0;-moz-border-ra
 dius-bottomright:0}.table-bordered caption+thead tr:first-child th:first-child,.table-bordered caption+tbody tr:first-child td:first-child,.table-bordered colgroup+thead tr:first-child th:first-child,.table-bordered colgroup+tbody tr:first-child td:first-child{-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-topleft:4px}.table-bordered caption+thead tr:first-child th:last-child,.table-bordered caption+tbody tr:first-child td:last-child,.table-bordered colgroup+thead tr:first-child th:last-child,.table-bordered colgroup+tbody tr:first-child td:last-child{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-moz-border-radius-topright:4px}.table-striped tbody>tr:nth-child(odd)>td,.table-striped tbody>tr:nth-child(odd)>th{background-color:#f9f9f9}.table-hover tbody tr:hover>td,.table-hover tbody tr:hover>th{background-color:#f5f5f5}table td[class*="span"],table th[class*="span"],.row-fluid table td[class*="span"],.row-fluid table th[class*="sp
 an"]{display:table-cell;float:none;margin-left:0}.table td.span1,.table th.span1{float:none;width:44px;margin-left:0}.table td.span2,.table th.span2{float:none;width:124px;margin-left:0}.table td.span3,.table th.span3{float:none;width:204px;margin-left:0}.table td.span4,.table th.span4{float:none;width:284px;margin-left:0}.table td.span5,.table th.span5{float:none;width:364px;margin-left:0}.table td.span6,.table th.span6{float:none;width:444px;margin-left:0}.table td.span7,.table th.span7{float:none;width:524px;margin-left:0}.table td.span8,.table th.span8{float:none;width:604px;margin-left:0}.table td.span9,.table th.span9{float:none;width:684px;margin-left:0}.table td.span10,.table th.span10{float:none;width:764px;margin-left:0}.table td.span11,.table th.span11{float:none;width:844px;margin-left:0}.table td.span12,.table th.span12{float:none;width:924px;margin-left:0}.table tbody tr.success>td{background-color:#dff0d8}.table tbody tr.error>td{background-color:#f2dede}.table tbody 
 tr.warning>td{background-color:#fcf8e3}.table tbody tr.info>td{background-color:#d9edf7}.table-hover tbody tr.success:hover>td{background-color:#d0e9c6}.table-hover tbody tr.error:hover>td{background-color:#ebcccc}.table-hover tbody tr.warning:hover>td{background-color:#faf2cc}.table-hover tbody tr.info:hover>td{background-color:#c4e3f3}[class^="icon-"],[class*=" icon-"]{display:inline-block;width:14px;height:14px;margin-top:1px;*margin-right:.3em;line-height:14px;vertical-align:text-top;background-image:url("../img/glyphicons-halflings.png");background-position:14px 14px;background-repeat:no-repeat}.icon-white,.nav-pills>.active>a>[class^="icon-"],.nav-pills>.active>a>[class*=" icon-"],.nav-list>.active>a>[class^="icon-"],.nav-list>.active>a>[class*=" icon-"],.navbar-inverse .nav>.active>a>[class^="icon-"],.navbar-inverse .nav>.active>a>[class*=" icon-"],.dropdown-menu>li>a:hover>[class^="icon-"],.dropdown-menu>li>a:focus>[class^="icon-"],.dropdown-menu>li>a:hover>[class*=" icon-"]
 ,.dropdown-menu>li>a:focus>[class*=" icon-"],.dropdown-menu>.active>a>[class^="icon-"],.dropdown-menu>.active>a>[class*=" icon-"],.dropdown-submenu:hover>a>[class^="icon-"],.dropdown-submenu:focus>a>[class^="icon-"],.dropdown-submenu:hover>a>[class*=" icon-"],.dropdown-submenu:focus>a>[class*=" icon-"]{background-image:url("../img/glyphicons-halflings-white.png")}.icon-glass{background-position:0 0}.icon-music{background-position:-24px 0}.icon-search{background-position:-48px 0}.icon-envelope{background-position:-72px 0}.icon-heart{background-position:-96px 0}.icon-star{background-position:-120px 0}.icon-star-empty{background-position:-144px 0}.icon-user{background-position:-168px 0}.icon-film{background-position:-192px 0}.icon-th-large{background-position:-216px 0}.icon-th{background-position:-240px 0}.icon-th-list{background-position:-264px 0}.icon-ok{background-position:-288px 0}.icon-remove{background-position:-312px 0}.icon-zoom-in{background-position:-336px 0}.icon-zoom-out{ba
 ckground-position:-360px 0}.icon-off{background-position:-384px 0}.icon-signal{background-position:-408px 0}.icon-cog{background-position:-432px 0}.icon-trash{background-position:-456px 0}.icon-home{background-position:0 -24px}.icon-file{background-position:-24px -24px}.icon-time{background-position:-48px -24px}.icon-road{background-position:-72px -24px}.icon-download-alt{background-position:-96px -24px}.icon-download{background-position:-120px -24px}.icon-upload{background-position:-144px -24px}.icon-inbox{background-position:-168px -24px}.icon-play-circle{background-position:-192px -24px}.icon-repeat{background-position:-216px -24px}.icon-refresh{background-position:-240px -24px}.icon-list-alt{background-position:-264px -24px}.icon-lock{background-position:-287px -24px}.icon-flag{background-position:-312px -24px}.icon-headphones{background-position:-336px -24px}.icon-volume-off{background-position:-360px -24px}.icon-volume-down{background-position:-384px -24px}.icon-volume-up{back
 ground-position:-408px -24px}.icon-qrcode{background-position:-432px -24px}.icon-barcode{background-position:-456px -24px}.icon-tag{background-position:0 -48px}.icon-tags{background-position:-25px -48px}.icon-book{background-position:-48px -48px}.icon-bookmark{background-position:-72px -48px}.icon-print{background-position:-96px -48px}.icon-camera{background-position:-120px -48px}.icon-font{background-position:-144px -48px}.icon-bold{background-position:-167px -48px}.icon-italic{background-position:-192px -48px}.icon-text-height{background-position:-216px -48px}.icon-text-width{background-position:-240px -48px}.icon-align-left{background-position:-264px -48px}.icon-align-center{background-position:-288px -48px}.icon-align-right{background-position:-312px -48px}.icon-align-justify{background-position:-336px -48px}.icon-list{background-position:-360px -48px}.icon-indent-left{background-position:-384px -48px}.icon-indent-right{background-position:-408px -48px}.icon-facetime-video{backg
 round-position:-432px -48px}.icon-picture{background-position:-456px -48px}.icon-pencil{background-position:0 -72px}.icon-map-marker{background-position:-24px -72px}.icon-adjust{background-position:-48px -72px}.icon-tint{background-position:-72px -72px}.icon-edit{background-position:-96px -72px}.icon-share{background-position:-120px -72px}.icon-check{background-position:-144px -72px}.icon-move{background-position:-168px -72px}.icon-step-backward{background-position:-192px -72px}.icon-fast-backward{background-position:-216px -72px}.icon-backward{background-position:-240px -72px}.icon-play{background-position:-264px -72px}.icon-pause{background-position:-288px -72px}.icon-stop{background-position:-312px -72px}.icon-forward{background-position:-336px -72px}.icon-fast-forward{background-position:-360px -72px}.icon-step-forward{background-position:-384px -72px}.icon-eject{background-position:-408px -72px}.icon-chevron-left{background-position:-432px -72px}.icon-chevron-right{background-p
 osition:-456px -72px}.icon-plus-sign{background-position:0 -96px}.icon-minus-sign{background-position:-24px -96px}.icon-remove-sign{background-position:-48px -96px}.icon-ok-sign{background-position:-72px -96px}.icon-question-sign{background-position:-96px -96px}.icon-info-sign{background-position:-120px -96px}.icon-screenshot{background-position:-144px -96px}.icon-remove-circle{background-position:-168px -96px}.icon-ok-circle{background-position:-192px -96px}.icon-ban-circle{background-position:-216px -96px}.icon-arrow-left{background-position:-240px -96px}.icon-arrow-right{background-position:-264px -96px}.icon-arrow-up{background-position:-289px -96px}.icon-arrow-down{background-position:-312px -96px}.icon-share-alt{background-position:-336px -96px}.icon-resize-full{background-position:-360px -96px}.icon-resize-small{background-position:-384px -96px}.icon-plus{background-position:-408px -96px}.icon-minus{background-position:-433px -96px}.icon-asterisk{background-position:-456px -9
 6px}.icon-exclamation-sign{background-position:0 -120px}.icon-gift{background-position:-24px -120px}.icon-leaf{background-position:-48px -120px}.icon-fire{background-position:-72px -120px}.icon-eye-open{background-position:-96px -120px}.icon-eye-close{background-position:-120px -120px}.icon-warning-sign{background-position:-144px -120px}.icon-plane{background-position:-168px -120px}.icon-calendar{background-position:-192px -120px}.icon-random{width:16px;background-position:-216px -120px}.icon-comment{background-position:-240px -120px}.icon-magnet{background-position:-264px -120px}.icon-chevron-up{background-position:-288px -120px}.icon-chevron-down{background-position:-313px -119px}.icon-retweet{background-position:-336px -120px}.icon-shopping-cart{background-position:-360px -120px}.icon-folder-close{width:16px;background-position:-384px -120px}.icon-folder-open{width:16px;background-position:-408px -120px}.icon-resize-vertical{background-position:-432px -119px}.icon-resize-horizont
 al{background-position:-456px -118px}.icon-hdd{background-position:0 -144px}.icon-bullhorn{background-position:-24px -144px}.icon-bell{background-position:-48px -144px}.icon-certificate{background-position:-72px -144px}.icon-thumbs-up{background-position:-96px -144px}.icon-thumbs-down{background-position:-120px -144px}.icon-hand-right{background-position:-144px -144px}.icon-hand-left{background-position:-168px -144px}.icon-hand-up{background-position:-192px -144px}.icon-hand-down{background-position:-216px -144px}.icon-circle-arrow-right{background-position:-240px -144px}.icon-circle-arrow-left{background-position:-264px -144px}.icon-circle-arrow-up{background-position:-288px -144px}.icon-circle-arrow-down{background-position:-312px -144px}.icon-globe{background-position:-336px -144px}.icon-wrench{background-position:-360px -144px}.icon-tasks{background-position:-384px -144px}.icon-filter{background-position:-408px -144px}.icon-briefcase{background-position:-432px -144px}.icon-fulls
 creen{background-position:-456px -144px}.dropup,.dropdown{position:relative}.dropdown-toggle{*margin-bottom:-3px}.dropdown-toggle:active,.open .dropdown-toggle{outline:0}.caret{display:inline-block;width:0;height:0;vertical-align:top;border-top:4px solid #000;border-right:4px solid transparent;border-left:4px solid transparent;content:""}.dropdown .caret{margin-top:8px;margin-left:2px}.dropdown-menu{position:absolute;top:100%;left:0;z-index:1000;display:none;float:left;min-width:160px;padding:5px 0;margin:2px 0 0;list-style:none;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);*border-right-width:2px;*border-bottom-width:2px;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 5px 10px rgba(0,0,0,0.2);-moz-box-shadow:0 5px 10px rgba(0,0,0,0.2);box-shadow:0 5px 10px rgba(0,0,0,0.2);-webkit-background-clip:padding-box;-moz-background-clip:padding;background-clip:padding-box}.dropdown-menu.pull-right{right:0;left:auto}.drop
 down-menu .divider{*width:100%;height:1px;margin:9px 1px;*margin:-5px 0 5px;overflow:hidden;background-color:#e5e5e5;border-bottom:1px solid #fff}.dropdown-menu>li>a{display:block;padding:3px 20px;clear:both;font-weight:normal;line-height:20px;color:#333;white-space:nowrap}.dropdown-menu>li>a:hover,.dropdown-menu>li>a:focus,.dropdown-submenu:hover>a,.dropdown-submenu:focus>a{color:#fff;text-decoration:none;background-color:#0081c2;background-image:-moz-linear-gradient(top,#08c,#0077b3);background-image:-webkit-gradient(linear,0 0,0 100%,from(#08c),to(#0077b3));background-image:-webkit-linear-gradient(top,#08c,#0077b3);background-image:-o-linear-gradient(top,#08c,#0077b3);background-image:linear-gradient(to bottom,#08c,#0077b3);background-repeat:repeat-x;filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0077b3',GradientType=0)}.dropdown-menu>.active>a,.dropdown-menu>.active>a:hover,.dropdown-menu>.active>a:focus{color:#fff;text-decoration:non
 e;background-color:#0081c2;background-image:-moz-linear-gradient(top,#08c,#0077b3);background-image:-webkit-gradient(linear,0 0,0 100%,from(#08c),to(#0077b3));background-image:-webkit-linear-gradient(top,#08c,#0077b3);background-image:-o-linear-gradient(top,#08c,#0077b3);background-image:linear-gradient(to bottom,#08c,#0077b3);background-repeat:repeat-x;outline:0;filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0077b3',GradientType=0)}.dropdown-menu>.disabled>a,.dropdown-menu>.disabled>a:hover,.dropdown-menu>.disabled>a:focus{color:#999}.dropdown-menu>.disabled>a:hover,.dropdown-menu>.disabled>a:focus{text-decoration:none;cursor:default;background-color:transparent;background-image:none;filter:progid:DXImageTransform.Microsoft.gradient(enabled=false)}.open{*z-index:1000}.open>.dropdown-menu{display:block}.dropdown-backdrop{position:fixed;top:0;right:0;bottom:0;left:0;z-index:990}.pull-right>.dropdown-menu{right:0;left:auto}.dropup .caret,.n
 avbar-fixed-bottom .dropdown .caret{border-top:0;border-bottom:4px solid #000;content:""}.dropup .dropdown-menu,.navbar-fixed-bottom .dropdown .dropdown-menu{top:auto;bottom:100%;margin-bottom:1px}.dropdown-submenu{position:relative}.dropdown-submenu>.dropdown-menu{top:0;left:100%;margin-top:-6px;margin-left:-1px;-webkit-border-radius:0 6px 6px 6px;-moz-border-radius:0 6px 6px 6px;border-radius:0 6px 6px 6px}.dropdown-submenu:hover>.dropdown-menu{display:block}.dropup .dropdown-submenu>.dropdown-menu{top:auto;bottom:0;margin-top:0;margin-bottom:-2px;-webkit-border-radius:5px 5px 5px 0;-moz-border-radius:5px 5px 5px 0;border-radius:5px 5px 5px 0}.dropdown-submenu>a:after{display:block;float:right;width:0;height:0;margin-top:5px;margin-right:-10px;border-color:transparent;border-left-color:#ccc;border-style:solid;border-width:5px 0 5px 5px;content:" "}.dropdown-submenu:hover>a:after{border-left-color:#fff}.dropdown-submenu.pull-left{float:none}.dropdown-submenu.pull-left>.dropdown-men
 u{left:-100%;margin-left:10px;-webkit-border-radius:6px 0 6px 6px;-moz-border-radius:6px 0 6px 6px;border-radius:6px 0 6px 6px}.dropdown .dropdown-menu .nav-header{padding-right:20px;padding-left:20px}.typeahead{z-index:1051;margin-top:2px;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.well{min-height:20px;padding:19px;margin-bottom:20px;background-color:#f5f5f5;border:1px solid #e3e3e3;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.05);box-shadow:inset 0 1px 1px rgba(0,0,0,0.05)}.well blockquote{border-color:#ddd;border-color:rgba(0,0,0,0.15)}.well-large{padding:24px;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.well-small{padding:9px;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}.fade{opacity:0;-webkit-transition:opacity .15s linear;-moz-transition:opacity .15s linear;-o-transition:opacity .15s linear;t
 ransition:opacity .15s linear}.fade.in{opacity:1}.collapse{position:relative;height:0;overflow:hidden;-webkit-transition:height .35s ease;-moz-transition:height .35s ease;-o-transition:height .35s ease;transition:height .35s ease}.collapse.in{height:auto}.close{float:right;font-size:20px;font-weight:bold;line-height:20px;color:#000;text-shadow:0 1px 0 #fff;opacity:.2;filter:alpha(opacity=20)}.close:hover,.close:focus{color:#000;text-decoration:none;cursor:pointer;opacity:.4;filter:alpha(opacity=40)}button.close{padding:0;cursor:pointer;background:transparent;border:0;-webkit-appearance:none}.btn{display:inline-block;*display:inline;padding:4px 12px;margin-bottom:0;*margin-left:.3em;font-size:14px;line-height:20px;color:#333;text-align:center;text-shadow:0 1px 1px rgba(255,255,255,0.75);vertical-align:middle;cursor:pointer;background-color:#f5f5f5;*background-color:#e6e6e6;background-image:-moz-linear-gradient(top,#fff,#e6e6e6);background-image:-webkit-gradient(linear,0 0,0 100%,from
 (#fff),to(#e6e6e6));background-image:-webkit-linear-gradient(top,#fff,#e6e6e6);background-image:-o-linear-gradient(top,#fff,#e6e6e6);background-image:linear-gradient(to bottom,#fff,#e6e6e6);background-repeat:repeat-x;border:1px solid #ccc;*border:0;border-color:#e6e6e6 #e6e6e6 #bfbfbf;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);border-bottom-color:#b3b3b3;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffffffff',endColorstr='#ffe6e6e6',GradientType=0);filter:progid:DXImageTransform.Microsoft.gradient(enabled=false);*zoom:1;-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05)}.btn:hover,.btn:focus,.btn:active,.btn.active,.btn.disabled,.btn[disabled]{color:#333;background-color:#e6e6e6;*background-color:#
 d9d9d9}.btn:active,.btn.active{background-color:#ccc \9}.btn:first-child{*margin-left:0}.btn:hover,.btn:focus{color:#333;text-decoration:none;background-position:0 -15px;-webkit-transition:background-position .1s linear;-moz-transition:background-position .1s linear;-o-transition:background-position .1s linear;transition:background-position .1s linear}.btn:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}.btn.active,.btn:active{background-image:none;outline:0;-webkit-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05)}.btn.disabled,.btn[disabled]{cursor:default;background-image:none;opacity:.65;filter:alpha(opacity=65);-webkit-box-shadow:none;-moz-box-shadow:none;box-shadow:none}.btn-large{padding:11px 19px;font-size:17.5px;-webkit-border-radius:6px;-moz-border-radius:6px;bord
 er-radius:6px}.btn-large [class^="icon-"],.btn-large [class*=" icon-"]{margin-top:4px}.btn-small{padding:2px 10px;font-size:11.9px;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}.btn-small [class^="icon-"],.btn-small [class*=" icon-"]{margin-top:0}.btn-mini [class^="icon-"],.btn-mini [class*=" icon-"]{margin-top:-1px}.btn-mini{padding:0 6px;font-size:10.5px;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}.btn-block{display:block;width:100%;padding-right:0;padding-left:0;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.btn-block+.btn-block{margin-top:5px}input[type="submit"].btn-block,input[type="reset"].btn-block,input[type="button"].btn-block{width:100%}.btn-primary.active,.btn-warning.active,.btn-danger.active,.btn-success.active,.btn-info.active,.btn-inverse.active{color:rgba(255,255,255,0.75)}.btn-primary{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#006dcc;*background-color:#04c;backgroun
 d-image:-moz-linear-gradient(top,#08c,#04c);background-image:-webkit-gradient(linear,0 0,0 100%,from(#08c),to(#04c));background-image:-webkit-linear-gradient(top,#08c,#04c);background-image:-o-linear-gradient(top,#08c,#04c);background-image:linear-gradient(to bottom,#08c,#04c);background-repeat:repeat-x;border-color:#04c #0044cc #002a80;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0044cc',GradientType=0);filter:progid:DXImageTransform.Microsoft.gradient(enabled=false)}.btn-primary:hover,.btn-primary:focus,.btn-primary:active,.btn-primary.active,.btn-primary.disabled,.btn-primary[disabled]{color:#fff;background-color:#04c;*background-color:#003bb3}.btn-primary:active,.btn-primary.active{background-color:#039 \9}.btn-warning{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#faa732;*background-color:#f89406;background-image:-moz-linear-gradient(top,#fbb450,#f894
 06);background-image:-webkit-gradient(linear,0 0,0 100%,from(#fbb450),to(#f89406));background-image:-webkit-linear-gradient(top,#fbb450,#f89406);background-image:-o-linear-gradient(top,#fbb450,#f89406);background-image:linear-gradient(to bottom,#fbb450,#f89406);background-repeat:repeat-x;border-color:#f89406 #f89406 #ad6704;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#fffbb450',endColorstr='#fff89406',GradientType=0);filter:progid:DXImageTransform.Microsoft.gradient(enabled=false)}.btn-warning:hover,.btn-warning:focus,.btn-warning:active,.btn-warning.active,.btn-warning.disabled,.btn-warning[disabled]{color:#fff;background-color:#f89406;*background-color:#df8505}.btn-warning:active,.btn-warning.active{background-color:#c67605 \9}.btn-danger{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#da4f49;*background-color:#bd362f;background-image:-moz-linear-gradient(top,#ee5f5b,#bd362f);back
 ground-image:-webkit-gradient(linear,0 0,0 100%,from(#ee5f5b),to(#bd362f));background-image:-webkit-linear-gradient(top,#ee5f5b,#bd362f);background-image:-o-linear-gradient(top,#ee5f5b,#bd362f);background-image:linear-gradient(to bottom,#ee5f5b,#bd362f);background-repeat:repeat-x;border-color:#bd362f #bd362f #802420;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffee5f5b',endColorstr='#ffbd362f',GradientType=0);filter:progid:DXImageTransform.Microsoft.gradient(enabled=false)}.btn-danger:hover,.btn-danger:focus,.btn-danger:active,.btn-danger.active,.btn-danger.disabled,.btn-danger[disabled]{color:#fff;background-color:#bd362f;*background-color:#a9302a}.btn-danger:active,.btn-danger.active{background-color:#942a25 \9}.btn-success{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#5bb75b;*background-color:#51a351;background-image:-moz-linear-gradient(top,#62c462,#51a351);background-image:-w
 ebkit-gradient(linear,0 0,0 100%,from(#62c462),to(#51a351));background-image:-webkit-linear-gradient(top,#62c462,#51a351);background-image:-o-linear-gradient(top,#62c462,#51a351);background-image:linear-gradient(to bottom,#62c462,#51a351);background-repeat:repeat-x;border-color:#51a351 #51a351 #387038;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff62c462',endColorstr='#ff51a351',GradientType=0);filter:progid:DXImageTransform.Microsoft.gradient(enabled=false)}.btn-success:hover,.btn-success:focus,.btn-success:active,.btn-success.active,.btn-success.disabled,.btn-success[disabled]{color:#fff;background-color:#51a351;*background-color:#499249}.btn-success:active,.btn-success.active{background-color:#408140 \9}.btn-info{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#49afcd;*background-color:#2f96b4;background-image:-moz-linear-gradient(top,#5bc0de,#2f96b4);background-image:-webkit-grad
 ient(linear,0 0,0 100%,from(#5bc0de),to(#2f96b4));background-image:-webkit-linear-gradient(top,#5bc0de,#2f96b4);background-image:-o-linear-gradient(top,#5bc0de,#2f96b4);background-image:linear-gradient(to bottom,#5bc0de,#2f96b4);background-repeat:repeat-x;border-color:#2f96b4 #2f96b4 #1f6377;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff5bc0de',endColorstr='#ff2f96b4',GradientType=0);filter:progid:DXImageTransform.Microsoft.gradient(enabled=false)}.btn-info:hover,.btn-info:focus,.btn-info:active,.btn-info.active,.btn-info.disabled,.btn-info[disabled]{color:#fff;background-color:#2f96b4;*background-color:#2a85a0}.btn-info:active,.btn-info.active{background-color:#24748c \9}.btn-inverse{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#363636;*background-color:#222;background-image:-moz-linear-gradient(top,#444,#222);background-image:-webkit-gradient(linear,0 0,0 100%,from(#444),to(#22
 2));background-image:-webkit-linear-gradient(top,#444,#222);background-image:-o-linear-gradient(top,#444,#222);background-image:linear-gradient(to bottom,#444,#222);background-repeat:repeat-x;border-color:#222 #222222 #000;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff444444',endColorstr='#ff222222',GradientType=0);filter:progid:DXImageTransform.Microsoft.gradient(enabled=false)}.btn-inverse:hover,.btn-inverse:focus,.btn-inverse:active,.btn-inverse.active,.btn-inverse.disabled,.btn-inverse[disabled]{color:#fff;background-color:#222;*background-color:#151515}.btn-inverse:active,.btn-inverse.active{background-color:#080808 \9}button.btn,input[type="submit"].btn{*padding-top:3px;*padding-bottom:3px}button.btn::-moz-focus-inner,input[type="submit"].btn::-moz-focus-inner{padding:0;border:0}button.btn.btn-large,input[type="submit"].btn.btn-large{*padding-top:7px;*padding-bottom:7px}button.btn.btn-small,inp
 ut[type="submit"].btn.btn-small{*padding-top:3px;*padding-bottom:3px}button.btn.btn-mini,input[type="submit"].btn.btn-mini{*padding-top:1px;*padding-bottom:1px}.btn-link,.btn-link:active,.btn-link[disabled]{background-color:transparent;background-image:none;-webkit-box-shadow:none;-moz-box-shadow:none;box-shadow:none}.btn-link{color:#08c;cursor:pointer;border-color:transparent;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-link:hover,.btn-link:focus{color:#005580;text-decoration:underline;background-color:transparent}.btn-link[disabled]:hover,.btn-link[disabled]:focus{color:#333;text-decoration:none}.btn-group{position:relative;display:inline-block;*display:inline;*margin-left:.3em;font-size:0;white-space:nowrap;vertical-align:middle;*zoom:1}.btn-group:first-child{*margin-left:0}.btn-group+.btn-group{margin-left:5px}.btn-toolbar{margin-top:10px;margin-bottom:10px;font-size:0}.btn-toolbar>.btn+.btn,.btn-toolbar>.btn-group+.btn,.btn-toolbar>.btn+.btn-group{margin-le
 ft:5px}.btn-group>.btn{position:relative;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-group>.btn+.btn{margin-left:-1px}.btn-group>.btn,.btn-group>.dropdown-menu,.btn-group>.popover{font-size:14px}.btn-group>.btn-mini{font-size:10.5px}.btn-group>.btn-small{font-size:11.9px}.btn-group>.btn-large{font-size:17.5px}.btn-group>.btn:first-child{margin-left:0;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-bottomleft:4px;-moz-border-radius-topleft:4px}.btn-group>.btn:last-child,.btn-group>.dropdown-toggle{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-moz-border-radius-topright:4px;-moz-border-radius-bottomright:4px}.btn-group>.btn.large:first-child{margin-left:0;-webkit-border-bottom-left-radius:6px;border-bottom-left-radius:6px;-webkit-border-top-left-radius:6px;border-top-left-radi
 us:6px;-moz-border-radius-bottomleft:6px;-moz-border-radius-topleft:6px}.btn-group>.btn.large:last-child,.btn-group>.large.dropdown-toggle{-webkit-border-top-right-radius:6px;border-top-right-radius:6px;-webkit-border-bottom-right-radius:6px;border-bottom-right-radius:6px;-moz-border-radius-topright:6px;-moz-border-radius-bottomright:6px}.btn-group>.btn:hover,.btn-group>.btn:focus,.btn-group>.btn:active,.btn-group>.btn.active{z-index:2}.btn-group .dropdown-toggle:active,.btn-group.open .dropdown-toggle{outline:0}.btn-group>.btn+.dropdown-toggle{*padding-top:5px;padding-right:8px;*padding-bottom:5px;padding-left:8px;-webkit-box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05)}.btn-group>.btn-mini+.dro
 pdown-toggle{*padding-top:2px;padding-right:5px;*padding-bottom:2px;padding-left:5px}.btn-group>.btn-small+.dropdown-toggle{*padding-top:5px;*padding-bottom:4px}.btn-group>.btn-large+.dropdown-toggle{*padding-top:7px;padding-right:12px;*padding-bottom:7px;padding-left:12px}.btn-group.open .dropdown-toggle{background-image:none;-webkit-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05)}.btn-group.open .btn.dropdown-toggle{background-color:#e6e6e6}.btn-group.open .btn-primary.dropdown-toggle{background-color:#04c}.btn-group.open .btn-warning.dropdown-toggle{background-color:#f89406}.btn-group.open .btn-danger.dropdown-toggle{background-color:#bd362f}.btn-group.open .btn-success.dropdown-toggle{background-color:#51a351}.btn-group.open .btn-info.dropdown-toggle{background-color:#2f96b4}.btn-group.open .btn-inverse.dropdown
 -toggle{background-color:#222}.btn .caret{margin-top:8px;margin-left:0}.btn-large .caret{margin-top:6px}.btn-large .caret{border-top-width:5px;border-right-width:5px;border-left-width:5px}.btn-mini .caret,.btn-small .caret{margin-top:8px}.dropup .btn-large .caret{border-bottom-width:5px}.btn-primary .caret,.btn-warning .caret,.btn-danger .caret,.btn-info .caret,.btn-success .caret,.btn-inverse .caret{border-top-color:#fff;border-bottom-color:#fff}.btn-group-vertical{display:inline-block;*display:inline;*zoom:1}.btn-group-vertical>.btn{display:block;float:none;max-width:100%;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-group-vertical>.btn+.btn{margin-top:-1px;margin-left:0}.btn-group-vertical>.btn:first-child{-webkit-border-radius:4px 4px 0 0;-moz-border-radius:4px 4px 0 0;border-radius:4px 4px 0 0}.btn-group-vertical>.btn:last-child{-webkit-border-radius:0 0 4px 4px;-moz-border-radius:0 0 4px 4px;border-radius:0 0 4px 4px}.btn-group-vertical>.btn-large:first-chi
 ld{-webkit-border-radius:6px 6px 0 0;-moz-border-radius:6px 6px 0 0;border-radius:6px 6px 0 0}.btn-group-vertical>.btn-large:last-child{-webkit-border-radius:0 0 6px 6px;-moz-border-radius:0 0 6px 6px;border-radius:0 0 6px 6px}.alert{padding:8px 35px 8px 14px;margin-bottom:20px;text-shadow:0 1px 0 rgba(255,255,255,0.5);background-color:#fcf8e3;border:1px solid #fbeed5;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.alert,.alert h4{color:#c09853}.alert h4{margin:0}.alert .close{position:relative;top:-2px;right:-21px;line-height:20px}.alert-success{color:#468847;background-color:#dff0d8;border-color:#d6e9c6}.alert-success h4{color:#468847}.alert-danger,.alert-error{color:#b94a48;background-color:#f2dede;border-color:#eed3d7}.alert-danger h4,.alert-error h4{color:#b94a48}.alert-info{color:#3a87ad;background-color:#d9edf7;border-color:#bce8f1}.alert-info h4{color:#3a87ad}.alert-block{padding-top:14px;padding-bottom:14px}.alert-block>p,.alert-block>ul{margin-bottom:0}
 .alert-block p+p{margin-top:5px}.nav{margin-bottom:20px;margin-left:0;list-style:none}.nav>li>a{display:block}.nav>li>a:hover,.nav>li>a:focus{text-decoration:none;background-color:#eee}.nav>li>a>img{max-width:none}.nav>.pull-right{float:right}.nav-header{display:block;padding:3px 15px;font-size:11px;font-weight:bold;line-height:20px;color:#999;text-shadow:0 1px 0 rgba(255,255,255,0.5);text-transform:uppercase}.nav li+.nav-header{margin-top:9px}.nav-list{padding-right:15px;padding-left:15px;margin-bottom:0}.nav-list>li>a,.nav-list .nav-header{margin-right:-15px;margin-left:-15px;text-shadow:0 1px 0 rgba(255,255,255,0.5)}.nav-list>li>a{padding:3px 15px}.nav-list>.active>a,.nav-list>.active>a:hover,.nav-list>.active>a:focus{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.2);background-color:#08c}.nav-list [class^="icon-"],.nav-list [class*=" icon-"]{margin-right:2px}.nav-list .divider{*width:100%;height:1px;margin:9px 1px;*margin:-5px 0 5px;overflow:hidden;background-color:#e5e5e5;border-
 bottom:1px solid #fff}.nav-tabs,.nav-pills{*zoom:1}.nav-tabs:before,.nav-pills:before,.nav-tabs:after,.nav-pills:after{display:table;line-height:0;content:""}.nav-tabs:after,.nav-pills:after{clear:both}.nav-tabs>li,.nav-pills>li{float:left}.nav-tabs>li>a,.nav-pills>li>a{padding-right:12px;padding-left:12px;margin-right:2px;line-height:14px}.nav-tabs{border-bottom:1px solid #ddd}.nav-tabs>li{margin-bottom:-1px}.nav-tabs>li>a{padding-top:8px;padding-bottom:8px;line-height:20px;border:1px solid transparent;-webkit-border-radius:4px 4px 0 0;-moz-border-radius:4px 4px 0 0;border-radius:4px 4px 0 0}.nav-tabs>li>a:hover,.nav-tabs>li>a:focus{border-color:#eee #eeeeee #ddd}.nav-tabs>.active>a,.nav-tabs>.active>a:hover,.nav-tabs>.active>a:focus{color:#555;cursor:default;background-color:#fff;border:1px solid #ddd;border-bottom-color:transparent}.nav-pills>li>a{padding-top:8px;padding-bottom:8px;margin-top:2px;margin-bottom:2px;-webkit-border-radius:5px;-moz-border-radius:5px;border-radius:5px
 }.nav-pills>.active>a,.nav-pills>.active>a:hover,.nav-pills>.active>a:focus{color:#fff;background-color:#08c}.nav-stacked>li{float:none}.nav-stacked>li>a{margin-right:0}.nav-tabs.nav-stacked{border-bottom:0}.nav-tabs.nav-stacked>li>a{border:1px solid #ddd;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.nav-tabs.nav-stacked>li:first-child>a{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-topright:4px;-moz-border-radius-topleft:4px}.nav-tabs.nav-stacked>li:last-child>a{-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-moz-border-radius-bottomright:4px;-moz-border-radius-bottomleft:4px}.nav-tabs.nav-stacked>li>a:hover,.nav-tabs.nav-stacked>li>a:focus{z-index:2;border-color:#ddd}.nav-pills.nav-stacked>li>a{margin-bottom:3px}.nav-pills.nav-stacked>li:last-child>a{margin-bottom:1px}.nav-tabs 
 .dropdown-menu{-webkit-border-radius:0 0 6px 6px;-moz-border-radius:0 0 6px 6px;border-radius:0 0 6px 6px}.nav-pills .dropdown-menu{-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.nav .dropdown-toggle .caret{margin-top:6px;border-top-color:#08c;border-bottom-color:#08c}.nav .dropdown-toggle:hover .caret,.nav .dropdown-toggle:focus .caret{border-top-color:#005580;border-bottom-color:#005580}.nav-tabs .dropdown-toggle .caret{margin-top:8px}.nav .active .dropdown-toggle .caret{border-top-color:#fff;border-bottom-color:#fff}.nav-tabs .active .dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.nav>.dropdown.active>a:hover,.nav>.dropdown.active>a:focus{cursor:pointer}.nav-tabs .open .dropdown-toggle,.nav-pills .open .dropdown-toggle,.nav>li.dropdown.open.active>a:hover,.nav>li.dropdown.open.active>a:focus{color:#fff;background-color:#999;border-color:#999}.nav li.dropdown.open .caret,.nav li.dropdown.open.active .caret,.nav li.dropdown.open a:hover 
 .caret,.nav li.dropdown.open a:focus .caret{border-top-color:#fff;border-bottom-color:#fff;opacity:1;filter:alpha(opacity=100)}.tabs-stacked .open>a:hover,.tabs-stacked .open>a:focus{border-color:#999}.tabbable{*zoom:1}.tabbable:before,.tabbable:after{display:table;line-height:0;content:""}.tabbable:after{clear:both}.tab-content{overflow:auto}.tabs-below>.nav-tabs,.tabs-right>.nav-tabs,.tabs-left>.nav-tabs{border-bottom:0}.tab-content>.tab-pane,.pill-content>.pill-pane{display:none}.tab-content>.active,.pill-content>.active{display:block}.tabs-below>.nav-tabs{border-top:1px solid #ddd}.tabs-below>.nav-tabs>li{margin-top:-1px;margin-bottom:0}.tabs-below>.nav-tabs>li>a{-webkit-border-radius:0 0 4px 4px;-moz-border-radius:0 0 4px 4px;border-radius:0 0 4px 4px}.tabs-below>.nav-tabs>li>a:hover,.tabs-below>.nav-tabs>li>a:focus{border-top-color:#ddd;border-bottom-color:transparent}.tabs-below>.nav-tabs>.active>a,.tabs-below>.nav-tabs>.active>a:hover,.tabs-below>.nav-tabs>.active>a:focus{bo
 rder-color:transparent #ddd #ddd #ddd}.tabs-left>.nav-tabs>li,.tabs-right>.nav-tabs>li{float:none}.tabs-left>.nav-tabs>li>a,.tabs-right>.nav-tabs>li>a{min-width:74px;margin-right:0;margin-bottom:3px}.tabs-left>.nav-tabs{float:left;margin-right:19px;border-right:1px solid #ddd}.tabs-left>.nav-tabs>li>a{margin-right:-1px;-webkit-border-radius:4px 0 0 4px;-moz-border-radius:4px 0 0 4px;border-radius:4px 0 0 4px}.tabs-left>.nav-tabs>li>a:hover,.tabs-left>.nav-tabs>li>a:focus{border-color:#eee #dddddd #eee #eeeeee}.tabs-left>.nav-tabs .active>a,.tabs-left>.nav-tabs .active>a:hover,.tabs-left>.nav-tabs .active>a:focus{border-color:#ddd transparent #ddd #ddd;*border-right-color:#fff}.tabs-right>.nav-tabs{float:right;margin-left:19px;border-left:1px solid #ddd}.tabs-right>.nav-tabs>li>a{margin-left:-1px;-webkit-border-radius:0 4px 4px 0;-moz-border-radius:0 4px 4px 0;border-radius:0 4px 4px 0}.tabs-right>.nav-tabs>li>a:hover,.tabs-right>.nav-tabs>li>a:focus{border-color:#eee #eeeeee #eee #d
 ddddd}.tabs-right>.nav-tabs .active>a,.tabs-right>.nav-tabs .active>a:hover,.tabs-right>.nav-tabs .active>a:focus{border-color:#ddd #ddd #ddd transparent;*border-left-color:#fff}.nav>.disabled>a{color:#999}.nav>.disabled>a:hover,.nav>.disabled>a:focus{text-decoration:none;cursor:default;background-color:transparent}.navbar{*position:relative;*z-index:2;margin-bottom:20px;overflow:visible}.navbar-inner{min-height:40px;padding-right:20px;padding-left:20px;background-color:#fafafa;background-image:-moz-linear-gradient(top,#fff,#f2f2f2);background-image:-webkit-gradient(linear,0 0,0 100%,from(#fff),to(#f2f2f2));background-image:-webkit-linear-gradient(top,#fff,#f2f2f2);background-image:-o-linear-gradient(top,#fff,#f2f2f2);background-image:linear-gradient(to bottom,#fff,#f2f2f2);background-repeat:repeat-x;border:1px solid #d4d4d4;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffffffff',endColorstr='#fff
 2f2f2',GradientType=0);*zoom:1;-webkit-box-shadow:0 1px 4px rgba(0,0,0,0.065);-moz-box-shadow:0 1px 4px rgba(0,0,0,0.065);box-shadow:0 1px 4px rgba(0,0,0,0.065)}.navbar-inner:before,.navbar-inner:after{display:table;line-height:0;content:""}.navbar-inner:after{clear:both}.navbar .container{width:auto}.nav-collapse.collapse{height:auto;overflow:visible}.navbar .brand{display:block;float:left;padding:10px 20px 10px;margin-left:-20px;font-size:20px;font-weight:200;color:#777;text-shadow:0 1px 0 #fff}.navbar .brand:hover,.navbar .brand:focus{text-decoration:none}.navbar-text{margin-bottom:0;line-height:40px;color:#777}.navbar-link{color:#777}.navbar-link:hover,.navbar-link:focus{color:#333}.navbar .divider-vertical{height:40px;margin:0 9px;border-right:1px solid #fff;border-left:1px solid #f2f2f2}.navbar .btn,.navbar .btn-group{margin-top:5px}.navbar .btn-group .btn,.navbar .input-prepend .btn,.navbar .input-append .btn,.navbar .input-prepend .btn-group,.navbar .input-append .btn-group{
 margin-top:0}.navbar-form{margin-bottom:0;*zoom:1}.navbar-form:before,.navbar-form:after{display:table;line-height:0;content:""}.navbar-form:after{clear:both}.navbar-form input,.navbar-form select,.navbar-form .radio,.navbar-form .checkbox{margin-top:5px}.navbar-form input,.navbar-form select,.navbar-form .btn{display:inline-block;margin-bottom:0}.navbar-form input[type="image"],.navbar-form input[type="checkbox"],.navbar-form input[type="radio"]{margin-top:3px}.navbar-form .input-append,.navbar-form .input-prepend{margin-top:5px;white-space:nowrap}.navbar-form .input-append input,.navbar-form .input-prepend input{margin-top:0}.navbar-search{position:relative;float:left;margin-top:5px;margin-bottom:0}.navbar-search .search-query{padding:4px 14px;margin-bottom:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:13px;font-weight:normal;line-height:1;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.navbar-static-top{position:static;margin-bottom:0}
 .navbar-static-top .navbar-inner{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.navbar-fixed-top,.navbar-fixed-bottom{position:fixed;right:0;left:0;z-index:1030;margin-bottom:0}.navbar-fixed-top .navbar-inner,.navbar-static-top .navbar-inner{border-width:0 0 1px}.navbar-fixed-bottom .navbar-inner{border-width:1px 0 0}.navbar-fixed-top .navbar-inner,.navbar-fixed-bottom .navbar-inner{padding-right:0;padding-left:0;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:940px}.navbar-fixed-top{top:0}.navbar-fixed-top .navbar-inner,.navbar-static-top .navbar-inner{-webkit-box-shadow:0 1px 10px rgba(0,0,0,0.1);-moz-box-shadow:0 1px 10px rgba(0,0,0,0.1);box-shadow:0 1px 10px rgba(0,0,0,0.1)}.navbar-fixed-bottom{bottom:0}.navbar-fixed-bottom .navbar-inner{-webkit-box-shadow:0 -1px 10px rgba(0,0,0,0.1);-moz-box-shadow:0 -1px 10px rgba(0,0,0,0.1);box-shadow:0 -1px 10px rgba(0
 ,0,0,0.1)}.navbar .nav{position:relative;left:0;display:block;float:left;margin:0 10px 0 0}.navbar .nav.pull-right{float:right;margin-right:0}.navbar .nav>li{float:left}.navbar .nav>li>a{float:none;padding:10px 15px 10px;color:#777;text-decoration:none;text-shadow:0 1px 0 #fff}.navbar .nav .dropdown-toggle .caret{margin-top:8px}.navbar .nav>li>a:focus,.navbar .nav>li>a:hover{color:#333;text-decoration:none;background-color:transparent}.navbar .nav>.active>a,.navbar .nav>.active>a:hover,.navbar .nav>.active>a:focus{color:#555;text-decoration:none;background-color:#e5e5e5;-webkit-box-shadow:inset 0 3px 8px rgba(0,0,0,0.125);-moz-box-shadow:inset 0 3px 8px rgba(0,0,0,0.125);box-shadow:inset 0 3px 8px rgba(0,0,0,0.125)}.navbar .btn-navbar{display:none;float:right;padding:7px 10px;margin-right:5px;margin-left:5px;color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#ededed;*background-color:#e5e5e5;background-image:-moz-linear-gradient(top,#f2f2f2,#e5e5e5);background-image:-
 webkit-gradient(linear,0 0,0 100%,from(#f2f2f2),to(#e5e5e5));background-image:-webkit-linear-gradient(top,#f2f2f2,#e5e5e5);background-image:-o-linear-gradient(top,#f2f2f2,#e5e5e5);background-image:linear-gradient(to bottom,#f2f2f2,#e5e5e5);background-repeat:repeat-x;border-color:#e5e5e5 #e5e5e5 #bfbfbf;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff2f2f2',endColorstr='#ffe5e5e5',GradientType=0);filter:progid:DXImageTransform.Microsoft.gradient(enabled=false);-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075);-moz-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075);box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075)}.navbar .btn-navbar:hover,.navbar .btn-navbar:focus,.navbar .btn-navbar:active,.navbar .btn-navbar.active,.navbar .btn-navbar.disabled,.navbar .btn-navbar[disabled]{color:#fff;background-color:#e5e5e5
 ;*background-color:#d9d9d9}.navbar .btn-navbar:active,.navbar .btn-navbar.active{background-color:#ccc \9}.navbar .btn-navbar .icon-bar{display:block;width:18px;height:2px;background-color:#f5f5f5;-webkit-border-radius:1px;-moz-border-radius:1px;border-radius:1px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,0.25);-moz-box-shadow:0 1px 0 rgba(0,0,0,0.25);box-shadow:0 1px 0 rgba(0,0,0,0.25)}.btn-navbar .icon-bar+.icon-bar{margin-top:3px}.navbar .nav>li>.dropdown-menu:before{position:absolute;top:-7px;left:9px;display:inline-block;border-right:7px solid transparent;border-bottom:7px solid #ccc;border-left:7px solid transparent;border-bottom-color:rgba(0,0,0,0.2);content:''}.navbar .nav>li>.dropdown-menu:after{position:absolute;top:-6px;left:10px;display:inline-block;border-right:6px solid transparent;border-bottom:6px solid #fff;border-left:6px solid transparent;content:''}.navbar-fixed-bottom .nav>li>.dropdown-menu:before{top:auto;bottom:-7px;border-top:7px solid #ccc;border-bottom:0;border-
 top-color:rgba(0,0,0,0.2)}.navbar-fixed-bottom .nav>li>.dropdown-menu:after{top:auto;bottom:-6px;border-top:6px solid #fff;border-bottom:0}.navbar .nav li.dropdown>a:hover .caret,.navbar .nav li.dropdown>a:focus .caret{border-top-color:#333;border-bottom-color:#333}.navbar .nav li.dropdown.open>.dropdown-toggle,.navbar .nav li.dropdown.active>.dropdown-toggle,.navbar .nav li.dropdown.open.active>.dropdown-toggle{color:#555;background-color:#e5e5e5}.navbar .nav li.dropdown>.dropdown-toggle .caret{border-top-color:#777;border-bottom-color:#777}.navbar .nav li.dropdown.open>.dropdown-toggle .caret,.navbar .nav li.dropdown.active>.dropdown-toggle .caret,.navbar .nav li.dropdown.open.active>.dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.navbar .pull-right>li>.dropdown-menu,.navbar .nav>li>.dropdown-menu.pull-right{right:0;left:auto}.navbar .pull-right>li>.dropdown-menu:before,.navbar .nav>li>.dropdown-menu.pull-right:before{right:12px;left:auto}.navbar .pull-righ
 t>li>.dropdown-menu:after,.navbar .nav>li>.dropdown-menu.pull-right:after{right:13px;left:auto}.navbar .pull-right>li>.dropdown-menu .dropdown-menu,.navbar .nav>li>.dropdown-menu.pull-right .dropdown-menu{right:100%;left:auto;margin-right:-1px;margin-left:0;-webkit-border-radius:6px 0 6px 6px;-moz-border-radius:6px 0 6px 6px;border-radius:6px 0 6px 6px}.navbar-inverse .navbar-inner{background-color:#1b1b1b;background-image:-moz-linear-gradient(top,#222,#111);background-image:-webkit-gradient(linear,0 0,0 100%,from(#222),to(#111));background-image:-webkit-linear-gradient(top,#222,#111);background-image:-o-linear-gradient(top,#222,#111);background-image:linear-gradient(to bottom,#222,#111);background-repeat:repeat-x;border-color:#252525;filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff222222',endColorstr='#ff111111',GradientType=0)}.navbar-inverse .brand,.navbar-inverse .nav>li>a{color:#999;text-shadow:0 -1px 0 rgba(0,0,0,0.25)}.navbar-inverse .brand:hover,.navbar-i
 nverse .nav>li>a:hover,.navbar-inverse .brand:focus,.navbar-inverse .nav>li>a:focus{color:#fff}.navbar-inverse .brand{color:#999}.navbar-inverse .navbar-text{color:#999}.navbar-inverse .nav>li>a:focus,.navbar-inverse .nav>li>a:hover{color:#fff;background-color:transparent}.navbar-inverse .nav .active>a,.navbar-inverse .nav .active>a:hover,.navbar-inverse .nav .active>a:focus{color:#fff;background-color:#111}.navbar-inverse .navbar-link{color:#999}.navbar-inverse .navbar-link:hover,.navbar-inverse .navbar-link:focus{color:#fff}.navbar-inverse .divider-vertical{border-right-color:#222;border-left-color:#111}.navbar-inverse .nav li.dropdown.open>.dropdown-toggle,.navbar-inverse .nav li.dropdown.active>.dropdown-toggle,.navbar-inverse .nav li.dropdown.open.active>.dropdown-toggle{color:#fff;background-color:#111}.navbar-inverse .nav li.dropdown>a:hover .caret,.navbar-inverse .nav li.dropdown>a:focus .caret{border-top-color:#fff;border-bottom-color:#fff}.navbar-inverse .nav li.dropdown>.
 dropdown-toggle .caret{border-top-color:#999;border-bottom-color:#999}.navbar-inverse .nav li.dropdown.open>.dropdown-toggle .caret,.navbar-inverse .nav li.dropdown.active>.dropdown-toggle .caret,.navbar-inverse .nav li.dropdown.open.active>.dropdown-toggle .caret{border-top-color:#fff;border-bottom-color:#fff}.navbar-inverse .navbar-search .search-query{color:#fff;background-color:#515151;border-color:#111;-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);-moz-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);box-shadow:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);-webkit-transition:none;-moz-transition:none;-o-transition:none;transition:none}.navbar-inverse .navbar-search .search-query:-moz-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query:-ms-input-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query::-webkit-input-placeholder{color:#ccc}.navbar-inverse .navbar-search .searc
 h-query:focus,.navbar-inverse .navbar-search .search-query.focused{padding:5px 15px;color:#333;text-shadow:0 1px 0 #fff;background-color:#fff;border:0;outline:0;-webkit-box-shadow:0 0 3px rgba(0,0,0,0.15);-moz-box-shadow:0 0 3px rgba(0,0,0,0.15);box-shadow:0 0 3px rgba(0,0,0,0.15)}.navbar-inverse .btn-navbar{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#0e0e0e;*background-color:#040404;background-image:-moz-linear-gradient(top,#151515,#040404);background-image:-webkit-gradient(linear,0 0,0 100%,from(#151515),to(#040404));background-image:-webkit-linear-gradient(top,#151515,#040404);background-image:-o-linear-gradient(top,#151515,#040404);background-image:linear-gradient(to bottom,#151515,#040404);background-repeat:repeat-x;border-color:#040404 #040404 #000;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff151515',endColorstr='#ff040404',GradientType=0);filter:progid:DXImageTransform.
 Microsoft.gradient(enabled=false)}.navbar-inverse .btn-navbar:hover,.navbar-inverse .btn-navbar:focus,.navbar-inverse .btn-navbar:active,.navbar-inverse .btn-navbar.active,.navbar-inverse .btn-navbar.disabled,.navbar-inverse .btn-navbar[disabled]{color:#fff;background-color:#040404;*background-color:#000}.navbar-inverse .btn-navbar:active,.navbar-inverse .btn-navbar.active{background-color:#000 \9}.breadcrumb{padding:8px 15px;margin:0 0 20px;list-style:none;background-color:#f5f5f5;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.breadcrumb>li{display:inline-block;*display:inline;text-shadow:0 1px 0 #fff;*zoom:1}.breadcrumb>li>.divider{padding:0 5px;color:#ccc}.breadcrumb>.active{color:#999}.pagination{margin:20px 0}.pagination ul{display:inline-block;*display:inline;margin-bottom:0;margin-left:0;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;*zoom:1;-webkit-box-shadow:0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:0 1px 2px rgba(0,0,0,0.05);box-sh
 adow:0 1px 2px rgba(0,0,0,0.05)}.pagination ul>li{display:inline}.pagination ul>li>a,.pagination ul>li>span{float:left;padding:4px 12px;line-height:20px;text-decoration:none;background-color:#fff;border:1px solid #ddd;border-left-width:0}.pagination ul>li>a:hover,.pagination ul>li>a:focus,.pagination ul>.active>a,.pagination ul>.active>span{background-color:#f5f5f5}.pagination ul>.active>a,.pagination ul>.active>span{color:#999;cursor:default}.pagination ul>.disabled>span,.pagination ul>.disabled>a,.pagination ul>.disabled>a:hover,.pagination ul>.disabled>a:focus{color:#999;cursor:default;background-color:transparent}.pagination ul>li:first-child>a,.pagination ul>li:first-child>span{border-left-width:1px;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-bottomleft:4px;-moz-border-radius-topleft:4px}.pagination ul>li:last-child>a,.pagination ul>li:last-child>span{-webkit-border-top-righ
 t-radius:4px;border-top-right-radius:4px;-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-moz-border-radius-topright:4px;-moz-border-radius-bottomright:4px}.pagination-centered{text-align:center}.pagination-right{text-align:right}.pagination-large ul>li>a,.pagination-large ul>li>span{padding:11px 19px;font-size:17.5px}.pagination-large ul>li:first-child>a,.pagination-large ul>li:first-child>span{-webkit-border-bottom-left-radius:6px;border-bottom-left-radius:6px;-webkit-border-top-left-radius:6px;border-top-left-radius:6px;-moz-border-radius-bottomleft:6px;-moz-border-radius-topleft:6px}.pagination-large ul>li:last-child>a,.pagination-large ul>li:last-child>span{-webkit-border-top-right-radius:6px;border-top-right-radius:6px;-webkit-border-bottom-right-radius:6px;border-bottom-right-radius:6px;-moz-border-radius-topright:6px;-moz-border-radius-bottomright:6px}.pagination-mini ul>li:first-child>a,.pagination-small ul>li:first-child>a,.pagination-mini ul>li:first
 -child>span,.pagination-small ul>li:first-child>span{-webkit-border-bottom-left-radius:3px;border-bottom-left-radius:3px;-webkit-border-top-left-radius:3px;border-top-left-radius:3px;-moz-border-radius-bottomleft:3px;-moz-border-radius-topleft:3px}.pagination-mini ul>li:last-child>a,.pagination-small ul>li:last-child>a,.pagination-mini ul>li:last-child>span,.pagination-small ul>li:last-child>span{-webkit-border-top-right-radius:3px;border-top-right-radius:3px;-webkit-border-bottom-right-radius:3px;border-bottom-right-radius:3px;-moz-border-radius-topright:3px;-moz-border-radius-bottomright:3px}.pagination-small ul>li>a,.pagination-small ul>li>span{padding:2px 10px;font-size:11.9px}.pagination-mini ul>li>a,.pagination-mini ul>li>span{padding:0 6px;font-size:10.5px}.pager{margin:20px 0;text-align:center;list-style:none;*zoom:1}.pager:before,.pager:after{display:table;line-height:0;content:""}.pager:after{clear:both}.pager li{display:inline}.pager li>a,.pager li>span{display:inline-blo
 ck;padding:5px 14px;background-color:#fff;border:1px solid #ddd;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.pager li>a:hover,.pager li>a:focus{text-decoration:none;background-color:#f5f5f5}.pager .next>a,.pager .next>span{float:right}.pager .previous>a,.pager .previous>span{float:left}.pager .disabled>a,.pager .disabled>a:hover,.pager .disabled>a:focus,.pager .disabled>span{color:#999;cursor:default;background-color:#fff}.modal-backdrop{position:fixed;top:0;right:0;bottom:0;left:0;z-index:1040;background-color:#000}.modal-backdrop.fade{opacity:0}.modal-backdrop,.modal-backdrop.fade.in{opacity:.8;filter:alpha(opacity=80)}.modal{position:fixed;top:10%;left:50%;z-index:1050;width:560px;margin-left:-280px;background-color:#fff;border:1px solid #999;border:1px solid rgba(0,0,0,0.3);*border:1px solid #999;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;outline:0;-webkit-box-shadow:0 3px 7px rgba(0,0,0,0.3);-moz-box-shadow:0 3px 7px rgba(0,0,0,
 0.3);box-shadow:0 3px 7px rgba(0,0,0,0.3);-webkit-background-clip:padding-box;-moz-background-clip:padding-box;background-clip:padding-box}.modal.fade{top:-25%;-webkit-transition:opacity .3s linear,top .3s ease-out;-moz-transition:opacity .3s linear,top .3s ease-out;-o-transition:opacity .3s linear,top .3s ease-out;transition:opacity .3s linear,top .3s ease-out}.modal.fade.in{top:10%}.modal-header{padding:9px 15px;border-bottom:1px solid #eee}.modal-header .close{margin-top:2px}.modal-header h3{margin:0;line-height:30px}.modal-body{position:relative;max-height:400px;padding:15px;overflow-y:auto}.modal-form{margin-bottom:0}.modal-footer{padding:14px 15px 15px;margin-bottom:0;text-align:right;background-color:#f5f5f5;border-top:1px solid #ddd;-webkit-border-radius:0 0 6px 6px;-moz-border-radius:0 0 6px 6px;border-radius:0 0 6px 6px;*zoom:1;-webkit-box-shadow:inset 0 1px 0 #fff;-moz-box-shadow:inset 0 1px 0 #fff;box-shadow:inset 0 1px 0 #fff}.modal-footer:before,.modal-footer:after{dis
 play:table;line-height:0;content:""}.modal-footer:after{clear:both}.modal-footer .btn+.btn{margin-bottom:0;margin-left:5px}.modal-footer .btn-group .btn+.btn{margin-left:-1px}.modal-footer .btn-block+.btn-block{margin-left:0}.tooltip{position:absolute;z-index:1030;display:block;font-size:11px;line-height:1.4;opacity:0;filter:alpha(opacity=0);visibility:visible}.tooltip.in{opacity:.8;filter:alpha(opacity=80)}.tooltip.top{padding:5px 0;margin-top:-3px}.tooltip.right{padding:0 5px;margin-left:3px}.tooltip.bottom{padding:5px 0;margin-top:3px}.tooltip.left{padding:0 5px;margin-left:-3px}.tooltip-inner{max-width:200px;padding:8px;color:#fff;text-align:center;text-decoration:none;background-color:#000;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.tooltip-arrow{position:absolute;width:0;height:0;border-color:transparent;border-style:solid}.tooltip.top .tooltip-arrow{bottom:0;left:50%;margin-left:-5px;border-top-color:#000;border-width:5px 5px 0}.tooltip.right .tooltip-
 arrow{top:50%;left:0;margin-top:-5px;border-right-color:#000;border-width:5px 5px 5px 0}.tooltip.left .tooltip-arrow{top:50%;right:0;margin-top:-5px;border-left-color:#000;border-width:5px 0 5px 5px}.tooltip.bottom .tooltip-arrow{top:0;left:50%;margin-left:-5px;border-bottom-color:#000;border-width:0 5px 5px}.popover{position:absolute;top:0;left:0;z-index:1010;display:none;max-width:276px;padding:1px;text-align:left;white-space:normal;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 5px 10px rgba(0,0,0,0.2);-moz-box-shadow:0 5px 10px rgba(0,0,0,0.2);box-shadow:0 5px 10px rgba(0,0,0,0.2);-webkit-background-clip:padding-box;-moz-background-clip:padding;background-clip:padding-box}.popover.top{margin-top:-10px}.popover.right{margin-left:10px}.popover.bottom{margin-top:10px}.popover.left{margin-left:-10px}.popover-title{padding:8px 14px;margin:0;font-size:14px;font-weight:
 normal;line-height:18px;background-color:#f7f7f7;border-bottom:1px solid #ebebeb;-w

<TRUNCATED>

[02/50] [abbrv] metron git commit: METRON-1553 Validate JIRA Script Error (nickwallen) closes apache/metron#1013

Posted by mm...@apache.org.
METRON-1553 Validate JIRA Script Error (nickwallen) closes apache/metron#1013


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/45e3ed87
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/45e3ed87
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/45e3ed87

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: 45e3ed87ca4aceda7c8387389fc3a286ee5264d9
Parents: dbcc52c
Author: nickwallen <ni...@nickallen.org>
Authored: Tue Jun 5 11:50:58 2018 -0400
Committer: nickallen <ni...@apache.org>
Committed: Tue Jun 5 11:50:58 2018 -0400

----------------------------------------------------------------------
 .../release-utils/validate-jira-for-release     | 75 ++++++++++++--------
 1 file changed, 46 insertions(+), 29 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/45e3ed87/dev-utilities/release-utils/validate-jira-for-release
----------------------------------------------------------------------
diff --git a/dev-utilities/release-utils/validate-jira-for-release b/dev-utilities/release-utils/validate-jira-for-release
index df5776c..5fd4367 100755
--- a/dev-utilities/release-utils/validate-jira-for-release
+++ b/dev-utilities/release-utils/validate-jira-for-release
@@ -137,35 +137,52 @@ for i in "$@"; do
   esac
 done
 
-WORKDIR="~/tmp"
 
 # ensure all required values are set
 if [ -z "$VERSION" ]; then
-	echo "Missing -v/--version is is required"
-	exit 1
+  help
+  echo "Missing -v/--version is is required"
+  exit 1
 fi
 if [ -z "$START" ]; then
-	echo "Missing -s/--start which is required"
-	exit 1
+  help
+  echo "Missing -s/--start which is required"
+  exit 1
 fi
 if [ -z "$END" ]; then
-	echo "Missing -e/--end which is required"
-	exit 1
+  help
+  echo "Missing -e/--end which is required"
+  exit 1
 fi
 if [ -z "$REPO" ]; then
-	echo "Missing -r/--repo which is required"
-	exit 1
+  help
+  echo "Missing -r/--repo which is required"
+  exit 1
 fi
 if [ -z "$BRANCH" ]; then
-	echo "Missing -b/--branch which is required"
-	exit 1
+  help
+  echo "Missing -b/--branch which is required"
+  exit 1
 fi
 
-# clone the metron repo and fetch all tags
-git clone $REPO "metron-$VERSION"
+TMPDIR="$HOME/tmp"
+WORKDIR="$TMPDIR/metron-$VERSION"
+
+# warn the user if the working directory exists
+if [ -d "$WORKDIR" ]; then
+  read -p "  directory exists [$WORKDIR].  overwrite existing repo? [yN] " -n 1 -r
+  echo
+  if [[ ! $REPLY =~ ^[Yy]$ ]]; then
+    exit 1
+  fi
+fi
+
+# fetch the repo and all tags
+rm -rf "$WORKDIR"
+git clone $REPO "$WORKDIR"
+cd "$WORKDIR"
 git checkout $BRANCH
-cd "$WORKDIR/metron-$VERSION"
-git fetch --all --tags
+git fetch --tags
 
 # find all JIRAs that have been committed since the last release
 GET_JIRAS="git log $START..$END --oneline | grep -E -o 'METRON[- ]*[0-9]+'"
@@ -177,21 +194,21 @@ printf "$FORMAT_STR" "JIRA" "STATUS" "FIX VERSION" "ASSIGNEE" "FIX"
 # for each JIRA since the last release tag...
 eval $GET_JIRAS | while read JIRA ; do
 
-	# fetch the JIRA content
-	URL="https://issues.apache.org/jira/si/jira.issueviews:issue-xml/$JIRA/$JIRA.xml"
-        CONTENT=`curl -s $URL`
+  # fetch the JIRA content
+  URL="https://issues.apache.org/jira/si/jira.issueviews:issue-xml/$JIRA/$JIRA.xml"
+  CONTENT=`curl -s $URL`
 
-	# painfully extract some fields
-	STATUS=`echo "$CONTENT" | grep "<status[^>]*>" | sed 's/^.*<status[^>]*>//' | sed 's/<.status>.*$//'`
-	ASSIGNEE=`echo "$CONTENT" | grep "<assignee[^>]*>" | sed 's/^.*<assignee[^>]*>//' | sed 's/<.assignee>.*$//'`
-	FIXV=`echo "$CONTENT" | grep "<fixVersion[^>]*>" | sed 's/^.*<fixVersion[^>]*>//' | sed 's/<.fixVersion>.*$//'`
+  # painfully extract some fields
+  STATUS=`echo "$CONTENT" | grep "<status[^>]*>" | sed 's/^.*<status[^>]*>//' | sed 's/<.status>.*$//'`
+  ASSIGNEE=`echo "$CONTENT" | grep "<assignee[^>]*>" | sed 's/^.*<assignee[^>]*>//' | sed 's/<.assignee>.*$//'`
+  FIXV=`echo "$CONTENT" | grep "<fixVersion[^>]*>" | sed 's/^.*<fixVersion[^>]*>//' | sed 's/<.fixVersion>.*$//'`
 
-	# the link is only populated, if there is something to fix
-	LINK=""
-	if [ "$FIXV" != "$VERSION" ] || [ "$STATUS" != "Done" ]; then
-		LINK="https://issues.apache.org/jira/browse/$JIRA"
-	fi
+  # the link is only populated, if there is something to fix
+  LINK=""
+  if [ "$FIXV" != "$VERSION" ] || [ "$STATUS" != "Done" ]; then
+    LINK="https://issues.apache.org/jira/browse/$JIRA"
+  fi
 
-	# show the JIRA
-      	printf "$FORMAT_STR" "$JIRA" "$STATUS" "$FIXV" "$ASSIGNEE" "$LINK"
+  # show the JIRA
+  printf "$FORMAT_STR" "$JIRA" "$STATUS" "$FIXV" "$ASSIGNEE" "$LINK"
 done


[13/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-parsers/parser-testing.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-parsers/parser-testing.html b/site/current-book/metron-platform/metron-parsers/parser-testing.html
index 0edfd53..ff22013 100644
--- a/site/current-book/metron-platform/metron-parsers/parser-testing.html
+++ b/site/current-book/metron-platform/metron-parsers/parser-testing.html
@@ -1,356 +1,180 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-parsers/parser-testing.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Parser Contribution and Testing</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Parser Contribution and Testing</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Parser Contribution and Testing</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                                  
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-down"></i>
-        Parsers</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>parser-testing</a>
-          </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-down"></span>Parsers</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/metron-parsers/3rdPartyParser.html" title="3rdPartyParser"><span class="none"></span>3rdPartyParser</a></li>
+    <li class="active"><a href="#"><span class="none"></span>parser-testing</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Parser Contribution and Testing</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Parser Contribution and Testing</h1>
 <p><a name="Parser_Contribution_and_Testing"></a></p>
-<p>So you want to contribute a parser to Apache Metron. First off, on behalf of the community, thank you very much! Now that you have implemented a parser by writing a java class which implements <tt>org.apache.metron.parsers.interfaces.MessageParser</tt> what are the testing expectations for a new parser?</p>
+<p>So you want to contribute a parser to Apache Metron.  First off, on behalf of the community, thank you very much!  Now that you have implemented a parser by writing a java class which implements <tt>org.apache.metron.parsers.interfaces.MessageParser</tt> what are the testing expectations for a new parser?</p>
 <p>It is expected that a new parser have two tests:</p>
-
 <ul>
-  
+
 <li>A JUnit test directly testing your parser class.</li>
-  
 <li>An Integration test validating that your parser class can parse messages inside the <tt>ParserBolt</tt>.</li>
 </ul>
 <div class="section">
 <h2><a name="The_JUnit_Test"></a>The JUnit Test</h2>
-<p>The JUnit Test should be focused on testing your Parser directly. You should feel free to use mocks or stubs or whatever else you need to completely test that unit of functionality.</p></div>
+<p>The JUnit Test should be focused on testing your Parser directly.  You should feel free to use mocks or stubs or whatever else you need to completely test that unit of functionality.</p></div>
 <div class="section">
 <h2><a name="The_Integration_Test"></a>The Integration Test</h2>
-<p>Integration tests are more structured. The intent is that the parser that you have implemented can be driven successfully from <tt>org.apache.metron.parsers.bolt.ParserBolt</tt>.</p>
+<p>Integration tests are more structured.  The intent is that the parser that you have implemented can be driven successfully from <tt>org.apache.metron.parsers.bolt.ParserBolt</tt>.</p>
 <p>The procedure for creating a new test is as follows:</p>
-
 <ul>
-  
+
 <li>Create an integration test that extends <tt>org.apache.metron.parsers.integration.ParserIntegrationTest</tt>
-  
 <ul>
-    
+
 <li>Override <tt>getSensorType()</tt> to return the sensor type to be used in the test (referred to as <tt>${sensor_type}</tt> at times)</li>
-    
 <li>Override <tt>getValidations()</tt> to indicate how you want the output of the parser to be validated (more on validations later)</li>
-    
 <li>Optionally <tt>readSensorConfig(String sensorType)</tt> to read the sensor config
-    
 <ul>
-      
-<li>By default, we will pull this from <tt>metron-parsers/src/main/config/zookeeper/parsers/${sensor_type}</tt>. Override if you want to provide your own</li>
-    </ul></li>
-    
+
+<li>By default, we will pull this from <tt>metron-parsers/src/main/config/zookeeper/parsers/${sensor_type}</tt>.  Override if you want to provide your own</li>
+</ul>
+</li>
 <li>Optionally <tt>readGlobalConfig()</tt> to return the global config
-    
 <ul>
-      
-<li>By default, we will pull this from <tt>metron-integration-test/src/main/config/zookeeper/global.json)</tt>. Override if you want to provide your own</li>
-    </ul></li>
-  </ul></li>
-  
+
+<li>By default, we will pull this from <tt>metron-integration-test/src/main/config/zookeeper/global.json)</tt>.  Override if you want to provide your own</li>
+</ul>
+</li>
+</ul>
+</li>
 <li>Place sample input data in <tt>metron-integration-test/src/main/sample/data/${sensor_type}/raw</tt>
-  
 <ul>
-    
+
 <li>It should be one line per input record.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Place expected output based on sample data in <tt>metron-integration-test/src/main/sample/data/${sensor_type}/parsed</tt>
-  
 <ul>
-    
+
 <li>Line <tt>k</tt> in the expected data should match with line <tt>k</tt></li>
-  </ul></li>
 </ul>
-<p>The way these tests function is by creating a <tt>ParserBolt</tt> instance with your specified global configuration and sensor configuration. It will then send your specified sample input data in line-by-line. It will then perform some basic sanity validation:</p>
-
+</li>
+</ul>
+<p>The way these tests function is by creating a <tt>ParserBolt</tt> instance with your specified global configuration and sensor configuration.  It will then send your specified sample input data in line-by-line.  It will then perform some basic sanity validation:</p>
 <ul>
-  
+
 <li>Ensure no errors were logged</li>
-  
 <li>Execute your specified validation methods</li>
 </ul>
 <div class="section">
 <h3><a name="Validations"></a>Validations</h3>
-<p>Validations are functions which indicate how one should validate the parsed messages. The basic one which is sufficient for most cases is <tt>org.apache.metron.parsers.integration.validation.SampleDataValidation</tt>. This will read the expected results from <tt>metron-integration-test/src/main/sample/data/${sensor_type}/parsed</tt> and validate that the actual parsed message conforms (excluding timestamp).</p>
+<p>Validations are functions which indicate how one should validate the parsed messages.  The basic one which is sufficient for most cases is <tt>org.apache.metron.parsers.integration.validation.SampleDataValidation</tt>.  This will read the expected results from <tt>metron-integration-test/src/main/sample/data/${sensor_type}/parsed</tt> and validate that the actual parsed message conforms (excluding timestamp).</p>
 <p>If you have special validations required, you may implement your own and return an instance of that in the <tt>getValidations()</tt> method of your Integration Test.</p></div>
 <div class="section">
 <h3><a name="Sample_Integration_Test"></a>Sample Integration Test</h3>
 <p>A sample integration test for the <tt>snort</tt> parser is as follows:</p>
 
-<div class="source">
-<div class="source">
-<pre>public class SnortIntegrationTest extends ParserIntegrationTest {
+<div>
+<div>
+<pre class="source">public class SnortIntegrationTest extends ParserIntegrationTest {
   @Override
   String getSensorType() {
     return &quot;snort&quot;;
@@ -364,23 +188,17 @@
   }
 }
 </pre></div></div></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-pcap-backend/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-pcap-backend/index.html b/site/current-book/metron-platform/metron-pcap-backend/index.html
index 01961eb..5206fd0 100644
--- a/site/current-book/metron-platform/metron-pcap-backend/index.html
+++ b/site/current-book/metron-platform/metron-pcap-backend/index.html
@@ -1,408 +1,217 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-pcap-backend/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Metron PCAP Backend</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Metron PCAP Backend</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron PCAP Backend</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Pcap-backend</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Metron PCAP Backend</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron PCAP Backend</h1>
 <p><a name="Metron_PCAP_Backend"></a></p>
 <p>The purpose of the Metron PCAP backend is to create a storm topology capable of rapidly ingesting raw packet capture data directly into HDFS from Kafka.</p>
-
 <ul>
-  
+
 <li><a href="#the-sensors-feeding-kafka">Sensors</a></li>
-  
 <li><a href="#the-pcap-topology">PCAP Topology</a></li>
-  
 <li><a href="#the-files-on-hdfs">HDFS Files</a></li>
-  
 <li><a href="#Configuration">Configuration</a></li>
-  
 <li><a href="#Starting_the_Topology">Starting the Topology</a></li>
-  
 <li><a href="#Utilities">Utilities</a>
-  
 <ul>
-    
+
 <li><a href="#Inspector_Utility">Inspector Utility</a></li>
-    
 <li><a href="#Query_Filter_Utility">Query Filter Utility</a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><a href="#Performance_Tuning">Performance Tuning</a></li>
 </ul>
 <div class="section">
 <h2><a name="The_Sensors_Feeding_Kafka"></a>The Sensors Feeding Kafka</h2>
-<p>This component must be fed by fast packet capture components upstream via Kafka. The two supported components shipped with Metron are as follows:</p>
-
+<p>This component must be fed by fast packet capture components upstream via Kafka.  The two supported components shipped with Metron are as follows:</p>
 <ul>
-  
+
 <li>The pycapa <a href="../../metron-sensors/pycapa/index.html">tool</a> aimed at low-volume packet capture</li>
-  
 <li>The <a class="externalLink" href="http://dpdk.org/">DPDK</a> based <a href="../../metron-sensors/fastcapa/index.html">tool</a> aimed at high-volume packet capture</li>
 </ul>
 <p>Both of these sensors feed kafka raw packet data directly into Kafka. The format of the record structure that this component expects is the following:</p>
-
 <ul>
-  
+
 <li>A key which is the byte representation of a 64-bit <tt>unsigned long</tt> representing a time-unit since the unix epoch</li>
-  
 <li>A value which is the raw packet data without header (either global pcap header or packet header)</li>
 </ul></div>
 <div class="section">
 <h2><a name="The_PCAP_Topology"></a>The PCAP Topology</h2>
-<p>The structure of the topology is extremely simple. In fact, it is a spout-only topology. The <tt>Storm Kafka</tt> spout is used but extended to allow a callback to be used rather than having a separate bolt. </p>
+<p>The structure of the topology is extremely simple.  In fact, it is a spout-only topology.  The <tt>Storm Kafka</tt> spout is used but extended to allow a callback to be used rather than having a separate bolt.</p>
 <p>The following happens as part of this spout for each packet:</p>
-
 <ul>
-  
+
 <li>A custom <tt>Scheme</tt> is used which attaches the appropriate headers to the packet (both global and packet headers) using the timestamp in the key and the raw packet data in the value.</li>
-  
 <li>A callback is called which appends the packet data to a sequence file in HDFS.</li>
 </ul></div>
 <div class="section">
 <h2><a name="The_Files_on_HDFS"></a>The Files on HDFS</h2>
 <p>The sequence files on HDFS fit the following pattern: <tt>$BASE_PATH/pcap_$TOPIC_$TS_$PARTITION_$UUID</tt></p>
 <p>where</p>
-
 <ul>
-  
+
 <li><tt>BASE_PATH</tt> is the base path to where pcap data is stored in HDFS</li>
-  
 <li><tt>TOPIC</tt> is the kafka topic</li>
-  
 <li><tt>TS</tt> is the timestamp, in nanoseconds since the unix epoch</li>
-  
 <li><tt>PARTITION</tt> is the kafka partition</li>
-  
 <li><tt>UUID</tt> the UUID for the storm worker</li>
 </ul>
 <p>These files contain a set of packet data with headers on them in sequence files.</p></div>
 <div class="section">
 <h2><a name="Configuration"></a>Configuration</h2>
 <p>The configuration file for the Flux topology is located at <tt>$METRON_HOME/config/pcap.properties</tt> and the possible options are as follows:</p>
-
 <ul>
-  
+
 <li><tt>spout.kafka.topic.pcap</tt> : The kafka topic to listen to</li>
-  
-<li><tt>storm.auto.credentials</tt> : The kerberos ticket renewal. If running on a kerberized cluster, this should be <tt>['org.apache.storm.security.auth.kerberos.AutoTGT']</tt></li>
-  
-<li><tt>kafka.security.protocol</tt> : The security protocol to use for kafka. This should be <tt>PLAINTEXT</tt> for a non-kerberized cluster and probably <tt>SASL_PLAINTEXT</tt> for a kerberized cluster.</li>
-  
-<li><tt>kafka.zk</tt> : The comma separated zookeeper quorum (i.e. host:2181,host2:2181)</li>
-  
+<li><tt>storm.auto.credentials</tt> : The kerberos ticket renewal.  If running on a kerberized cluster, this should be <tt>['org.apache.storm.security.auth.kerberos.AutoTGT']</tt></li>
+<li><tt>kafka.security.protocol</tt> : The security protocol to use for kafka.  This should be <tt>PLAINTEXT</tt> for a non-kerberized cluster and probably <tt>SASL_PLAINTEXT</tt> for a kerberized cluster.</li>
+<li><tt>kafka.zk</tt> : The comma separated zookeeper quorum (i.e.  host:2181,host2:2181)</li>
 <li><tt>kafka.pcap.start</tt> : One of <tt>EARLIEST</tt>, <tt>LATEST</tt>, <tt>UNCOMMITTED_EARLIEST</tt>, <tt>UNCOMMITTED_LATEST</tt> representing where to start listening on the queue.</li>
-  
 <li><tt>kafka.pcap.numPackets</tt> : The number of packets to keep in one file.</li>
-  
-<li><tt>kafka.pcap.maxTimeMS</tt> : The number of packets to keep in one file in terms of duration (in milliseconds). For instance, you may only want to keep an hour&#x2019;s worth of packets in a given file.</li>
-  
-<li><tt>kafka.pcap.ts_scheme</tt> : One of <tt>FROM_KEY</tt> or <tt>FROM_VALUE</tt>. You really only want <tt>FROM_KEY</tt> as that fits the current tooling. <tt>FROM_VALUE</tt> assumes that fully headerized packets are coming in on the value, which is legacy.</li>
-  
+<li><tt>kafka.pcap.maxTimeMS</tt> : The number of packets to keep in one file in terms of duration (in milliseconds).  For instance, you may only want to keep an hour&#x2019;s worth of packets in a given file.</li>
+<li><tt>kafka.pcap.ts_scheme</tt> : One of <tt>FROM_KEY</tt> or <tt>FROM_VALUE</tt>.  You really only want <tt>FROM_KEY</tt> as that fits the current tooling.  <tt>FROM_VALUE</tt> assumes that fully headerized packets are coming in on the value, which is legacy.</li>
 <li><tt>kafka.pcap.out</tt> : The directory in HDFS to store the packet capture data</li>
-  
-<li><tt>kafka.pcap.ts_granularity</tt> : The granularity of timing used in the timestamps. One of <tt>MILLISECONDS</tt>, <tt>MICROSECONDS</tt>, or <tt>NANOSECONDS</tt> representing milliseconds, microseconds or nanoseconds since the unix epoch (respectively).</li>
+<li><tt>kafka.pcap.ts_granularity</tt> : The granularity of timing used in the timestamps.  One of <tt>MILLISECONDS</tt>, <tt>MICROSECONDS</tt>, or <tt>NANOSECONDS</tt> representing milliseconds, microseconds or nanoseconds since the unix epoch (respectively).</li>
 </ul></div>
 <div class="section">
 <h2><a name="Starting_the_Topology"></a>Starting the Topology</h2>
-<p>To assist in starting the topology, a utility script which takes no arguments has been created to make this very simple. Simply, execute <tt>$METRON_HOME/bin/start_pcap_topology.sh</tt>.</p></div>
+<p>To assist in starting the topology, a utility script which takes no arguments has been created to make this very simple.  Simply, execute <tt>$METRON_HOME/bin/start_pcap_topology.sh</tt>.</p></div>
 <div class="section">
 <h2><a name="Utilities"></a>Utilities</h2>
 <div class="section">
 <h3><a name="Inspector_Utility"></a>Inspector Utility</h3>
 <p>In order to ensure that data can be read back out, a utility, <tt>$METRON_HOME/bin/pcap_inspector.sh</tt> has been created to read portions of the sequence files.</p>
 
-<div class="source">
-<div class="source">
-<pre>usage: PcapInspector
+<div>
+<div>
+<pre class="source">usage: PcapInspector
  -h,--help               Generate Help screen
  -i,--input &lt;SEQ_FILE&gt;   Input sequence file on HDFS
  -n,--num_packets &lt;N&gt;    Number of packets to dump
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Query_Filter_Utility"></a>Query Filter Utility</h3>
 <p>This tool exposes the two methods for filtering PCAP data via a command line tool:</p>
-
 <ul>
-  
+
 <li>fixed</li>
-  
 <li>query (via Stellar)</li>
 </ul>
-<p>The tool is executed via </p>
+<p>The tool is executed via</p>
 
-<div class="source">
-<div class="source">
-<pre>${metron_home}/bin/pcap_query.sh [fixed|query]
+<div>
+<div>
+<pre class="source">${metron_home}/bin/pcap_query.sh [fixed|query]
 </pre></div></div>
+
 <div class="section">
 <h4><a name="Usage"></a>Usage</h4>
 
-<div class="source">
-<div class="source">
-<pre>usage: Fixed filter options
+<div>
+<div>
+<pre class="source">usage: Fixed filter options
  -bop,--base_output_path &lt;arg&gt;   Query result output path. Default is
                                  '/tmp'
  -bp,--base_path &lt;arg&gt;           Base PCAP data path. Default is
@@ -426,9 +235,9 @@
  -st,--start_time &lt;arg&gt;          (required) Packet start time range.
 </pre></div></div>
 
-<div class="source">
-<div class="source">
-<pre>usage: Query filter options
+<div>
+<div>
+<pre class="source">usage: Query filter options
  -bop,--base_output_path &lt;arg&gt;   Query result output path. Default is
                                  '/tmp'
  -bp,--base_path &lt;arg&gt;           Base PCAP data path. Default is
@@ -444,28 +253,22 @@
  -q,--query &lt;arg&gt;                Query string to use as a filter
  -st,--start_time &lt;arg&gt;          (required) Packet start time range.
 </pre></div></div>
-<p>The Query filter&#x2019;s <tt>--query</tt> argument specifies the Stellar expression to execute on each packet. To interact with the packet, a few variables are exposed:</p>
 
+<p>The Query filter&#x2019;s <tt>--query</tt> argument specifies the Stellar expression to execute on each packet.  To interact with the packet, a few variables are exposed:</p>
 <ul>
-  
+
 <li><tt>packet</tt> : The packet data (a <tt>byte[]</tt>)</li>
-  
 <li><tt>ip_src_addr</tt> : The source address for the packet (a <tt>String</tt>)</li>
-  
 <li><tt>ip_src_port</tt> : The source port for the packet (an <tt>Integer</tt>)</li>
-  
 <li><tt>ip_dst_addr</tt> : The destination address for the packet (a <tt>String</tt>)</li>
-  
 <li><tt>ip_dst_port</tt> : The destination port for the packet (an <tt>Integer</tt>)</li>
 </ul></div>
 <div class="section">
 <h4><a name="Binary_Regex"></a>Binary Regex</h4>
-<p>Filtering can be done both by the packet header as well as via a binary regular expression which can be run on the packet payload itself. This filter can be specified via:</p>
-
+<p>Filtering can be done both by the packet header as well as via a binary regular expression which can be run on the packet payload itself.  This filter can be specified via:</p>
 <ul>
-  
+
 <li>The <tt>-pf</tt> or <tt>--packet_filter</tt> options for the fixed query filter</li>
-  
 <li>The <tt>BYTEARRAY_MATCHER(pattern, data)</tt> Stellar function. The first argument is the regex pattern and the second argument is the data. The packet data will be exposed via the<tt>packet</tt> variable in Stellar.</li>
 </ul>
 <p>The format of this regular expression is described <a class="externalLink" href="https://github.com/nishihatapalmer/byteseek/blob/master/sequencesyntax.md">here</a>.</p></div></div></div>
@@ -475,9 +278,9 @@
 <div class="section">
 <h3><a name="pcap.properties_file"></a>pcap.properties file</h3>
 
-<div class="source">
-<div class="source">
-<pre>spout.kafka.topic.pcap=pcap
+<div>
+<div>
+<pre class="source">spout.kafka.topic.pcap=pcap
 storm.topology.workers=16
 kafka.spout.parallelism=128
 kafka.pcap.numPackets=1000000000
@@ -485,14 +288,15 @@ kafka.pcap.maxTimeMS=0
 hdfs.replication=1
 hdfs.sync.every=10000
 </pre></div></div>
+
 <p>You&#x2019;ll notice that the number of kakfa partitions equals the spout parallelism, and this is no coincidence. The ordering guarantees for a partition in Kafka enforces that you may have no more consumers than 1 per topic. Any additional parallelism will leave you with dormant threads consuming resources but performing no additional work. For our cluster with 4 Storm Supervisors, we found 16 workers to provide optimal throughput as well. We were largely IO bound rather than CPU bound with the incoming PCAP data.</p></div>
 <div class="section">
 <h3><a name="remote.yaml"></a>remote.yaml</h3>
 <p>In the flux file, we introduced the following configuration:</p>
 
-<div class="source">
-<div class="source">
-<pre>name: &quot;pcap&quot;
+<div>
+<div>
+<pre class="source">name: &quot;pcap&quot;
 config:
     topology.workers: ${storm.topology.workers}
     topology.worker.childopts: ${topology.worker.childopts}
@@ -615,6 +419,7 @@ spouts:
       - ref: &quot;writerConfig&quot;
 
 </pre></div></div>
+
 <div class="section">
 <h4><a name="Flux_Changes_Introduced"></a>Flux Changes Introduced</h4>
 <div class="section">
@@ -623,9 +428,9 @@ spouts:
 <div class="section">
 <h5><a name="Kafka_configuration"></a>Kafka configuration</h5>
 
-<div class="source">
-<div class="source">
-<pre>poll.timeout.ms
+<div>
+<div>
+<pre class="source">poll.timeout.ms
 offset.commit.period.ms
 session.timeout.ms
 max.uncommitted.offsets
@@ -633,44 +438,40 @@ max.poll.interval.ms
 max.poll.records
 receive.buffer.bytes
 max.partition.fetch.bytes
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h5><a name="Writer_Configuration"></a>Writer Configuration</h5>
 <p>This is a combination of settings for the HDFSWriter (see pcap.properties values above) as well as HDFS.</p>
 <p><b>HDFS config</b></p>
 <p>Component config HashMap with the following properties:</p>
 
-<div class="source">
-<div class="source">
-<pre>io.file.buffer.size
+<div>
+<div>
+<pre class="source">io.file.buffer.size
 dfs.blocksize
 </pre></div></div>
+
 <p><b>Writer config</b></p>
 <p>References the HDFS props component specified above.</p>
 
-<div class="source">
-<div class="source">
-<pre> -   name: &quot;withHDFSConfig&quot;
+<div>
+<div>
+<pre class="source"> -   name: &quot;withHDFSConfig&quot;
      args:
        - ref: &quot;hdfsProps&quot;
 </pre></div></div></div></div></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-writer/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-writer/index.html b/site/current-book/metron-platform/metron-writer/index.html
index b9c2125..cb1a26f 100644
--- a/site/current-book/metron-platform/metron-writer/index.html
+++ b/site/current-book/metron-platform/metron-writer/index.html
@@ -1,271 +1,93 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-writer/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Writer</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Writer</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Writer</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Writer</a>
-          </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <!-- Licensed to the Apache Software Foundation (ASF) under one or more
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
@@ -278,19 +100,21 @@ Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
-limitations under the License. --><h1>Writer</h1>
+limitations under the License.
+ -->
+<h1>Writer</h1>
 <p><a name="Writer"></a></p>
 <div class="section">
 <h2><a name="Introduction"></a>Introduction</h2>
-<p>The writer module provides some utilties for writing to outside components from within Storm. This includes managing bulk writing. An implemention is included for writing to HDFS in this module. Other writers can be found in their own modules.</p></div>
+<p>The writer module provides some utilties for writing to outside components from within Storm.  This includes managing bulk writing.  An implemention is included for writing to HDFS in this module. Other writers can be found in their own modules.</p></div>
 <div class="section">
 <h2><a name="HDFS_Writer"></a>HDFS Writer</h2>
-<p>The HDFS writer included here expands on what Storm has in several ways. There&#x2019;s customization in syncing to HDFS, rotation policy, etc. In addition, the writer allows for users to define output paths based on the fields in the provided JSON message. This can be defined using Stellar.</p>
+<p>The HDFS writer included here expands on what Storm has in several ways. There&#x2019;s customization in syncing to HDFS, rotation policy, etc. In addition, the writer allows for users to define output paths based on the fields in the provided JSON message.  This can be defined using Stellar.</p>
 <p>To manage the output path, a base path argument is provided by the Flux file, with the FileNameFormat as follows</p>
 
-<div class="source">
-<div class="source">
-<pre>    -   id: &quot;fileNameFormat&quot;
+<div>
+<div>
+<pre class="source">    -   id: &quot;fileNameFormat&quot;
         className: &quot;org.apache.storm.hdfs.bolt.format.DefaultFileNameFormat&quot;
         configMethods:
             -   name: &quot;withPrefix&quot;
@@ -303,33 +127,36 @@ limitations under the License. --><h1>Writer</h1>
                 args:
                     - &quot;/apps/metron/&quot;
 </pre></div></div>
-<p>This means that all output will land in <tt>/apps/metron/</tt>. With no further adjustment, it will be <tt>/apps/metron/&lt;sensor&gt;/</tt>. However, by modifying the sensor&#x2019;s JSON config, it is possible to provide additional pathing based on the the message itself.</p>
+
+<p>This means that all output will land in <tt>/apps/metron/</tt>.  With no further adjustment, it will be <tt>/apps/metron/&lt;sensor&gt;/</tt>. However, by modifying the sensor&#x2019;s JSON config, it is possible to provide additional pathing based on the the message itself.</p>
 <p>E.g.</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;index&quot;: &quot;bro&quot;,
   &quot;batchSize&quot;: 5,
   &quot;outputPathFunction&quot;: &quot;FORMAT('uid-%s', uid)&quot;
 }
 </pre></div></div>
+
 <p>will land data in <tt>/apps/metron/uid-&lt;uid&gt;/</tt>.</p>
 <p>For example, if the data contains uid&#x2019;s 1, 3, and 5, there will be 3 output folders in HDFS:</p>
 
-<div class="source">
-<div class="source">
-<pre>/apps/metron/uid-1/
+<div>
+<div>
+<pre class="source">/apps/metron/uid-1/
 /apps/metron/uid-3/
 /apps/metron/uid-5/
 </pre></div></div>
+
 <p>The Stellar function must return a String, but is not limited to FORMAT functions. Other functions, such as <tt>TO_LOWER</tt>, <tt>TO_UPPER</tt>, etc. are all available for use. Typically, it&#x2019;s preferable to do nontrivial transformations as part of enrichment and simply reference the output here.</p>
 <p>If no Stellar function is provided, it will default to putting the sensor in a folder, as above.</p>
-<p>A caveat is that the writer will only allow a certain number of files to be created at once. HdfsWriter has a function <tt>withMaxOpenFiles</tt> allowing this to be set. The default is 500. This can be set in Flux:</p>
+<p>A caveat is that the writer will only allow a certain number of files to be created at once.  HdfsWriter has a function <tt>withMaxOpenFiles</tt> allowing this to be set.  The default is 500.  This can be set in Flux:</p>
 
-<div class="source">
-<div class="source">
-<pre>    -   id: &quot;hdfsWriter&quot;
+<div>
+<div>
+<pre class="source">    -   id: &quot;hdfsWriter&quot;
         className: &quot;org.apache.metron.writer.hdfs.HdfsWriter&quot;
         configMethods:
             -   name: &quot;withFileNameFormat&quot;
@@ -341,23 +168,17 @@ limitations under the License. --><h1>Writer</h1>
             -   name: &quot;withMaxOpenFiles&quot;
                 args: 500
 </pre></div></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[47/50] [abbrv] metron git commit: METRON-1601: Rename metaalert alert nested field to metron_alert to avoid collision closes apache/incubator-metron#1049

Posted by mm...@apache.org.
METRON-1601: Rename metaalert alert nested field to metron_alert to avoid collision closes apache/incubator-metron#1049


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/edec7b18
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/edec7b18
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/edec7b18

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: edec7b18bc40e40f35916489ba2fde2ca4916dc5
Parents: 40796c0
Author: cstella <ce...@gmail.com>
Authored: Fri Jun 8 09:49:54 2018 -0400
Committer: cstella <ce...@gmail.com>
Committed: Fri Jun 8 09:49:54 2018 -0400

----------------------------------------------------------------------
 .../METRON/CURRENT/package/files/bro_index.template       |  2 +-
 .../METRON/CURRENT/package/files/error_index.template     |  2 +-
 .../METRON/CURRENT/package/files/metaalert_index.template |  2 +-
 .../METRON/CURRENT/package/files/snort_index.template     |  2 +-
 .../METRON/CURRENT/package/files/yaf_index.template       |  2 +-
 .../e2e/mock-data/alerts_ui_e2e_index.template            |  2 +-
 .../app/alerts/alert-details/alert-details.component.ts   |  2 +-
 .../src/app/alerts/alerts-list/alerts-list.component.ts   |  2 +-
 .../alerts-list/table-view/table-view.component.html      | 10 +++++-----
 .../alerts/alerts-list/table-view/table-view.component.ts |  2 +-
 .../metron-alerts/src/app/model/alert-source.ts           |  2 +-
 .../controller/MetaAlertControllerIntegrationTest.java    |  4 ++--
 metron-platform/metron-elasticsearch/README.md            |  8 ++++----
 .../ElasticsearchMetaAlertIntegrationTest.java            | 10 +++++-----
 .../java/org/apache/metron/indexing/dao/MetaAlertDao.java |  2 +-
 .../apache/metron/indexing/dao/InMemoryMetaAlertDao.java  |  2 +-
 16 files changed, 28 insertions(+), 28 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template
----------------------------------------------------------------------
diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template
index 30f2591..17ad4d2 100644
--- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template
+++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template
@@ -143,7 +143,7 @@
         "uid": {
           "type": "keyword"
         },
-        "alert": {
+        "metron_alert": {
           "type": "nested"
         },
         "ip_src_addr": {

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/error_index.template
----------------------------------------------------------------------
diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/error_index.template b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/error_index.template
index 00aaf87..d119509 100644
--- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/error_index.template
+++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/error_index.template
@@ -38,7 +38,7 @@
         "error_type": {
           "type": "keyword"
         },
-        "alert": {
+        "metron_alert": {
           "type": "nested"
         }
       }

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/metaalert_index.template
----------------------------------------------------------------------
diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/metaalert_index.template b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/metaalert_index.template
index 24e8357..05d5e32 100644
--- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/metaalert_index.template
+++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/metaalert_index.template
@@ -36,7 +36,7 @@
           "type": "date",
           "format": "epoch_millis"
         },
-        "alert": {
+        "metron_alert": {
           "type": "nested"
         },
         "source:type": {

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/snort_index.template
----------------------------------------------------------------------
diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/snort_index.template b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/snort_index.template
index 43b3ca4..f7c6e59 100644
--- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/snort_index.template
+++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/snort_index.template
@@ -198,7 +198,7 @@
         "guid": {
           "type": "keyword"
         },
-        "alert": {
+        "metron_alert": {
           "type": "nested"
         }
       }

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/yaf_index.template
----------------------------------------------------------------------
diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/yaf_index.template b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/yaf_index.template
index b6965f9..f4093ba 100644
--- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/yaf_index.template
+++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/yaf_index.template
@@ -208,7 +208,7 @@
           "type": "text",
           "fielddata": "true"
         },
-        "alert": {
+        "metron_alert": {
           "type": "nested"
         },
         "guid": {

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-interface/metron-alerts/e2e/mock-data/alerts_ui_e2e_index.template
----------------------------------------------------------------------
diff --git a/metron-interface/metron-alerts/e2e/mock-data/alerts_ui_e2e_index.template b/metron-interface/metron-alerts/e2e/mock-data/alerts_ui_e2e_index.template
index caf593c..0d6b420 100644
--- a/metron-interface/metron-alerts/e2e/mock-data/alerts_ui_e2e_index.template
+++ b/metron-interface/metron-alerts/e2e/mock-data/alerts_ui_e2e_index.template
@@ -143,7 +143,7 @@
         "uid": {
           "type": "keyword"
         },
-        "alert": {
+        "metron_alert": {
           "type": "nested"
         },
         "ip_src_addr": {

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
----------------------------------------------------------------------
diff --git a/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts b/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
index e1c1685..c8d0d7a 100644
--- a/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
+++ b/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
@@ -92,7 +92,7 @@ export class AlertDetailsComponent implements OnInit {
     this.searchService.getAlert(this.alertSourceType, this.alertId).subscribe(alertSource => {
       this.alertSource = alertSource;
       this.selectedAlertState = this.getAlertState(alertSource['alert_status']);
-      this.alertSources = (alertSource.alert && alertSource.alert.length > 0) ? alertSource.alert : [alertSource];
+      this.alertSources = (alertSource.metron_alert && alertSource.metron_alert.length > 0) ? alertSource.metron_alert : [alertSource];
       this.setComments(alertSource);
 
       if (fireToggleEditor) {

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-interface/metron-alerts/src/app/alerts/alerts-list/alerts-list.component.ts
----------------------------------------------------------------------
diff --git a/metron-interface/metron-alerts/src/app/alerts/alerts-list/alerts-list.component.ts b/metron-interface/metron-alerts/src/app/alerts/alerts-list/alerts-list.component.ts
index 776a083..a70f2b4 100644
--- a/metron-interface/metron-alerts/src/app/alerts/alerts-list/alerts-list.component.ts
+++ b/metron-interface/metron-alerts/src/app/alerts/alerts-list/alerts-list.component.ts
@@ -219,7 +219,7 @@ export class AlertsListComponent implements OnInit, OnDestroy {
 
   onSelectedAlertsChange(selectedAlerts) {
     this.selectedAlerts = selectedAlerts;
-    this.isMetaAlertPresentInSelectedAlerts = this.selectedAlerts.some(alert => (alert.source.alert && alert.source.alert.length > 0));
+    this.isMetaAlertPresentInSelectedAlerts = this.selectedAlerts.some(alert => (alert.source.metron_alert && alert.source.metron_alert.length > 0));
 
     if (selectedAlerts.length > 0) {
       this.pause();

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.html
----------------------------------------------------------------------
diff --git a/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.html b/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.html
index d2b1108..78410af 100644
--- a/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.html
+++ b/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.html
@@ -26,7 +26,7 @@
     <tbody>
     <ng-container *ngFor="let alert of alerts; let alertIndex = index;">
 
-      <ng-container *ngIf="!alert.source.alert || alert.source.alert.length === 0">
+      <ng-container *ngIf="!alert.source.metron_alert || alert.source.metron_alert.length === 0">
         <tr (click)="showDetails($event, alert)" [ngClass]="{'selected' : selectedAlerts.indexOf(alert) != -1}">
           <td width="15" class="icon-cell"></td>
           <td (click)="addFilter(threatScoreFieldName, alert.source[threatScoreFieldName])">
@@ -50,7 +50,7 @@
         </tr>
       </ng-container>
 
-      <ng-container *ngIf="alert.source.alert && alert.source.alert.length > 0">
+      <ng-container *ngIf="alert.source.metron_alert && alert.source.metron_alert.length > 0">
         <tr (click)="showDetails($event, alert)" [ngClass]="{'selected' : selectedAlerts.indexOf(alert) != -1}">
           <td width="15" class="icon-cell dropdown-cell" (click)="toggleExpandCollapse($event, alert)">
             <i class="fa" aria-hidden="true"
@@ -62,7 +62,7 @@
           </td>
           <td [attr.colspan]="alertsColumnsToDisplay.length - 1">
             <a (click)="addFilter('guid', alert.id)" [attr.title]="alert.id" style="color:#689AA9"> {{ alert.source['name'] ? alert.source['name'] : alert.id | centerEllipses:20:cell }}</a>
-              <span> ({{ alert.source.alert.length }})</span>
+              <span> ({{ alert.source.metron_alert.length }})</span>
           </td>
           <td>
             <a *ngIf="isStatusFieldPresent" (click)="addFilter('alert_status', alert.source['alert_status'])" style="color:#689AA9">
@@ -80,7 +80,7 @@
             <label attr.for="{{ alert.id }}"></label>
           </td>
         </tr>
-        <tr *ngFor="let metaAlerts of alert.source.alert; let metaAlertIndex = index;" (click)="showMetaAlertDetails($event, metaAlerts)"
+        <tr *ngFor="let metaAlerts of alert.source.metron_alert; let metaAlertIndex = index;" (click)="showMetaAlertDetails($event, metaAlerts)"
             [ngClass]="{'selected' : selectedAlerts.indexOf(metaAlerts) != -1 , 'd-none': metaAlertsDisplayState[alert.id] === metronAlertDisplayState.COLLAPSE}">
           <td width="15" class="icon-cell" class="dropdown-cell"></td>
           <td (click)="addFilter(threatScoreFieldName, alert.source[threatScoreFieldName])" style="padding-left: 15px">
@@ -114,4 +114,4 @@
   <div class="col-md-3 push-md-5">
     <metron-table-pagination [(pagination)]="pagination" (pageChange)="onPageChange()"> </metron-table-pagination>
   </div>
-</div>
\ No newline at end of file
+</div>

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.ts
----------------------------------------------------------------------
diff --git a/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.ts b/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.ts
index 0176ff0..411baab 100644
--- a/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.ts
+++ b/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.ts
@@ -122,7 +122,7 @@ export class TableViewComponent implements OnInit, OnChanges, OnDestroy {
 
   updateExpandedStateForChangedData(expandedMetaAlerts: string[]) {
     this.alerts.forEach(alert => {
-      if (alert.source.alert && alert.source.alert.length > 0) {
+      if (alert.source.metron_alert && alert.source.metron_alert.length > 0) {
         this.metaAlertsDisplayState[alert.id] = expandedMetaAlerts.indexOf(alert.id) === -1 ?
                                                   MetronAlertDisplayState.COLLAPSE : MetronAlertDisplayState.EXPAND;
       }

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-interface/metron-alerts/src/app/model/alert-source.ts
----------------------------------------------------------------------
diff --git a/metron-interface/metron-alerts/src/app/model/alert-source.ts b/metron-interface/metron-alerts/src/app/model/alert-source.ts
index d5477dc..3330960 100644
--- a/metron-interface/metron-alerts/src/app/model/alert-source.ts
+++ b/metron-interface/metron-alerts/src/app/model/alert-source.ts
@@ -44,7 +44,7 @@ export class AlertSource {
   guid: string;
   sig_id: number;
   sig_generator: number;
-  alert: AlertSource[] = [];
+  metron_alert: AlertSource[] = [];
   comments: AlertComment[] = [];
   'threat:triage:score': number;
   'threatinteljoinbolt:joiner:ts': number;

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/MetaAlertControllerIntegrationTest.java
----------------------------------------------------------------------
diff --git a/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/MetaAlertControllerIntegrationTest.java b/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/MetaAlertControllerIntegrationTest.java
index 3e69e37..f86f227 100644
--- a/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/MetaAlertControllerIntegrationTest.java
+++ b/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/MetaAlertControllerIntegrationTest.java
@@ -98,8 +98,8 @@ public class MetaAlertControllerIntegrationTest extends DaoControllerTest {
 
   /**
    * [
-   *{"guid":"meta_1","alert":[{"guid":"bro_1"}],"average":"5.0","min":"5.0","median":"5.0","max":"5.0","count":"1.0","sum":"5.0"},
-   *{"guid":"meta_2","alert":[{"guid":"bro_1"},{"guid":"bro_2"},{"guid":"snort_1"}],"average":"5.0","min":"0.0","median":"5.0","max":"10.0","count":"3.0","sum":"15.0"}
+   *{"guid":"meta_1","metron_alert":[{"guid":"bro_1"}],"average":"5.0","min":"5.0","median":"5.0","max":"5.0","count":"1.0","sum":"5.0"},
+   *{"guid":"meta_2","metron_alert":[{"guid":"bro_1"},{"guid":"bro_2"},{"guid":"snort_1"}],"average":"5.0","min":"0.0","median":"5.0","max":"10.0","count":"3.0","sum":"15.0"}
    * ]
    */
   @Multiline

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-platform/metron-elasticsearch/README.md
----------------------------------------------------------------------
diff --git a/metron-platform/metron-elasticsearch/README.md b/metron-platform/metron-elasticsearch/README.md
index 1e15018..d889e27 100644
--- a/metron-platform/metron-elasticsearch/README.md
+++ b/metron-platform/metron-elasticsearch/README.md
@@ -271,13 +271,13 @@ Notes on other settings for types in ES
 
 ## Using Metron with Elasticsearch 5.6.2
 
-There is a requirement that all sensors templates have a nested alert field defined.  This field is a dummy field.  See [Ignoring Unmapped Fields](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_ignoring_unmapped_fields) for more information
+There is a requirement that all sensors templates have a nested `metron_alert` field defined.  This field is a dummy field.  See [Ignoring Unmapped Fields](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_ignoring_unmapped_fields) for more information
 
 Without this field, an error will be thrown during ALL searches (including from UIs, resulting in no alerts being found for any sensor). This error will be found in the REST service's logs.
 
 Exception seen:
 ```
-QueryParsingException[[nested] failed to find nested object under path [alert]];
+QueryParsingException[[nested] failed to find nested object under path [metron_alert]];
 ```
 
 There are two steps to resolve this issue.  First is to update the Elasticsearch template for each sensor, so any new indices have the field. This requires retrieving the template, removing an extraneous JSON field so we can put it back later, and adding our new field.
@@ -290,7 +290,7 @@ export SENSOR="bro"
 curl -XGET "http://${ELASTICSEARCH}:9200/_template/${SENSOR}_index*?pretty=true" -o "${SENSOR}.template"
 sed -i '' '2d;$d' ./${SENSOR}.template
 sed -i '' '/"properties" : {/ a\
-"alert": { "type": "nested"},' ${SENSOR}.template
+"metron_alert": { "type": "nested"},' ${SENSOR}.template
 ```
 
 To manually verify this, you can optionally pretty print it again with:
@@ -309,7 +309,7 @@ To update existing indexes, update Elasticsearch mappings with the new field for
 curl -XPUT "http://${ELASTICSEARCH}:9200/${SENSOR}_index*/_mapping/${SENSOR}_doc" -d '
 {
   "properties" : {
-    "alert" : {
+    "metron_alert" : {
       "type" : "nested"
     }
   }

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java b/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
index 5222a38..b001050 100644
--- a/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
+++ b/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
@@ -92,7 +92,7 @@ public class ElasticsearchMetaAlertIntegrationTest {
   /**
    {
      "properties": {
-       "alert": {
+       "metron_alert": {
          "type": "nested"
        }
      }
@@ -175,7 +175,7 @@ public class ElasticsearchMetaAlertIntegrationTest {
            "score" : {
              "type" : "integer"
            },
-           "alert" : {
+           "metron_alert" : {
              "type" : "nested"
            }
          }
@@ -761,7 +761,7 @@ public class ElasticsearchMetaAlertIntegrationTest {
     SearchResponse searchResponse = metaDao.search(new SearchRequest() {
       {
         setQuery(
-            "(ip_src_addr:192.168.1.1 AND ip_src_port:8009) OR (alert.ip_src_addr:192.168.1.1 AND alert.ip_src_port:8009)");
+            "(ip_src_addr:192.168.1.1 AND ip_src_port:8009) OR (metron_alert.ip_src_addr:192.168.1.1 AND metron_alert.ip_src_port:8009)");
         setIndices(Collections.singletonList(MetaAlertDao.METAALERT_TYPE));
         setFrom(0);
         setSize(5);
@@ -781,7 +781,7 @@ public class ElasticsearchMetaAlertIntegrationTest {
       {
         setQuery(
             "(ip_src_addr:192.168.1.1 AND ip_src_port:8010)"
-                + " OR (alert.ip_src_addr:192.168.1.1 AND alert.ip_src_port:8010)");
+                + " OR (metron_alert.ip_src_addr:192.168.1.1 AND metron_alert.ip_src_port:8010)");
         setIndices(Collections.singletonList("*"));
         setFrom(0);
         setSize(5);
@@ -804,7 +804,7 @@ public class ElasticsearchMetaAlertIntegrationTest {
       {
         setQuery(
             "(ip_src_addr:192.168.1.3 AND ip_src_port:8008)"
-                + " OR (alert.ip_src_addr:192.168.1.3 AND alert.ip_src_port:8008)");
+                + " OR (metron_alert.ip_src_addr:192.168.1.3 AND metron_alert.ip_src_port:8008)");
         setIndices(Collections.singletonList("*"));
         setFrom(0);
         setSize(1);

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/MetaAlertDao.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/MetaAlertDao.java b/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/MetaAlertDao.java
index 4530d2a..8807849 100644
--- a/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/MetaAlertDao.java
+++ b/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/MetaAlertDao.java
@@ -70,7 +70,7 @@ public interface MetaAlertDao extends IndexDao {
   String METAALERT_DOC = METAALERT_TYPE + "_doc";
   String THREAT_FIELD_DEFAULT = "threat:triage:score";
   String THREAT_SORT_DEFAULT = "sum";
-  String ALERT_FIELD = "alert";
+  String ALERT_FIELD = "metron_alert";
   String STATUS_FIELD = "status";
   String GROUPS_FIELD = "groups";
 

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/dao/InMemoryMetaAlertDao.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/dao/InMemoryMetaAlertDao.java b/metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/dao/InMemoryMetaAlertDao.java
index baa5416..5ab5c48 100644
--- a/metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/dao/InMemoryMetaAlertDao.java
+++ b/metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/dao/InMemoryMetaAlertDao.java
@@ -61,7 +61,7 @@ public class InMemoryMetaAlertDao implements MetaAlertDao {
   /**
    * {
    * "indices": ["metaalert"],
-   * "query": "alert|guid:${GUID}",
+   * "query": "metron_alert|guid:${GUID}",
    * "from": 0,
    * "size": 10,
    * "sort": [


[50/50] [abbrv] metron git commit: METRON-1612 Fix website download links (justinleet) closes apache/metron#1058

Posted by mm...@apache.org.
METRON-1612 Fix website download links (justinleet) closes apache/metron#1058


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/77570464
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/77570464
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/77570464

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: 77570464b4db205c3a3b95fcc3a0e8f7f6a1c540
Parents: 32d132b
Author: justinleet <ju...@gmail.com>
Authored: Mon Jun 11 10:35:09 2018 -0400
Committer: leet <le...@apache.org>
Committed: Mon Jun 11 10:35:09 2018 -0400

----------------------------------------------------------------------
 site/documentation/index.md | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/77570464/site/documentation/index.md
----------------------------------------------------------------------
diff --git a/site/documentation/index.md b/site/documentation/index.md
index 585f656..f842a67 100644
--- a/site/documentation/index.md
+++ b/site/documentation/index.md
@@ -82,18 +82,18 @@ title: Apache Metron Documentation
                 <a href="http://www.apache.org/dyn/closer.cgi/metron/0.5.0/apache-metron-0.5.0.tar.gz" target="new">
                     apache-metron-0.5.0.tar.gz
                 </a>
-                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.asc" target="new">
+                 [ <a href="https://www.apache.org/dist/metron/0.5.0/apache-metron-0.5.0.tar.gz.asc" target="new">
                     PGP
                  </a>]
-                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha1" target="new">
-                    SHA
+                 [ <a href="https://www.apache.org/dist/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha1" target="new">
+                    SHA1
                  </a>]
-                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha512" target="new">
+                 [ <a href="https://www.apache.org/dist/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha256" target="new">
+                    SHA256
+                 </a>]
+                 [ <a href="https://www.apache.org/dist/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha512" target="new">
                     SHA512
                  </a>]
-                 [ <a href="https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.md5" target="new">
-                    MD5
-                  </a>]
                  [ <a href="/current-book/index.html" target="new"> Book </a>]
                 </li>
             </ul>


[03/50] [abbrv] metron git commit: METRON-1584 Indexing Topology Crashes with Invalid Message (nickwallen) closes apache/metron#1036

Posted by mm...@apache.org.
METRON-1584 Indexing Topology Crashes with Invalid Message (nickwallen) closes apache/metron#1036


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/b6808f76
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/b6808f76
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/b6808f76

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: b6808f76134260e8a3c24c7641d6e4a235ef48e1
Parents: 45e3ed8
Author: nickwallen <ni...@nickallen.org>
Authored: Wed Jun 6 11:07:30 2018 -0400
Committer: nickallen <ni...@apache.org>
Committed: Wed Jun 6 11:07:30 2018 -0400

----------------------------------------------------------------------
 .../bolt/BulkMessageWriterBoltTest.java         | 52 +++++++++--
 .../metron/writer/BulkWriterComponent.java      | 36 +++++++-
 .../writer/bolt/BulkMessageWriterBolt.java      | 92 +++++++++++++++-----
 3 files changed, 145 insertions(+), 35 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/b6808f76/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/bolt/BulkMessageWriterBoltTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/bolt/BulkMessageWriterBoltTest.java b/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/bolt/BulkMessageWriterBoltTest.java
index dedf5e6..52516ac 100644
--- a/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/bolt/BulkMessageWriterBoltTest.java
+++ b/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/bolt/BulkMessageWriterBoltTest.java
@@ -118,28 +118,32 @@ public class BulkMessageWriterBoltTest extends BaseEnrichmentBoltTest {
   private MessageGetStrategy messageGetStrategy;
 
   @Test
-  public void testSensorTypeMissing() throws Exception {
+  public void testSourceTypeMissing() throws Exception {
+
+    // setup the bolt
     BulkMessageWriterBolt bulkMessageWriterBolt = new BulkMessageWriterBolt("zookeeperUrl")
-            .withBulkMessageWriter(bulkMessageWriter).withMessageGetter(MessageGetters.JSON_FROM_FIELD.name())
+            .withBulkMessageWriter(bulkMessageWriter)
+            .withMessageGetter(MessageGetters.JSON_FROM_FIELD.name())
             .withMessageGetterField("message");
     bulkMessageWriterBolt.setCuratorFramework(client);
     bulkMessageWriterBolt.setZKCache(cache);
     bulkMessageWriterBolt.getConfigurations().updateSensorIndexingConfig(sensorType,
             new FileInputStream(sampleSensorIndexingConfigPath));
 
+    // initialize the bolt
     bulkMessageWriterBolt.declareOutputFields(declarer);
-    verify(declarer, times(1)).declareStream(eq("error"), argThat(
-            new FieldsMatcher("message")));
     Map stormConf = new HashMap();
     bulkMessageWriterBolt.prepare(stormConf, topologyContext, outputCollector);
-    BulkWriterComponent<JSONObject> component = mock(BulkWriterComponent.class);
-    bulkMessageWriterBolt.setWriterComponent(component);
-    verify(bulkMessageWriter, times(1)).init(eq(stormConf),any(TopologyContext.class), any(WriterConfiguration.class));
+
+    // create a message with no source type
     JSONObject message = (JSONObject) new JSONParser().parse(sampleMessageString);
     message.remove("source.type");
     when(tuple.getValueByField("message")).thenReturn(message);
+
+    // the tuple should be handled as an error and ack'd
     bulkMessageWriterBolt.execute(tuple);
-    verify(component, times(1)).error(eq("null"), any(), any(), any());
+    verify(outputCollector, times(1)).emit(eq(Constants.ERROR_STREAM), any());
+    verify(outputCollector, times(1)).ack(tuple);
   }
 
   @Test
@@ -295,4 +299,36 @@ public class BulkMessageWriterBoltTest extends BaseEnrichmentBoltTest {
     assertEquals(3, tupleList.size());
     verify(outputCollector, times(5)).ack(tuple);  // 3 messages + 2nd tick
   }
+
+  /**
+   * If an invalid message is sent to indexing, the message should be handled as an error
+   * and the topology should continue processing.
+   */
+  @Test
+  public void testMessageInvalid() throws Exception {
+    FakeClock clock = new FakeClock();
+
+    // setup the bolt
+    BulkMessageWriterBolt bolt = new BulkMessageWriterBolt("zookeeperUrl")
+            .withBulkMessageWriter(bulkMessageWriter)
+            .withMessageGetter(MessageGetters.JSON_FROM_POSITION.name())
+            .withMessageGetterField("message");
+    bolt.setCuratorFramework(client);
+    bolt.setZKCache(cache);
+    bolt.getConfigurations().updateSensorIndexingConfig(sensorType, new FileInputStream(sampleSensorIndexingConfigPath));
+
+    // initialize the bolt
+    bolt.declareOutputFields(declarer);
+    Map stormConf = new HashMap();
+    bolt.prepare(stormConf, topologyContext, outputCollector, clock);
+
+    // execute a tuple that contains an invalid message
+    byte[] invalidJSON = "this is not valid JSON".getBytes();
+    when(tuple.getBinary(0)).thenReturn(invalidJSON);
+    bolt.execute(tuple);
+
+    // the tuple should be handled as an error and ack'd
+    verify(outputCollector, times(1)).emit(eq(Constants.ERROR_STREAM), any());
+    verify(outputCollector, times(1)).ack(tuple);
+  }
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/b6808f76/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/BulkWriterComponent.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/BulkWriterComponent.java b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/BulkWriterComponent.java
index 37c624f..dfa265d 100644
--- a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/BulkWriterComponent.java
+++ b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/BulkWriterComponent.java
@@ -19,6 +19,7 @@
 package org.apache.metron.writer;
 
 import com.google.common.collect.Iterables;
+import org.apache.commons.lang.exception.ExceptionUtils;
 import org.apache.metron.common.Constants;
 import org.apache.metron.common.configuration.writer.WriterConfiguration;
 import org.apache.metron.common.error.MetronError;
@@ -41,6 +42,8 @@ import java.util.Map;
 import java.util.Optional;
 import java.util.concurrent.TimeUnit;
 
+import static java.lang.String.format;
+
 /**
  * This component implements message batching, with both flush on queue size, and flush on queue timeout.
  * There is a queue for each sensorType.
@@ -115,15 +118,40 @@ public class BulkWriterComponent<MESSAGE_T> {
   }
 
   public void error(String sensorType, Throwable e, Iterable<Tuple> tuples, MessageGetStrategy messageGetStrategy) {
-    tuples.forEach(t -> collector.ack(t));
+    LOG.error(format("Failing %d tuple(s); sensorType=%s", Iterables.size(tuples), sensorType), e);
     MetronError error = new MetronError()
             .withSensorType(sensorType)
             .withErrorType(Constants.ErrorType.INDEXING_ERROR)
             .withThrowable(e);
-    if(!Iterables.isEmpty(tuples)) {
-      LOG.error("Failing {} tuples", Iterables.size(tuples), e);
-    }
     tuples.forEach(t -> error.addRawMessage(messageGetStrategy.get(t)));
+    handleError(tuples, error);
+  }
+
+  /**
+   * Error a set of tuples that may not contain a valid message.
+   *
+   * <p>Without a valid message, the source type is unknown.
+   * <p>Without a valid message, the JSON message cannot be added to the error.
+   *
+   * @param e The exception that occurred.
+   * @param tuples The tuples to error that may not contain valid messages.
+   */
+  public void error(Throwable e, Iterable<Tuple> tuples) {
+    LOG.error(format("Failing %d tuple(s)", Iterables.size(tuples)), e);
+    MetronError error = new MetronError()
+            .withErrorType(Constants.ErrorType.INDEXING_ERROR)
+            .withThrowable(e);
+    handleError(tuples, error);
+  }
+
+  /**
+   * Errors a set of tuples.
+   *
+   * @param tuples The tuples to error.
+   * @param error
+   */
+  private void handleError(Iterable<Tuple> tuples, MetronError error) {
+    tuples.forEach(t -> collector.ack(t));
     ErrorUtils.handleError(collector, error);
   }
 

http://git-wip-us.apache.org/repos/asf/metron/blob/b6808f76/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
index b5b97d8..1d8f0c6 100644
--- a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
+++ b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
@@ -234,41 +234,87 @@ public class BulkMessageWriterBolt extends ConfiguredIndexingBolt {
 
     try
     {
-      JSONObject message = (JSONObject) messageGetStrategy.get(tuple);
+      JSONObject message = getMessage(tuple);
+      if(message == null) {
+        handleMissingMessage(tuple);
+        return;
+      }
+
       String sensorType = MessageUtils.getSensorType(message);
+      if(sensorType == null) {
+        handleMissingSensorType(tuple, message);
+        return;
+      }
+
       LOG.trace("Writing enrichment message: {}", message);
       WriterConfiguration writerConfiguration = configurationTransformation.apply(
               new IndexingWriterConfiguration(bulkMessageWriter.getName(), getConfigurations()));
-      if(sensorType == null) {
-        //sensor type somehow ended up being null.  We want to error this message directly.
-        getWriterComponent().error("null"
-                             , new Exception("Sensor type is not specified for message "
-                                            + message.toJSONString()
-                                            )
-                             , ImmutableList.of(tuple)
-                             , messageGetStrategy
-                             );
-      }
-      else {
-        if (writerConfiguration.isDefault(sensorType)) {
-          //want to warn, but not fail the tuple
-          collector.reportError(new Exception("WARNING: Default and (likely) unoptimized writer config used for " + bulkMessageWriter.getName() + " writer and sensor " + sensorType));
-        }
 
-        getWriterComponent().write(sensorType
-                , tuple
-                , message
-                , bulkMessageWriter
-                , writerConfiguration
-                , messageGetStrategy
-        );
+      if (writerConfiguration.isDefault(sensorType)) {
+        //want to warn, but not fail the tuple
+        collector.reportError(new Exception("WARNING: Default and (likely) unoptimized writer config used for " + bulkMessageWriter.getName() + " writer and sensor " + sensorType));
       }
+
+      getWriterComponent().write(sensorType
+              , tuple
+              , message
+              , bulkMessageWriter
+              , writerConfiguration
+              , messageGetStrategy
+      );
     }
     catch(Exception e) {
       throw new RuntimeException("This should have been caught in the writerComponent.  If you see this, file a JIRA", e);
     }
   }
 
+  /**
+   * Retrieves the JSON message contained in a tuple.
+   *
+   * @param tuple The tuple containing a JSON message.
+   * @return The JSON message contained in the tuple. If none, returns null.
+   */
+  private JSONObject getMessage(Tuple tuple) {
+    JSONObject message = null;
+    try {
+      message = (JSONObject) messageGetStrategy.get(tuple);
+
+    } catch(Throwable e) {
+      LOG.error("Unable to retrieve message from tuple", e);
+    }
+
+    return message;
+  }
+
+  /**
+   * Handles error processing when a message is missing a sensor type.
+   *
+   * @param tuple The tuple.
+   * @param message The message with no sensor type.
+   */
+  private void handleMissingSensorType(Tuple tuple, JSONObject message) {
+    // sensor type somehow ended up being null.  We want to error this message directly.
+    LOG.debug("Message is missing sensor type");
+    getWriterComponent().error("null",
+            new Exception("Sensor type is not specified for message " + message.toJSONString()),
+            ImmutableList.of(tuple),
+            messageGetStrategy
+    );
+  }
+
+  /**
+   * Handles error processing when a tuple does not contain a valid message.
+   *
+   * @param tuple The tuple.
+   */
+  private void handleMissingMessage(Tuple tuple) {
+    LOG.debug("Unable to extract message from tuple; expected valid JSON");
+    getWriterComponent().error(
+            new Exception("Unable to extract message from tuple; expected valid JSON"),
+            ImmutableList.of(tuple)
+    );
+  }
+
   @Override
   public void declareOutputFields(OutputFieldsDeclarer declarer) {
     declarer.declareStream(Constants.ERROR_STREAM, new Fields("message"));


[18/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-elasticsearch/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-elasticsearch/index.html b/site/current-book/metron-platform/metron-elasticsearch/index.html
index ff4bfb0..cc360b1 100644
--- a/site/current-book/metron-platform/metron-elasticsearch/index.html
+++ b/site/current-book/metron-platform/metron-elasticsearch/index.html
@@ -1,380 +1,484 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-elasticsearch/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Elasticsearch in Metron</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Elasticsearch in Metron</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Elasticsearch in Metron</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Elasticsearch</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Elasticsearch in Metron</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Elasticsearch in Metron</h1>
 <p><a name="Elasticsearch_in_Metron"></a></p>
 <div class="section">
+<h2><a name="Table_of_Contents"></a>Table of Contents</h2>
+<ul>
+
+<li><a href="#Introduction">Introduction</a></li>
+<li><a href="#Properties">Properties</a></li>
+<li><a href="#Upgrading_to_5.6.2">Upgrading to 5.6.2</a></li>
+<li><a href="#Type_Mappings">Type Mappings</a></li>
+<li><a href="#Using_Metron_with_Elasticsearch_5.6.2">Using Metron with Elasticsearch 5.6.2</a></li>
+<li><a href="#Installing_Elasticsearch_Templates">Installing Elasticsearch Templates</a></li>
+</ul></div>
+<div class="section">
 <h2><a name="Introduction"></a>Introduction</h2>
 <p>Elasticsearch can be used as the real-time portion of the datastore resulting from <a href="../metron-indexing/index.html">metron-indexing</a>.</p></div>
 <div class="section">
 <h2><a name="Properties"></a>Properties</h2>
 <div class="section">
 <h3><a name="es.clustername"></a><tt>es.clustername</tt></h3>
-<p>The name of the elasticsearch Cluster. See <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#cluster.name">here</a></p></div>
+<p>The name of the elasticsearch Cluster.  See <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#cluster.name">here</a></p></div>
 <div class="section">
 <h3><a name="es.ip"></a><tt>es.ip</tt></h3>
 <p>Specifies the nodes in the elasticsearch cluster to use for writing. The format is one of the following:</p>
-
 <ul>
-  
+
 <li>A hostname or IP address with a port (e.g. <tt>hostname1:1234</tt>), in which case <tt>es.port</tt> is ignored.</li>
-  
 <li>A hostname or IP address without a port (e.g. <tt>hostname1</tt>), in which case <tt>es.port</tt> is used.</li>
-  
-<li>A string containing a CSV of hostnames without ports (e.g. <tt>hostname1,hostname2,hostname3</tt>) without spaces between. <tt>es.port</tt> is assumed to be the port for each host.</li>
-  
-<li>A string containing a CSV of hostnames with ports (e.g. <tt>hostname1:1234,hostname2:1234,hostname3:1234</tt>) without spaces between. <tt>es.port</tt> is ignored.</li>
-  
-<li>A list of hostnames with ports (e.g. <tt>[ &quot;hostname1:1234&quot;, &quot;hostname2:1234&quot;]</tt>). Note, <tt>es.port</tt> is NOT used in this construction.</li>
+<li>A string containing a CSV of hostnames without ports (e.g. <tt>hostname1,hostname2,hostname3</tt>) without spaces between.  <tt>es.port</tt> is assumed to be the port for each host.</li>
+<li>A string containing a CSV of hostnames with ports (e.g. <tt>hostname1:1234,hostname2:1234,hostname3:1234</tt>) without spaces between.  <tt>es.port</tt> is ignored.</li>
+<li>A list of hostnames with ports (e.g. <tt>[ &quot;hostname1:1234&quot;, &quot;hostname2:1234&quot;]</tt>).  Note, <tt>es.port</tt> is NOT used in this construction.</li>
 </ul></div>
 <div class="section">
 <h3><a name="es.port"></a><tt>es.port</tt></h3>
-<p>The port for the elasticsearch hosts. This will be used in accordance with the discussion of <tt>es.ip</tt>.</p></div>
+<p>The port for the elasticsearch hosts.  This will be used in accordance with the discussion of <tt>es.ip</tt>.</p></div>
 <div class="section">
 <h3><a name="es.date.format"></a><tt>es.date.format</tt></h3>
-<p>The date format to use when constructing the indices. For every message, the date format will be applied to the current time and that will become the last part of the index name where the message is written to.</p>
+<p>The date format to use when constructing the indices.  For every message, the date format will be applied to the current time and that will become the last part of the index name where the message is written to.</p>
 <p>For instance, an <tt>es.date.format</tt> of <tt>yyyy.MM.dd.HH</tt> would have the consequence that the indices would roll hourly, whereas an <tt>es.date.format</tt> of <tt>yyyy.MM.dd</tt> would have the consequence that the indices would roll daily.</p></div></div>
 <div class="section">
-<h2><a name="Using_Metron_with_Elasticsearch_2.x"></a>Using Metron with Elasticsearch 2.x</h2>
-<p>With Elasticsearch 2.x, there is a requirement that all sensors templates have a nested alert field defined. This field is a dummy field, and will be obsolete in Elasticsearch 5.x. See <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_ignoring_unmapped_fields">Ignoring Unmapped Fields</a> for more information</p>
+<h2><a name="Upgrading_to_5.6.2"></a>Upgrading to 5.6.2</h2>
+<p>Users should be prepared to re-index when migrating from Elasticsearch 2.3.3 to 5.6.2. There are a number of template changes, most notably around string type handling, that may cause issues when upgrading.</p>
+<p><a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/5.6/setup-upgrade.html">https://www.elastic.co/guide/en/elasticsearch/reference/5.6/setup-upgrade.html</a></p>
+<p>Be aware that if you add a new string value and want to be able to filter and search on this value from the Alerts UI, you <b>must</b> add a mapping for that type to the appropriate Elasticsearch template. Below is more detail on how to choose the appropriate mapping type for your string value.</p></div>
+<div class="section">
+<h2><a name="Type_Mappings"></a>Type Mappings</h2>
+<p>Type mappings have changed quite a bit from ES 2.x -&gt; 5.x. Here is a brief rundown of the biggest changes. More detailed references from Elasticsearch are provided in the <a href="#Type_Mapping_References">Type Mapping References</a> section below.</p>
+<ul>
+
+<li>string fields replaced by text/keyword type</li>
+<li>strings have new default mappings as follows
+
+<div>
+<div>
+<pre class="source">{
+  &quot;type&quot;: &quot;text&quot;,
+  &quot;fields&quot;: {
+    &quot;keyword&quot;: {
+      &quot;type&quot;: &quot;keyword&quot;,
+      &quot;ignore_above&quot;: 256
+    }
+  }
+}
+</pre></div></div>
+</li>
+<li>
+
+<p>There is no longer a <tt>_timestamp</tt> field that you can set &#x201c;enabled&#x201d; on. This field now causes an exception on templates. Replace with an application-created timestamp of &#x201c;date&#x201d; type.</p>
+</li>
+</ul>
+<p>The semantics for string types have changed. In 2.x, you have the concept of index settings as either &#x201c;analyzed&#x201d; or &#x201c;not_analyzed&#x201d; which basically means &#x201c;full text&#x201d; and &#x201c;keyword&#x201d;, respectively. Analyzed text basically means the indexer will split the text using a text analyzer thus allowing you to search on substrings within the original text. &#x201c;New York&#x201d; is split and indexed as two buckets, &#x201c;New&#x201d; and &#x201c;York&#x201d;, so you can search or query for aggregate counts for those terms independently and will match against the individual terms &#x201c;New&#x201d; or &#x201c;York.&#x201d; &#x201c;Keyword&#x201d; means that the original text will not be split/analyzed during indexing and instead treated as a whole unit, i.e. &#x201c;New&#x201d; or &#x201c;York&#x201d; will not match in searches against the document containing &#x201c;New York&#x201d;, but searching on &#x201c;New York&#x201d; as the f
 ull city name will. In 5.x language instead of using the &#x201c;index&#x201d; setting, you now set the &#x201c;type&#x201d; to either &#x201c;text&#x201d; for full text, or &#x201c;keyword&#x201d; for keywords.</p>
+<p>Below is a table depicting the changes to how String types are now handled.</p>
+
+<table border="0" class="table table-striped">
+
+<tr class="a">
+	
+<th>sort, aggregate, or access values</th>
+	
+<th>ES 2.x</th>
+	
+<th>ES 5.x</th>
+	
+<th>Example</th>
+</tr>
+
+<tr class="b">
+	
+<td>no</td>
+	
+<td>
+
+<div>
+<pre><tt>&quot;my_property&quot; : {
+  &quot;type&quot;: &quot;string&quot;,
+  &quot;index&quot;: &quot;analyzed&quot;
+}
+</tt></pre></div>
+	</td>
+	
+<td>
+
+<div>
+<pre><tt>&quot;my_property&quot; : {
+  &quot;type&quot;: &quot;text&quot;
+}
+</tt></pre></div>
+    Additional defaults: &quot;index&quot;: &quot;true&quot;, &quot;fielddata&quot;: &quot;false&quot;
+	</td>
+	
+<td>
+		&quot;New York&quot; handled via in-mem search as &quot;New&quot; and &quot;York&quot; buckets. <b>No</b> aggregation or sort.
+	</td>
+</tr>
+
+<tr class="a">
+	
+<td>
+	yes
+	</td>
+	
+<td>
+
+<div>
+<pre><tt>&quot;my_property&quot;: {
+  &quot;type&quot;: &quot;string&quot;,
+  &quot;index&quot;: &quot;analyzed&quot;
+}
+</tt></pre></div>
+	</td>
+	
+<td>
+
+<div>
+<pre><tt>&quot;my_property&quot;: {
+  &quot;type&quot;: &quot;text&quot;,
+  &quot;fielddata&quot;: &quot;true&quot;
+}
+</tt></pre></div>
+	</td>
+	
+<td>
+	&quot;New York&quot; handled via in-mem search as &quot;New&quot; and &quot;York&quot; buckets. <b>Can</b> aggregate and sort.
+	</td>
+</tr>
+
+<tr class="b">
+	
+<td>
+	yes
+	</td>
+	
+<td>
+
+<div>
+<pre><tt>&quot;my_property&quot;: {
+  &quot;type&quot;: &quot;string&quot;,
+  &quot;index&quot;: &quot;not_analyzed&quot;
+}
+</tt></pre></div>
+	</td>
+	
+<td>
+
+<div>
+<pre><tt>&quot;my_property&quot; : {
+  &quot;type&quot;: &quot;keyword&quot;
+}
+</tt></pre></div>
+	</td>
+	
+<td>
+	&quot;New York&quot; searchable as single value. <b>Can</b> aggregate and sort. A search for &quot;New&quot; or &quot;York&quot; will not match against the whole value.
+	</td>
+</tr>
+
+<tr class="a">
+	
+<td>
+	yes
+	</td>
+	
+<td>
+
+<div>
+<pre><tt>&quot;my_property&quot;: {
+  &quot;type&quot;: &quot;string&quot;,
+  &quot;index&quot;: &quot;analyzed&quot;
+}
+</tt></pre></div>
+	</td>
+	
+<td>
+
+<div>
+<pre><tt>&quot;my_property&quot;: {
+  &quot;type&quot;: &quot;text&quot;,
+  &quot;fields&quot;: {
+    &quot;keyword&quot;: {
+      &quot;type&quot;: &quot;keyword&quot;,
+      &quot;ignore_above&quot;: 256
+    }
+  }
+}
+</tt></pre></div>
+	</td>
+	
+<td>
+	&quot;New York&quot; searchable as single value or as text document, can aggregate and sort on the sub term &quot;keyword.&quot;
+	</td>
+</tr>
+</table>
+
+<p>If you want to set default string behavior for all strings for a given index and type, you can do so with a mapping similar to the following (replace ${your_type_here} accordingly):</p>
+
+<div>
+<div>
+<pre class="source"># curl -XPUT 'http://${ES_HOST}:${ES_PORT}/_template/default_string_template' -d '
+{
+  &quot;template&quot;: &quot;*&quot;,
+  &quot;mappings&quot; : {
+    &quot;${your_type_here}&quot;: {
+      &quot;dynamic_templates&quot;: [
+        {
+          &quot;strings&quot;: {
+            &quot;match_mapping_type&quot;: &quot;string&quot;,
+            &quot;mapping&quot;: {
+              &quot;type&quot;: &quot;text&quot;
+            }
+          }
+        }
+      ]
+    }
+  }
+}
+'
+</pre></div></div>
+
+<p>By specifying the &#x201c;template&#x201d; property with value &#x201c;*&#x201d; the template will apply to all indexes that have documents indexed of the specified type (${your_type_here}). This results in the following template.</p>
+
+<div>
+<div>
+<pre class="source"># curl -XGET 'http://${ES_HOST}:${ES_PORT}/_template/default_string_template?pretty'
+{
+  &quot;default_string_template&quot; : {
+    &quot;order&quot; : 0,
+    &quot;template&quot; : &quot;*&quot;,
+    &quot;settings&quot; : { },
+    &quot;mappings&quot; : {
+      &quot;${your_type_here}&quot; : {
+        &quot;dynamic_templates&quot; : [
+          {
+            &quot;strings&quot; : {
+              &quot;match_mapping_type&quot; : &quot;string&quot;,
+              &quot;mapping&quot; : {
+                &quot;type&quot; : &quot;text&quot;
+              }
+            }
+          }
+        ]
+      }
+    },
+    &quot;aliases&quot; : { }
+  }
+}
+</pre></div></div>
+
+<p>Notes on other settings for types in ES</p>
+<ul>
+
+<li>doc_values
+<ul>
+
+<li>on-disk data structure</li>
+<li>provides access for sorting, aggregation, and field values</li>
+<li>stores same values as _source, but in column-oriented fashion better for sorting and aggregating</li>
+<li>not supported on text fields</li>
+<li>enabled by default</li>
+</ul>
+</li>
+<li>fielddata
+<ul>
+
+<li>in-memory data structure</li>
+<li>provides access for sorting, aggregation, and field values</li>
+<li>primarily for text fields</li>
+<li>disabled by default because the heap space required can be large</li>
+</ul>
+</li>
+</ul>
+<div class="section">
+<div class="section">
+<div class="section">
+<h5><a name="Type_Mapping_References"></a>Type Mapping References</h5>
+<ul>
+
+<li><a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/5.6/mapping.html">https://www.elastic.co/guide/en/elasticsearch/reference/5.6/mapping.html</a></li>
+<li><a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/5.6/breaking_50_mapping_changes.html">https://www.elastic.co/guide/en/elasticsearch/reference/5.6/breaking_50_mapping_changes.html</a></li>
+<li><a class="externalLink" href="https://www.elastic.co/blog/strings-are-dead-long-live-strings">https://www.elastic.co/blog/strings-are-dead-long-live-strings</a></li>
+</ul></div></div></div></div>
+<div class="section">
+<h2><a name="Using_Metron_with_Elasticsearch_5.6.2"></a>Using Metron with Elasticsearch 5.6.2</h2>
+<p>There is a requirement that all sensors templates have a nested alert field defined.  This field is a dummy field.  See <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_ignoring_unmapped_fields">Ignoring Unmapped Fields</a> for more information</p>
 <p>Without this field, an error will be thrown during ALL searches (including from UIs, resulting in no alerts being found for any sensor). This error will be found in the REST service&#x2019;s logs.</p>
 <p>Exception seen:</p>
 
-<div class="source">
-<div class="source">
-<pre>QueryParsingException[[nested] failed to find nested object under path [alert]];
+<div>
+<div>
+<pre class="source">QueryParsingException[[nested] failed to find nested object under path [alert]];
 </pre></div></div>
-<p>There are two steps to resolve this issue. First is to update the Elasticsearch template for each sensor, so any new indices have the field. This requires retrieving the template, removing an extraneous JSON field so we can put it back later, and adding our new field.</p>
-<p>Make sure to set the ELASTICSEARCH variable appropriately. $SENSOR can contain wildcards, so if rollover has occurred, it&#x2019;s not necessary to do each index individually. The example here appends <tt>index*</tt> to get all indexes for a the provided sensor.</p>
 
-<div class="source">
-<div class="source">
-<pre>export ELASTICSEARCH=&quot;node1&quot;
+<p>There are two steps to resolve this issue.  First is to update the Elasticsearch template for each sensor, so any new indices have the field. This requires retrieving the template, removing an extraneous JSON field so we can put it back later, and adding our new field.</p>
+<p>Make sure to set the ELASTICSEARCH variable appropriately. $SENSOR can contain wildcards, so if rollover has occurred, it&#x2019;s not necessary to do each index individually. The example here appends <tt>index*</tt> to get all indexes for the provided sensor.</p>
+
+<div>
+<div>
+<pre class="source">export ELASTICSEARCH=&quot;node1&quot;
 export SENSOR=&quot;bro&quot;
 curl -XGET &quot;http://${ELASTICSEARCH}:9200/_template/${SENSOR}_index*?pretty=true&quot; -o &quot;${SENSOR}.template&quot;
 sed -i '' '2d;$d' ./${SENSOR}.template
 sed -i '' '/&quot;properties&quot; : {/ a\
 &quot;alert&quot;: { &quot;type&quot;: &quot;nested&quot;},' ${SENSOR}.template
 </pre></div></div>
+
 <p>To manually verify this, you can optionally pretty print it again with:</p>
 
-<div class="source">
-<div class="source">
-<pre>python -m json.tool bro.template
+<div>
+<div>
+<pre class="source">python -m json.tool bro.template
 </pre></div></div>
+
 <p>We&#x2019;ll want to put the template back into Elasticsearch:</p>
 
-<div class="source">
-<div class="source">
-<pre>curl -XPUT &quot;http://${ELASTICSEARCH}:9200/_template/${SENSOR}_index&quot; -d @${SENSOR}.template
+<div>
+<div>
+<pre class="source">curl -XPUT &quot;http://${ELASTICSEARCH}:9200/_template/${SENSOR}_index&quot; -d @${SENSOR}.template
 </pre></div></div>
-<p>To update existing indexes, update Elasticsearch mappings with the new field for each sensor. </p>
 
-<div class="source">
-<div class="source">
-<pre>curl -XPUT &quot;http://${ELASTICSEARCH}:9200/${SENSOR}_index*/_mapping/${SENSOR}_doc&quot; -d '
+<p>To update existing indexes, update Elasticsearch mappings with the new field for each sensor.</p>
+
+<div>
+<div>
+<pre class="source">curl -XPUT &quot;http://${ELASTICSEARCH}:9200/${SENSOR}_index*/_mapping/${SENSOR}_doc&quot; -d '
 {
-        &quot;properties&quot; : {
-          &quot;alert&quot; : {
-            &quot;type&quot; : &quot;nested&quot;
-          }
-        }
+  &quot;properties&quot; : {
+    &quot;alert&quot; : {
+      &quot;type&quot; : &quot;nested&quot;
+    }
+  }
 }
 '
 rm ${SENSOR}.template
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Installing_Elasticsearch_Templates"></a>Installing Elasticsearch Templates</h2>
 <p>The stock set of Elasticsearch templates for bro, snort, yaf, error index and meta index are installed automatically during the first time install and startup of Metron Indexing service.</p>
-<p>It is possible that Elasticsearch service is not available when the Metron Indexing Service startup, in that case the Elasticsearch template will not be installed. </p>
+<p>It is possible that Elasticsearch service is not available when the Metron Indexing Service startup, in that case the Elasticsearch template will not be installed.</p>
 <p>For such a scenario, an Admin can have the template installed in two ways:</p>
 <p><i>Method 1</i> - Manually from the Ambari UI by following the flow: Ambari UI -&gt; Services -&gt; Metron -&gt; Service Actions -&gt; Elasticsearch Template Install</p>
 <p><i>Method 2</i> - Stop the Metron Indexing service, and start it again from Ambari UI. Note that the Metron Indexing service tracks if it has successfully installed the Elasticsearch templates, and will attempt to do so each time it is Started until successful.</p>
-
 <blockquote>
+
 <p>Note: If you have made any customization to your index templates, then installing Elasticsearch templates afresh will lead to overwriting your existing changes. Please exercise caution.</p>
 </blockquote></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-enrichment/Performance.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-enrichment/Performance.html b/site/current-book/metron-platform/metron-enrichment/Performance.html
new file mode 100644
index 0000000..136d939
--- /dev/null
+++ b/site/current-book/metron-platform/metron-enrichment/Performance.html
@@ -0,0 +1,802 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-enrichment/Performance.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Enrichment Performance</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Enrichment Performance</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-down"></span>Enrichment</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>Performance</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Enrichment Performance</h1>
+<p><a name="Enrichment_Performance"></a></p>
+<p>This guide defines a set of benchmarks used to measure the performance of the Enrichment topology.  The guide also provides detailed steps on how to execute those benchmarks along with advice for tuning the Unified Enrichment topology.</p>
+<ul>
+
+<li><a href="#Benchmarks">Benchmarks</a></li>
+<li><a href="#Benchmark_Execution">Benchmark Execution</a></li>
+<li><a href="#Performance_Tuning">Performance Tuning</a></li>
+<li><a href="#Benchmark_Results">Benchmark Results</a></li>
+</ul>
+<div class="section">
+<h2><a name="Benchmarks"></a>Benchmarks</h2>
+<p>The following section describes a set of enrichments that will be used to benchmark the performance of the Enrichment topology.</p>
+<ul>
+
+<li><a href="#Geo_IP_Enrichment">Geo IP Enrichment</a></li>
+<li><a href="#HBase_Enrichment">HBase Enrichment</a></li>
+<li><a href="#Stellar_Enrichment">Stellar Enrichment</a></li>
+</ul>
+<div class="section">
+<h3><a name="Geo_IP_Enrichment"></a>Geo IP Enrichment</h3>
+<p>This benchmark measures the performance of executing a Geo IP enrichment.  Given a valid IP address the enrichment will append detailed location information for that IP.  The location information is sourced from an external Geo IP data source like <a class="externalLink" href="https://github.com/maxmind/GeoIP2-java">Maxmind</a>.</p>
+<div class="section">
+<h4><a name="Configuration"></a>Configuration</h4>
+<p>Adding the following Stellar expression to the Enrichment topology configuration will define a Geo IP enrichment.</p>
+
+<div>
+<div>
+<pre class="source">geo := GEO_GET(ip_dst_addr)
+</pre></div></div>
+
+<p>After the enrichment process completes, the  telemetry message will contain a set of fields with location information for the given IP address.</p>
+
+<div>
+<div>
+<pre class="source">{
+   &quot;ip_dst_addr&quot;:&quot;151.101.129.140&quot;,
+   ...
+   &quot;geo.city&quot;:&quot;San Francisco&quot;,
+   &quot;geo.country&quot;:&quot;US&quot;,
+   &quot;geo.dmaCode&quot;:&quot;807&quot;,
+   &quot;geo.latitude&quot;:&quot;37.7697&quot;,
+   &quot;geo.location_point&quot;:&quot;37.7697,-122.3933&quot;,
+   &quot;geo.locID&quot;:&quot;5391959&quot;,
+   &quot;geo.longitude&quot;:&quot;-122.3933&quot;,
+   &quot;geo.postalCode&quot;:&quot;94107&quot;,
+ }
+</pre></div></div>
+</div></div>
+<div class="section">
+<h3><a name="HBase_Enrichment"></a>HBase Enrichment</h3>
+<p>This benchmark measures the performance of executing an enrichment that retrieves data from an external HBase table. This type of enrichment is useful for enriching telemetry from an Asset Database or other source of relatively static data.</p>
+<div class="section">
+<h4><a name="Configuration"></a>Configuration</h4>
+<p>Adding the following Stellar expression to the Enrichment topology configuration will define an Hbase enrichment.  This looks up the &#x2018;ip_dst_addr&#x2019; within an HBase table &#x2018;top-1m&#x2019; and returns a hostname.</p>
+
+<div>
+<div>
+<pre class="source">top1m := ENRICHMENT_GET('top-1m', ip_dst_addr, 'top-1m', 't')
+</pre></div></div>
+
+<p>After the telemetry has been enriched, it will contain the host and IP elements that were retrieved from the HBase table.</p>
+
+<div>
+<div>
+<pre class="source">{
+	&quot;ip_dst_addr&quot;:&quot;151.101.2.166&quot;,
+	...
+	&quot;top1m.host&quot;:&quot;earther.com&quot;,
+	&quot;top1m.ip&quot;:&quot;151.101.2.166&quot;
+}
+</pre></div></div>
+</div></div>
+<div class="section">
+<h3><a name="Stellar_Enrichment"></a>Stellar Enrichment</h3>
+<p>This benchmark measures the performance of executing a basic Stellar expression.  In this benchmark, the enrichment is purely a computational task that has no dependence on an external system like a database.</p>
+<div class="section">
+<h4><a name="Configuration"></a>Configuration</h4>
+<p>Adding the following Stellar expression to the Enrichment topology configuration will define a basic Stellar enrichment.  The following returns true if the IP is in the given subnet and false otherwise.</p>
+
+<div>
+<div>
+<pre class="source">local := IN_SUBNET(ip_dst_addr, '192.168.0.0/24')
+</pre></div></div>
+
+<p>After the telemetry has been enriched, it will contain a field with a boolean value indicating whether the IP was within the given subnet.</p>
+
+<div>
+<div>
+<pre class="source">{
+	&quot;ip_dst_addr&quot;:&quot;151.101.2.166&quot;,
+	...
+	&quot;local&quot;:false
+}
+</pre></div></div>
+</div></div></div>
+<div class="section">
+<h2><a name="Benchmark_Execution"></a>Benchmark Execution</h2>
+<p>This section describes the steps necessary to execute the performance benchmarks for the Enrichment topology.</p>
+<ul>
+
+<li><a href="#Prepare_Enrichment_Data">Prepare Enrichment Data</a></li>
+<li><a href="#Load_HBase_with_Enrichment_Data">Load HBase with Enrichment Data</a></li>
+<li><a href="#Configure_the_Enrichments">Configure the Enrichments</a></li>
+<li><a href="#Create_Input_Telemetry">Create Input Telemetry</a></li>
+<li><a href="#Cluster_Setup">Cluster Setup</a></li>
+<li><a href="#Monitoring">Monitoring</a></li>
+</ul>
+<div class="section">
+<h3><a name="Prepare_Enrichment_Data"></a>Prepare Enrichment Data</h3>
+<p>The Alexa Top 1 Million was used as a data source for these benchmarks.</p>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Download the <a class="externalLink" href="http://s3.amazonaws.com/alexa-static/top-1m.csv.zip">Alexa Top 1 Million</a> or another similar data set with a variety of valid hostnames.</p>
+</li>
+<li>
+
+<p>For each hostname, query DNS to retrieve an associated IP address.</p>
+<p>A script like the following can be used for this.  There is no need to do this for all 1 million entries in the data set. Doing this for around 10,000 records is sufficient.</p>
+
+<div>
+<div>
+<pre class="source">import dns.resolver
+import csv
+#
+resolver = dns.resolver.Resolver()
+resolver.nameservers = ['8.8.8.8', '8.8.4.4']
+#
+with open('top-1m.csv', 'r') as infile:
+  with open('top-1m-with-ip.csv', 'w') as outfile:
+    #
+    reader = csv.reader(infile, delimiter=',')
+    writer = csv.writer(outfile, delimiter=',')
+    for row in reader:
+      #
+      host = row[1]
+      try:
+        response = resolver.query(host, &quot;A&quot;)
+        for record in response:
+          ip = record
+          writer.writerow([host, ip])
+          print &quot;host={}, ip={}&quot;.format(host, ip)
+        #
+      except:
+        pass
+</pre></div></div>
+</li>
+<li>
+
+<p>The resulting data set contains an IP to hostname mapping.</p>
+
+<div>
+<div>
+<pre class="source">$ head top-1m-with-ip.csv
+google.com,172.217.9.46
+youtube.com,172.217.4.78
+facebook.com,157.240.18.35
+baidu.com,220.181.57.216
+baidu.com,111.13.101.208
+baidu.com,123.125.114.144
+wikipedia.org,208.80.154.224
+yahoo.com,98.139.180.180
+yahoo.com,206.190.39.42
+reddit.com,151.101.1.140
+</pre></div></div>
+</li>
+</ol></div>
+<div class="section">
+<h3><a name="Load_HBase_with_Enrichment_Data"></a>Load HBase with Enrichment Data</h3>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Create an HBase table for this data.</p>
+<p>Ensure that the table is evenly distributed across the HBase nodes.  This can be done by pre-splitting the table or splitting the data after loading it.</p>
+
+<div>
+<div>
+<pre class="source">create 'top-1m', 't', {SPLITS =&gt; ['2','4','6','8','a','c','e']}
+</pre></div></div>
+</li>
+<li>
+
+<p>Create a configuration file called <tt>extractor.json</tt>.  This defines how the data will be loaded into the HBase table.</p>
+
+<div>
+<div>
+<pre class="source">&gt; cat extractor.json
+{
+    &quot;config&quot;: {
+        &quot;columns&quot;: {
+            &quot;host&quot; : 0,
+            &quot;ip&quot;: 1
+        },
+        &quot;indicator_column&quot;: &quot;ip&quot;,
+        &quot;type&quot;: &quot;top-1m&quot;,
+        &quot;separator&quot;: &quot;,&quot;
+    },
+    &quot;extractor&quot;: &quot;CSV&quot;
+}
+</pre></div></div>
+</li>
+<li>
+
+<p>Use the <tt>flatfile_loader.sh</tt> to load the data into the HBase table.</p>
+
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/flatfile_loader.sh \
+	-e extractor.json \
+	-t top-1m \
+	-c t \
+	-i top-1m-with-ip.csv
+</pre></div></div>
+</li>
+</ol></div>
+<div class="section">
+<h3><a name="Configure_the_Enrichments"></a>Configure the Enrichments</h3>
+<ol style="list-style-type: decimal">
+
+<li>Define the Enrichments using the REPL.
+
+<div>
+<div>
+<pre class="source">&gt; $METRON_HOME/bin/stellar -z $ZOOKEEPER
+Stellar, Go!
+[Stellar]&gt;&gt;&gt; conf
+{
+  &quot;enrichment&quot;: {
+    &quot;fieldMap&quot;: {
+     &quot;stellar&quot; : {
+       &quot;config&quot; : {
+         &quot;geo&quot; : &quot;GEO_GET(ip_dst_addr)&quot;,
+         &quot;top1m&quot; : &quot;ENRICHMENT_GET('top-1m', ip_dst_addr, 'top-1m', 't')&quot;,
+         &quot;local&quot; : &quot;IN_SUBNET(ip_dst_addr, '192.168.0.0/24')&quot;
+       }
+     }
+    },
+    &quot;fieldToTypeMap&quot;: {
+    }
+  },
+  &quot;threatIntel&quot;: {
+  }
+}
+[Stellar]&gt;&gt;&gt; CONFIG_PUT(&quot;ENRICHMENT&quot;, conf, &quot;asa&quot;)
+</pre></div></div>
+</li>
+</ol></div>
+<div class="section">
+<h3><a name="Create_Input_Telemetry"></a>Create Input Telemetry</h3>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Create a template file that defines what your input telemetry will look-like.</p>
+
+<div>
+<div>
+<pre class="source">&gt; cat asa.template
+{&quot;ciscotag&quot;: &quot;ASA-1-123123&quot;, &quot;source.type&quot;: &quot;asa&quot;, &quot;ip_dst_addr&quot;: &quot;$DST_ADDR&quot;, &quot;original_string&quot;: &quot;&lt;134&gt;Feb 22 17:04:43 AHOSTNAME %ASA-1-123123: Built inbound ICMP connection for faddr 192.168.11.8/50244 gaddr 192.168.1.236/0 laddr 192.168.1.1/161&quot;, &quot;ip_src_addr&quot;: &quot;192.168.1.35&quot;, &quot;syslog_facility&quot;: &quot;local1&quot;, &quot;action&quot;: &quot;built&quot;, &quot;syslog_host&quot;: &quot;AHOSTNAME&quot;, &quot;timestamp&quot;: &quot;$METRON_TS&quot;, &quot;protocol&quot;: &quot;icmp&quot;, &quot;guid&quot;: &quot;$METRON_GUID&quot;, &quot;syslog_severity&quot;: &quot;info&quot;}
+</pre></div></div>
+</li>
+<li>
+
+<p>Use the template file along with the enrichment data to create input telemetry with varying IP addresses.</p>
+
+<div>
+<div>
+<pre class="source">for i in $(head top-1m-with-ip.csv | awk -F, '{print $2}');do
+	cat asa.template | sed &quot;s/\$DST_ADDR/$i/&quot;;
+done &gt; asa.input.template
+</pre></div></div>
+</li>
+<li>
+
+<p>Use the <tt>load_tool.sh</tt> script to push messages onto the input topic <tt>enrichments</tt> and monitor the output topic <tt>indexing</tt>.  See more information in the Performance <a href="metron-contrib/metron-performance/index.html">README.md</a>.</p>
+<p>If the topology is keeping up, obviously the events per second produced on the input topic should roughly match the output topic.</p>
+</li>
+</ol></div>
+<div class="section">
+<h3><a name="Cluster_Setup"></a>Cluster Setup</h3>
+<div class="section">
+<h4><a name="Isolation"></a>Isolation</h4>
+<p>The Enrichment topology depends on an environment with at least two and often three components that work together; Storm, Kafka, and HBase.  When any of two of these are run on the same node, it can be difficult to identify which of them is becoming a bottleneck.  This can cause poor and highly volatile performance as each steals resources from the other.</p>
+<p>It is highly recommended that each of these systems be fully isolated from the others.  Storm should be run on nodes that are completely isolated from Kafka and HBase.</p></div></div>
+<div class="section">
+<h3><a name="Monitoring"></a>Monitoring</h3>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>The <tt>load_test.sh</tt> script will report the throughput for the input and output topics.</p>
+<ul>
+
+<li>
+
+<p>The input throughput should roughly match the output throughput if the topology is able to handle a given load.</p>
+</li>
+<li>
+
+<p>Not only are the raw throughput numbers important, but also the consistency of what is reported over time.  If the reported throughput is sporadic, then further tuning may be required.</p>
+</li>
+</ul>
+</li>
+<li>
+
+<p>The Storm UI is obviously an important source of information.  The bolt capacity, complete latency, and any reported errors are all important to monitor</p>
+</li>
+<li>
+
+<p>The load reported by the OS is also an important metric to monitor.</p>
+<ul>
+
+<li>
+
+<p>The load metric should be monitored to ensure that each node is being pushed sufficiently, but not too much.</p>
+</li>
+<li>
+
+<p>The load should be evenly distributed across each node.  If the load is uneven, this may indicate a problem.</p>
+</li>
+</ul>
+<p>A simple script like the following is sufficient for the task.</p>
+
+<div>
+<div>
+<pre class="source">for host in $(cat cluster.txt); do
+  echo $host;
+  ssh root@$host 'uptime';
+done
+</pre></div></div>
+</li>
+<li>
+
+<p>Monitoring the Kafka offset lags indicates how far behind a consumer may be.  This can be very useful to determine if the topology is keeping up.</p>
+
+<div>
+<div>
+<pre class="source">${KAFKA_HOME}/bin/kafka-consumer-groups.sh \
+    --command-config=/tmp/consumergroup.config \
+    --describe \
+    --group enrichments \
+    --bootstrap-server $BROKERLIST \
+    --new-consumer
+</pre></div></div>
+</li>
+<li>
+
+<p>A tool like <a class="externalLink" href="https://github.com/yahoo/kafka-manager">Kafka Manager</a> is also very useful for monitoring the input and output topics during test execution.</p>
+</li>
+</ol></div></div>
+<div class="section">
+<h2><a name="Performance_Tuning"></a>Performance Tuning</h2>
+<p>The approach to tuning the topology will look something like the following.  More detailed tuning information is available next to each named parameter</p>
+<ul>
+
+<li>
+
+<p>Start the tuning process with a single worker.  After tuning the bolts within a single worker, scale out with additional worker processes.</p>
+</li>
+<li>
+
+<p>Initially set the thread pool size to 1.  Increase this value slowly only after tuning the other parameters first.  Consider that each worker has its own thread pool and the total size of this thread pool should be far less than the total number of cores available in the cluster.</p>
+</li>
+<li>
+
+<p>Initially set each bolt parallelism hint to the number of partitions on the input Kafka topic.  Monitor bolt capacity and increase the parallelism hint for any bolt whose capacity is close to or exceeds 1.</p>
+</li>
+<li>
+
+<p>If the topology is not able to keep-up with a given input, then increasing the parallelism is the primary means to scale up.</p>
+</li>
+<li>
+
+<p>Parallelism units can be used for determining how to distribute processing tasks across the topology.  The sum of parallelism can be close to, but should not far exceed this value.</p>
+<p>(number of worker nodes in cluster * number cores per worker node) - (number of acker tasks)</p>
+</li>
+<li>
+
+<p>The throughput that the topology is able to sustain should be relatively consistent.  If the throughput fluctuates greatly, increase back pressure using <a href="#topology.max.spout.pending"><tt>topology.max.spout.pending</tt></a>.</p>
+</li>
+</ul>
+<div class="section">
+<h3><a name="Parameters"></a>Parameters</h3>
+<p>The following parameters are useful for tuning the &#x201c;Unified&#x201d; Enrichment topology.</p>
+<p>WARNING: Some of the parameter names have been reused from the &#x201c;Split/Join&#x201d; topology so the name may not be appropriate. This will be corrected in the future.</p>
+<ul>
+
+<li><a href="#enrichment.workers"><tt>enrichment.workers</tt></a></li>
+<li><a href="#enrichment.acker.executors"><tt>enrichment.acker.executors</tt></a></li>
+<li><a href="#topology.worker.childopts"><tt>topology.worker.childopts</tt></a></li>
+<li><a href="#topology.max.spout.pending"><tt>topology.max.spout.pending</tt></a></li>
+<li><a href="#kafka.spout.parallelism"><tt>kafka.spout.parallelism</tt></a></li>
+<li><a href="#enrichment.join.parallelism"><tt>enrichment.join.parallelism</tt></a></li>
+<li><a href="#threat.intel.join.parallelism"><tt>threat.intel.join.parallelism</tt></a></li>
+<li><a href="#kafka.writer.parallelism"><tt>kafka.writer.parallelism</tt></a></li>
+<li><a href="#enrichment.join.cache.size"><tt>enrichment.join.cache.size</tt></a></li>
+<li><a href="#threat.intel.join.cache.size"><tt>threat.intel.join.cache.size</tt></a></li>
+<li><a href="#metron.threadpool.size"><tt>metron.threadpool.size</tt></a></li>
+<li><a href="#metron.threadpool.type"><tt>metron.threadpool.type</tt></a></li>
+</ul>
+<div class="section">
+<h4><a name="enrichment.workers"></a><tt>enrichment.workers</tt></h4>
+<p>The number of worker processes for the enrichment topology.</p>
+<ul>
+
+<li>
+
+<p>Start by tuning only a single worker.  Maximize throughput for that worker, then increase the number of workers.</p>
+</li>
+<li>
+
+<p>The throughput should scale relatively linearly as workers are added.  This reaches a limit as the number of workers running on a single node saturate the resources available.  When this happens, adding workers, but on additional nodes should allow further scaling.</p>
+</li>
+<li>
+
+<p>Increase parallelism before attempting to increase the number of workers.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="enrichment.acker.executors"></a><tt>enrichment.acker.executors</tt></h4>
+<p>The number of ackers within the topology.</p>
+<ul>
+
+<li>
+
+<p>This should most often be equal to the number of workers defined in <tt>enrichment.workers</tt>.</p>
+</li>
+<li>
+
+<p>Within the Storm UI, click the &#x201c;Show System Stats&#x201d; button.  This will display a bolt named <tt>__acker</tt>.  If the capacity of this bolt is too high, then increase the number of ackers.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="topology.worker.childopts"></a><tt>topology.worker.childopts</tt></h4>
+<p>This parameter accepts arguments that will be passed to the JVM created for each Storm worker.  This allows for control over the heap size, garbage collection, and any other JVM-specific parameter.</p>
+<ul>
+
+<li>
+
+<p>Start with a 2G heap and increase as needed.  Running with 8G was found to be beneficial, but will vary depending on caching needs.</p>
+<p><tt>-Xms8g -Xmx8g</tt></p>
+</li>
+<li>
+
+<p>The Garbage First Garbage Collector (G1GC) is recommended along with a cap on the amount of time spent in garbage collection.  This is intended to help address small object allocation issues due to our extensive use of caches.</p>
+<p><tt>-XX:+UseG1GC -XX:MaxGCPauseMillis=100</tt></p>
+</li>
+<li>
+
+<p>If the caches in use are very large (as defined by either <a href="#enrichment.join.cache.size"><tt>enrichment.join.cache.size</tt></a> or <a href="#threat.intel.join.cache.size"><tt>threat.intel.join.cache.size</tt></a>) and performance is poor, turning on garbage collection logging might be helpful.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="topology.max.spout.pending"></a><tt>topology.max.spout.pending</tt></h4>
+<p>This limits the number of unacked tuples that the spout can introduce into the topology.</p>
+<ul>
+
+<li>
+
+<p>Decreasing this value will increase back pressure and allow the topology to consume messages at a pace that is maintainable.</p>
+</li>
+<li>
+
+<p>If the spout throws &#x2018;Commit Failed Exceptions&#x2019; then the topology is not keeping up.  Decreasing this value is one way to ensure that messages can be processed before they time out.</p>
+</li>
+<li>
+
+<p>If the topology&#x2019;s throughput is unsteady and inconsistent, decrease this value.  This should help the topology consume messages at a maintainable pace.</p>
+</li>
+<li>
+
+<p>If the bolt capacity is low, the topology can handle additional load.  Increase this value so that more tuples are introduced into the topology which should increase the bolt capacity.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="kafka.spout.parallelism"></a><tt>kafka.spout.parallelism</tt></h4>
+<p>The parallelism of the Kafka spout within the topology.  Defines the maximum number of executors for each worker dedicated to running the spout.</p>
+<ul>
+
+<li>
+
+<p>The spout parallelism should most often be set to the number of partitions of the input Kafka topic.</p>
+</li>
+<li>
+
+<p>If the enrichment bolt capacity is low, increasing the parallelism of the spout can introduce additional load on the topology.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="enrichment.join.parallelism"></a><tt>enrichment.join.parallelism</tt></h4>
+<p>The parallelism hint for the enrichment bolt.  Defines the maximum number of executors within each worker dedicated to running the enrichment bolt.</p>
+<p>WARNING: The property name does not match its current usage in the Unified topology.  This property name may change in the near future as it has been reused from the Split-Join topology.</p>
+<ul>
+
+<li>
+
+<p>If the capacity of the enrichment bolt is high, increasing the parallelism will introduce additional executors to bring the bolt capacity down.</p>
+</li>
+<li>
+
+<p>If the throughput of the topology is too low, increase this value.  This allows additional tuples to be enriched in parallel.</p>
+</li>
+<li>
+
+<p>Increasing parallelism on the enrichment bolt will at some point put pressure on the downstream threat intel and output bolts.  As this value is increased, monitor the capacity of the downstream bolts to ensure that they do not become a bottleneck.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="threat.intel.join.parallelism"></a><tt>threat.intel.join.parallelism</tt></h4>
+<p>The parallelism hint for the threat intel bolt.  Defines the maximum number of executors within each worker dedicated to running the threat intel bolt.</p>
+<p>WARNING: The property name does not match its current usage in the Unified topology.  This property name may change in the near future as it has been reused from the Split-Join topology.</p>
+<ul>
+
+<li>
+
+<p>If the capacity of the threat intel bolt is high, increasing the parallelism will introduce additional executors to bring the bolt capacity down.</p>
+</li>
+<li>
+
+<p>If the throughput of the topology is too low, increase this value.  This allows additional tuples to be enriched in parallel.</p>
+</li>
+<li>
+
+<p>Increasing parallelism on this bolt will at some point put pressure on the downstream output bolt.  As this value is increased, monitor the capacity of the output bolt to ensure that it does not become a bottleneck.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="kafka.writer.parallelism"></a><tt>kafka.writer.parallelism</tt></h4>
+<p>The parallelism hint for the output bolt which writes to the output Kafka topic.  Defines the maximum number of executors within each worker dedicated to running the output bolt.</p>
+<ul>
+
+<li>If the capacity of the output bolt is high, increasing the parallelism will introduce additional executors to bring the bolt capacity down.</li>
+</ul></div>
+<div class="section">
+<h4><a name="enrichment.join.cache.size"></a><tt>enrichment.join.cache.size</tt></h4>
+<p>The Enrichment bolt maintains a cache so that if the same enrichment occurs repetitively, the value can be retrieved from the cache instead of it being recomputed.</p>
+<p>There is a great deal of repetition in network telemetry, which leads to a great deal of repetition for the enrichments that operate on that telemetry.  Having a highly performant cache is one of the most critical factors driving performance.</p>
+<p>WARNING: The property name does not match its current usage in the Unified topology.  This property name may change in the near future as it has been reused from the Split-Join topology.</p>
+<ul>
+
+<li>
+
+<p>Increase the size of the cache to improve the rate of cache hits.</p>
+</li>
+<li>
+
+<p>Increasing the size of the cache may require that you increase the worker heap size using `topology.worker.childopts&#x2019;.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="threat.intel.join.cache.size"></a><tt>threat.intel.join.cache.size</tt></h4>
+<p>The Threat Intel bolt maintains a cache so that if the same enrichment occurs repetitively, the value can be retrieved from the cache instead of it being recomputed.</p>
+<p>There is a great deal of repetition in network telemetry, which leads to a great deal of repetition for the enrichments that operate on that telemetry.  Having a highly performant cache is one of the most critical factors driving performance.</p>
+<p>WARNING: The property name does not match its current usage in the Unified topology.  This property name may change in the near future as it has been reused from the Split-Join topology.</p>
+<ul>
+
+<li>
+
+<p>Increase the size of the cache to improve the rate of cache hits.</p>
+</li>
+<li>
+
+<p>Increasing the size of the cache may require that you increase the worker heap size using `topology.worker.childopts&#x2019;.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="metron.threadpool.size"></a><tt>metron.threadpool.size</tt></h4>
+<p>This value defines the number of threads maintained within a pool to execute each enrichment.  This value can either be a fixed number or it can be a multiple of the number of cores (5C = 5 times the number of cores).</p>
+<p>The enrichment bolt maintains a static thread pool that is used to execute each enrichment.  This thread pool is shared by all of the executors running within the same worker.</p>
+<p>WARNING: This value must be manually defined within the flux file at <tt>$METRON_HOME/flux/enrichment/remote-unified.yaml</tt>.  This value cannot be altered within Ambari at this time.</p>
+<ul>
+
+<li>
+
+<p>Start with a thread pool size of 1.  Adjust this value after tuning all other parameters first.  Only increase this value if testing shows performance improvements in your environment given your workload.</p>
+</li>
+<li>
+
+<p>If the thread pool size is too large this will cause the work to be shuffled amongst multiple CPU cores, which significantly decreases performance.  Using a smaller thread pool helps pin work to a single core.</p>
+</li>
+<li>
+
+<p>If the thread pool size is too small this can negatively impact IO-intensive workloads.  Increasing the thread pool size, helps when using IO-intensive workloads with a significant cache miss rate.   A thread pool size of 3-5 can help in these cases.</p>
+</li>
+<li>
+
+<p>Most workloads will make significant use of the cache and so 1-2 threads will most likely be optimal.</p>
+</li>
+<li>
+
+<p>The bolt uses a static thread pool.  To scale out, but keep the work mostly pinned to a CPU core, add more Storm workers while keeping the thread pool size low.</p>
+</li>
+<li>
+
+<p>If a larger thread pool increases load on the system, but decreases the throughput, then it is likely that the system is thrashing.  In this case the thread pool size should be decreased.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="metron.threadpool.type"></a><tt>metron.threadpool.type</tt></h4>
+<p>The enrichment bolt maintains a static thread pool that is used to execute each enrichment.  This thread pool is shared by all of the executors running within the same worker.</p>
+<p>Defines the type of thread pool used.  This value can be either &#x201c;FIXED&#x201d; or &#x201c;WORK_STEALING&#x201d;.</p>
+<p>Currently, this value must be manually defined within the flux file at <tt>$METRON_HOME/flux/enrichment/remote-unified.yaml</tt>.  This value cannot be altered within Ambari.</p></div></div>
+<div class="section">
+<h3><a name="Benchmark_Results"></a>Benchmark Results</h3>
+<p>This section describes one execution of these benchmarks to help provide an understanding of what reasonably tuned parameters might look-like.</p>
+<p>These parameters and the throughput reported are highly dependent on the workload and resources available. The throughput is what was achievable given a reasonable amount of tuning on a small, dedicated cluster.  The throughput is largely dependent on the enrichments performed and the distribution of data within the incoming telemetry.</p>
+<p>The Enrichment topology has been show to scale relatively linearly.  Adding more resources allows for more complex enrichments, across more diverse data sets, at higher volumes.  The throughput that one might see in production largely depends on how much hardware can be committed to the task.</p>
+<div class="section">
+<h4><a name="Environment"></a>Environment</h4>
+<ul>
+
+<li>
+
+<p>Apache Metron 0.5.0 (pre-release) March, 2018</p>
+<ul>
+
+<li>This included <a class="externalLink" href="https://github.com/apache/metron/pull/947">a patch to the underlying caching mechanism</a> that greatly improves performance.</li>
+</ul>
+</li>
+<li>
+
+<p>Cisco UCS nodes</p>
+<ul>
+
+<li>32 core, 64-bit CPU (Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz)</li>
+<li>256 GB RAM</li>
+<li>x2 10G NIC bonded</li>
+<li>x4 6TB 7200 RPM disks</li>
+</ul>
+</li>
+<li>
+
+<p>Storm Supervisors are isolated and running on a dedicated set of 3 nodes.</p>
+</li>
+<li>
+
+<p>Kafka Brokers are isolated and running on a separate, dedicated set of 3 nodes.</p>
+</li>
+</ul></div>
+<div class="section">
+<h4><a name="Results"></a>Results</h4>
+<ul>
+
+<li>
+
+<p>These benchmarks executed all 3 enrichments simultaneously; the <a href="#Geo_IP_Enrichment">Geo IP Enrichment</a>, <a href="#Stellar_Enrichment">Stellar Enrichment</a> and the <a href="#HBase_Enrichment">HBase Enrichment</a>.</p>
+</li>
+<li>
+
+<p>The data used to drive the benchmark includes 10,000 unique IP addresses.  The telemetry was populated with IP addresses such that 10% of these IPs were chosen 80% of the time.  This bias was designed to mimic the typical distribution seen in real-world telemetry.</p>
+</li>
+<li>
+
+<p>The Unified Enrichment topology was able to sustain 308,000 events per second on a small, dedicated 3 node cluster.</p>
+</li>
+<li>
+
+<p>The values used to achieve these results with the Unified Enrichment topology follows.  You should not attempt to use these parameters in your topology directly.  These are specific to the environment and workload and should only be used as a guideline.</p>
+
+<div>
+<div>
+<pre class="source">enrichment.workers=9
+enrichment.acker.executors=9
+enrichment.join.cache.size=100000
+threat.intel.join.cache.size=100000
+kafka.spout.parallelism=27
+enrichment.join.parallelism=54
+threat.intel.join.parallelism=9
+kafka.writer.parallelism=27
+topology.worker.childopts=-XX:+UseG1GC -Xms8g -Xmx8g -XX:MaxGCPauseMillis=100
+topology.max.spout.pending=3000
+metron.threadpool.size=1
+metron.threadpool.type=FIXED
+</pre></div></div>
+</li>
+</ul></div></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>


[06/50] [abbrv] metron git commit: METRON-1568: Stellar should have a _ special variable which returns the message in map form closes apache/incubator-metron#1021

Posted by mm...@apache.org.
METRON-1568: Stellar should have a _ special variable which returns the message in map form closes apache/incubator-metron#1021


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/1b9828e6
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/1b9828e6
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/1b9828e6

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: 1b9828e6a68fd39fd308ebb80f1d79cd490ceaec
Parents: 523c38c
Author: cstella <ce...@gmail.com>
Authored: Thu Jun 7 10:54:05 2018 -0400
Committer: cstella <ce...@gmail.com>
Committed: Thu Jun 7 10:54:05 2018 -0400

----------------------------------------------------------------------
 .../enrichment/handler/StellarConfig.java       |  11 +-
 .../StellarTransformationTest.java              |  30 ++-
 .../adapters/stellar/StellarAdapterTest.java    |  26 ++
 .../integration/EnrichmentIntegrationTest.java  |   2 +
 .../main/config/zookeeper/enrichments/test.json |   1 +
 .../metron/pcap/filter/PcapFieldResolver.java   |   5 +
 metron-stellar/stellar-common/README.md         |   1 +
 .../metron/stellar/common/utils/ConcatMap.java  | 256 +++++++++++++++++++
 .../common/utils/StellarProcessorUtils.java     |  51 +++-
 .../metron/stellar/dsl/MapVariableResolver.java |   8 +
 .../metron/stellar/dsl/VariableResolver.java    |   1 +
 .../dsl/functions/DataStructureFunctions.java   |   4 +
 .../stellar/common/utils/ConcatMapTest.java     |  83 ++++++
 .../stellar/dsl/functions/BasicStellarTest.java |  18 ++
 14 files changed, 487 insertions(+), 10 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
index 62e0263..9f96a22 100644
--- a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
+++ b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
@@ -19,6 +19,7 @@ package org.apache.metron.common.configuration.enrichment.handler;
 
 import org.apache.metron.stellar.common.StellarAssignment;
 import org.apache.metron.stellar.common.StellarProcessor;
+import org.apache.metron.stellar.dsl.VariableResolver;
 import org.json.simple.JSONObject;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -142,8 +143,14 @@ public class StellarConfig implements Config {
   {
 
     Map<String, Object> messageSegment = new HashMap<>();
-    for(String variable : stellarFields) {
-      messageSegment.put(variable, message.get(variable));
+    if(stellarFields.contains(VariableResolver.ALL_FIELDS)) {
+      //we need to include all of the fields here.
+      messageSegment.putAll(message);
+    }
+    else {
+      for (String variable : stellarFields) {
+        messageSegment.put(variable, message.get(variable));
+      }
     }
     return messageSegment;
   }

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java b/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
index fc91844..3b7c7bb 100644
--- a/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
+++ b/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
@@ -78,6 +78,32 @@ public class StellarTransformationTest {
   @Multiline
   public static String badConfig;
 
+  /**
+   { "fieldTransformations" : [
+        { "transformation" : "STELLAR"
+        ,"output" : [ "new_field"]
+        ,"config" : {
+          "new_field" : "MAP_GET('source.type', _)"
+                    }
+        }
+                                ]
+      }
+   */
+ @Multiline
+ public static String configAll;
+
+  @Test
+  public void testConfigAll() throws Exception {
+    SensorParserConfig c = SensorParserConfig.fromBytes(Bytes.toBytes(configAll));
+    JSONObject input = new JSONObject();
+    input.put("source.type", "test");
+    for (FieldTransformer handler : c.getFieldTransformations()) {
+      handler.transformAndUpdate(input, Context.EMPTY_CONTEXT());
+    }
+    Assert.assertEquals(2, input.size());
+    Assert.assertTrue(input.containsKey("new_field"));
+    Assert.assertEquals("test", input.get("new_field"));
+  }
 
   /** { "fieldTransformations" : [
         { "transformation" : "STELLAR"
@@ -92,8 +118,8 @@ public class StellarTransformationTest {
                                 ]
       }
    */
-  @Multiline
-  public static String configRename;
+ @Multiline
+ public static String configRename;
 
  @Test
  public void testStellarRename() throws Exception {

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/adapters/stellar/StellarAdapterTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/adapters/stellar/StellarAdapterTest.java b/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/adapters/stellar/StellarAdapterTest.java
index cd1006c..e03e69f 100644
--- a/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/adapters/stellar/StellarAdapterTest.java
+++ b/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/adapters/stellar/StellarAdapterTest.java
@@ -165,6 +165,7 @@ public class StellarAdapterTest extends StellarEnrichmentTest {
   @Multiline
   public static String mapConfig_default;
 
+
   private void testMapEnrichment(String config, String field) throws Exception {
     JSONObject message = getMessage();
     EnrichmentConfig enrichmentConfig = JSONUtils.INSTANCE.load(config, EnrichmentConfig.class);
@@ -185,4 +186,29 @@ public class StellarAdapterTest extends StellarEnrichmentTest {
   public void testMapEnrichment_default() throws Exception {
     testMapEnrichment(mapConfig_default, "");
   }
+
+  /**
+   {
+    "fieldMap": {
+      "stellar" : {
+        "config" : [
+            "stmt1 := MAP_GET('source.type', _)"
+        ]
+      }
+    }
+  }
+   */
+  @Multiline
+  public static String allVariableConfig;
+
+  @Test
+  public void testAllVariableUsage() throws Exception {
+    JSONObject message = getMessage();
+    EnrichmentConfig enrichmentConfig = JSONUtils.INSTANCE.load(allVariableConfig, EnrichmentConfig.class);
+    Assert.assertNotNull(enrichmentConfig.getEnrichmentConfigs().get("stellar"));
+    ConfigHandler handler = enrichmentConfig.getEnrichmentConfigs().get("stellar");
+    JSONObject enriched = enrich(message, "", handler);
+    Assert.assertEquals("stellar_test", enriched.get("stmt1"));
+  }
+
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/integration/EnrichmentIntegrationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/integration/EnrichmentIntegrationTest.java b/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/integration/EnrichmentIntegrationTest.java
index 2e22eab..188f18b 100644
--- a/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/integration/EnrichmentIntegrationTest.java
+++ b/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/integration/EnrichmentIntegrationTest.java
@@ -294,6 +294,8 @@ public class EnrichmentIntegrationTest extends BaseIntegrationTest {
     Assert.assertEquals(1, jsonDoc.get("one"));
     Assert.assertEquals(1, jsonDoc.get("map.blah"));
     Assert.assertNotNull(jsonDoc.get("foo"));
+    Assert.assertNotNull(jsonDoc.get("alt_src_type"));
+    Assert.assertEquals("test", jsonDoc.get("alt_src_type"));
     Assert.assertEquals("TEST", jsonDoc.get("ALL_CAPS"));
     Assert.assertNotNull(jsonDoc.get("bar"));
     Assert.assertEquals("TEST", jsonDoc.get("bar"));

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-platform/metron-integration-test/src/main/config/zookeeper/enrichments/test.json
----------------------------------------------------------------------
diff --git a/metron-platform/metron-integration-test/src/main/config/zookeeper/enrichments/test.json b/metron-platform/metron-integration-test/src/main/config/zookeeper/enrichments/test.json
index 4130943..9b997a6 100644
--- a/metron-platform/metron-integration-test/src/main/config/zookeeper/enrichments/test.json
+++ b/metron-platform/metron-integration-test/src/main/config/zookeeper/enrichments/test.json
@@ -19,6 +19,7 @@
                       "map" : "{ 'blah' : 1}"
                       ,"one" : "MAP_GET('blah', map)"
                       ,"foo": "1 + 1"
+                      ,"alt_src_type" : "MAP_GET('source.type', _)"
                       }
           ,"ALL_CAPS" : "TO_UPPER(source.type)"
           ,"src_enrichment" : {

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-platform/metron-pcap/src/main/java/org/apache/metron/pcap/filter/PcapFieldResolver.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-pcap/src/main/java/org/apache/metron/pcap/filter/PcapFieldResolver.java b/metron-platform/metron-pcap/src/main/java/org/apache/metron/pcap/filter/PcapFieldResolver.java
index 4ea6836..e3ac7e5 100644
--- a/metron-platform/metron-pcap/src/main/java/org/apache/metron/pcap/filter/PcapFieldResolver.java
+++ b/metron-platform/metron-pcap/src/main/java/org/apache/metron/pcap/filter/PcapFieldResolver.java
@@ -18,6 +18,8 @@
 
 package org.apache.metron.pcap.filter;
 
+import com.google.common.collect.ImmutableList;
+import org.apache.metron.stellar.common.utils.ConcatMap;
 import org.apache.metron.stellar.dsl.VariableResolver;
 
 import java.util.HashMap;
@@ -32,6 +34,9 @@ public class PcapFieldResolver implements VariableResolver {
 
   @Override
   public Object resolve(String variable) {
+    if(variable.equals(VariableResolver.ALL_FIELDS)) {
+      return new ConcatMap(ImmutableList.of(fieldsMap));
+    }
     return fieldsMap.get(variable);
   }
 

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-stellar/stellar-common/README.md
----------------------------------------------------------------------
diff --git a/metron-stellar/stellar-common/README.md b/metron-stellar/stellar-common/README.md
index 291f1be..dcf0dbf 100644
--- a/metron-stellar/stellar-common/README.md
+++ b/metron-stellar/stellar-common/README.md
@@ -37,6 +37,7 @@ For a variety of components (threat intelligence triage and field transformation
 
 The Stellar language supports the following:
 * Referencing fields in the enriched JSON
+* Referencing all fields in the enriched JSON via the `_` reserved variable name.
 * String literals are quoted with either `'` or `"`
 * String literals support escaping for `'`, `"`, `\t`, `\r`, `\n`, and backslash 
   * The literal `'\'foo\''` would represent `'foo'`

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/ConcatMap.java
----------------------------------------------------------------------
diff --git a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/ConcatMap.java b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/ConcatMap.java
new file mode 100644
index 0000000..53a52a6
--- /dev/null
+++ b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/ConcatMap.java
@@ -0,0 +1,256 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.stellar.common.utils;
+
+
+import com.esotericsoftware.kryo.Kryo;
+import com.esotericsoftware.kryo.KryoSerializable;
+import com.esotericsoftware.kryo.io.Input;
+import com.esotericsoftware.kryo.io.Output;
+import com.google.common.base.Joiner;
+import com.google.common.collect.Iterables;
+import com.google.common.collect.Sets;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * ConcatMap is a lazy concatenation of a list of Maps.  It is lazy in that it does not construct
+ * a union of all of the maps, but rather keeps the maps separate.  Key/Value resolution is
+ * done via a first-wins strategy (i.e. the first map which has a key will be used).
+ *
+ * Also, note, that this is an immutable map, so operations which require mutation will have
+ * UnsupportedOperationException thrown.
+ */
+public class ConcatMap implements Map<String, Object>, Serializable, KryoSerializable {
+  List<Map> variableMappings = new ArrayList<>();
+  public ConcatMap(List<Map> variableMappings) {
+    this.variableMappings = variableMappings;
+  }
+
+  @Override
+  public int size() {
+    int size = 0;
+    for(Map m : variableMappings) {
+      size += m.size();
+    }
+    return size;
+  }
+
+  @Override
+  public boolean isEmpty() {
+    boolean isEmpty = true;
+    for(Map m : variableMappings) {
+      isEmpty &= m.isEmpty();
+    }
+    return isEmpty;
+  }
+
+  /**
+   * If any maps contains the key, then this will return true.
+   * @param key
+   * @return
+   */
+  @Override
+  public boolean containsKey(Object key) {
+    for(Map m : variableMappings) {
+      if(m.containsKey(key)) {
+        return true;
+      }
+    }
+    return false;
+  }
+
+  /**
+   *
+   * If any maps contains the value, then this will return true.
+   * @param value
+   * @return
+   */
+  @Override
+  public boolean containsValue(Object value) {
+    for(Map m : variableMappings) {
+      if(m.containsValue(value)) {
+        return true;
+      }
+    }
+    return false;
+  }
+
+  /**
+   * The first map which contains the key will have the associated value returned.
+   * @param key
+   * @return
+   */
+  @Override
+  public Object get(Object key) {
+    Object ret = null;
+    for(Map m : variableMappings) {
+      ret = m.get(key);
+      if(ret != null) {
+        break;
+      }
+    }
+    return ret;
+  }
+
+  /**
+   * This is an immutable map and this operation is not supported.
+   * @param key
+   * @param value
+   * @return
+   */
+  @Override
+  public Object put(String key, Object value) {
+    throw new UnsupportedOperationException("Merged map is immutable.");
+  }
+
+  /**
+   *
+   * This is an immutable map and this operation is not supported.
+   * @param key
+   * @return
+   */
+  @Override
+  public Object remove(Object key) {
+    throw new UnsupportedOperationException("Merged map is immutable.");
+  }
+
+  /**
+   *
+   * This is an immutable map and this operation is not supported.
+   * @param m
+   */
+  @Override
+  public void putAll(Map<? extends String, ?> m) {
+    throw new UnsupportedOperationException("Merged map is immutable.");
+  }
+
+  /**
+   *
+   * This is an immutable map and this operation is not supported.
+   */
+  @Override
+  public void clear() {
+    throw new UnsupportedOperationException("Merged map is immutable.");
+  }
+
+  @Override
+  public Set<String> keySet() {
+    Set<String> ret = null;
+    for(Map m : variableMappings) {
+      if(ret == null) {
+        ret = m.keySet();
+      }
+      else {
+        ret = Sets.union(ret, m.keySet());
+      }
+    }
+    return ret;
+  }
+
+  /**
+   * Note: this makes a copy of the values, so it is not fundamentally lazy.
+   * @return
+   */
+  @Override
+  public Collection<Object> values() {
+    Collection<Object> ret = new ArrayList<>(size());
+    for(Map m : variableMappings) {
+      ret.addAll(m.values());
+    }
+    return ret;
+  }
+
+  /**
+   * This is a lazy entry collection of the associated maps.  If there are duplicate keys, they will appear
+   * twice here, so be careful.
+   * @return
+   */
+  @Override
+  public Set<Entry<String, Object>> entrySet() {
+    Set<Entry<String, Object>> ret = null;
+    for(Map m : variableMappings) {
+      if(ret == null) {
+        ret = m.entrySet();
+      }
+      else {
+        ret = Sets.union(ret, m.entrySet());
+      }
+    }
+    return ret;
+  }
+
+  @Override
+  public String toString() {
+    Iterable<Iterable<Map.Entry<Object, Object>>> transformed =
+            Iterables.transform(variableMappings, x -> x.entrySet());
+    Iterable<Map.Entry<Object, Object>> it = Iterables.filter( Iterables.concat(transformed)
+                                                             , x -> x.getValue() != null
+                                                             );
+    return "{" + Joiner.on(", ").join(it) + "}";
+  }
+
+  @Override
+  public boolean equals(Object o) {
+    if (this == o) return true;
+    if (o == null || getClass() != o.getClass()) return false;
+
+    ConcatMap concatMap = (ConcatMap) o;
+
+    return variableMappings != null ? variableMappings.equals(concatMap.variableMappings) : concatMap.variableMappings == null;
+
+  }
+
+  @Override
+  public int hashCode() {
+    return variableMappings != null ? variableMappings.hashCode() : 0;
+  }
+
+  @Override
+  public void write(Kryo kryo, Output output) {
+    int numVariableMappings = variableMappings.isEmpty()?0:variableMappings.size();
+    output.writeShort(numVariableMappings);
+    for(Map m : variableMappings) {
+      byte[] b = m == null?new byte[]{}:SerDeUtils.toBytes(m);
+      output.writeInt(b.length);
+      if(b.length > 0) {
+        output.writeBytes(b);
+      }
+    }
+  }
+
+  @Override
+  public void read(Kryo kryo, Input input) {
+    int numVariableMappings = input.readShort();
+    variableMappings = new ArrayList<>(numVariableMappings);
+    for(int i = 0;i < numVariableMappings;++i) {
+      int size = input.readInt();
+      if(size > 0) {
+        byte[] bytes = input.readBytes(size);
+        Map m = SerDeUtils.fromBytes(bytes, Map.class);
+        variableMappings.add(m);
+      }
+    }
+  }
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/StellarProcessorUtils.java
----------------------------------------------------------------------
diff --git a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/StellarProcessorUtils.java b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/StellarProcessorUtils.java
index 4ad5a40..51e7aaa 100644
--- a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/StellarProcessorUtils.java
+++ b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/StellarProcessorUtils.java
@@ -62,14 +62,14 @@ public class StellarProcessorUtils {
    * validates successfully and produces a result that can be serialized correctly.
    *
    * @param expression The expression to execute.
-   * @param variables The variables to expose to the expression.
+   * @param varResolver The variable resolver to use
    * @param context The execution context.
    * @return The result of executing the expression.
    */
-  public static Object run(String expression, Map<String, Object> variables, Context context) {
+  public static Object run(String expression, VariableResolver varResolver, Context context) {
 
     validate(expression, context);
-    Object result = execute(expression, variables, context);
+    Object result = execute(expression, varResolver, context);
     ensureKryoSerializable(result, expression);
     ensureJavaSerializable(result, expression);
 
@@ -77,19 +77,44 @@ public class StellarProcessorUtils {
   }
 
   /**
+   * Execute and validate a Stellar expression.
+   *
+   * <p>This is intended for use while unit testing Stellar expressions.  This ensures that the expression
+   * validates successfully and produces a result that can be serialized correctly.
+   *
+   * @param expression The expression to execute.
+   * @param variables The variables to expose to the expression.
+   * @param context The execution context.
+   * @return The result of executing the expression.
+   */
+  public static Object run(String expression, Map<String, Object> variables, Context context) {
+    VariableResolver varResolver = new DefaultVariableResolver(
+            x -> {
+              if(x.equals(MapVariableResolver.ALL_FIELDS)) {
+                return variables;
+              }
+              return variables.get(x);
+            }
+            ,x-> x.equals(MapVariableResolver.ALL_FIELDS) || variables.containsKey(x)
+    );
+    return run(expression, varResolver, context);
+  }
+
+  /**
    * Execute a Stellar expression.
    *
    * @param expression The expression to execute.
-   * @param variables The variables available to the expression.
+   * @param variableResolver Variable Resolver to use
    * @param context The execution context.
    * @return The result of executing the expression.
    */
-  private static Object execute(String expression, Map<String, Object> variables, Context context) {
+  private static Object execute(String expression, VariableResolver variableResolver, Context context) {
 
     StellarProcessor processor = new StellarProcessor();
+
     Object result = processor.parse(
             expression,
-            new DefaultVariableResolver(x -> variables.get(x), x -> variables.containsKey(x)),
+            variableResolver,
             StellarFunctions.FUNCTION_RESOLVER(),
             context);
 
@@ -180,6 +205,20 @@ public class StellarProcessorUtils {
    * validates successfully and produces a result that can be serialized correctly.
    *
    * @param expression The expression to execute.
+   * @param variables The variables to expose to the expression.
+   * @return The result of executing the expression.
+   */
+  public static Object run(String expression, VariableResolver variables) {
+    return run(expression, variables, Context.EMPTY_CONTEXT());
+  }
+
+  /**
+   * Execute and validate a Stellar expression.
+   *
+   * <p>This is intended for use while unit testing Stellar expressions.  This ensures that the expression
+   * validates successfully and produces a result that can be serialized correctly.
+   *
+   * @param expression The expression to execute.
    * @param context The execution context.
    * @return The result of executing the expression.
    */

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/MapVariableResolver.java
----------------------------------------------------------------------
diff --git a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/MapVariableResolver.java b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/MapVariableResolver.java
index 4c02d99..872211d 100644
--- a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/MapVariableResolver.java
+++ b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/MapVariableResolver.java
@@ -18,12 +18,16 @@
 
 package org.apache.metron.stellar.dsl;
 
+
+import org.apache.metron.stellar.common.utils.ConcatMap;
+
 import java.util.ArrayList;
 import java.util.List;
 import java.util.Map;
 
 public class MapVariableResolver implements VariableResolver {
 
+
   List<Map> variableMappings = new ArrayList<>();
 
   public MapVariableResolver(Map variableMappingOne, Map... variableMapping) {
@@ -45,6 +49,10 @@ public class MapVariableResolver implements VariableResolver {
 
   @Override
   public Object resolve(String variable) {
+    if(variable != null && variable.equals(VariableResolver.ALL_FIELDS)) {
+      return new ConcatMap(variableMappings);
+    }
+
     for (Map variableMapping : variableMappings) {
       Object o = variableMapping.get(variable);
       if (o != null) {

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/VariableResolver.java
----------------------------------------------------------------------
diff --git a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/VariableResolver.java b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/VariableResolver.java
index f2624e9..fb95d27 100644
--- a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/VariableResolver.java
+++ b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/VariableResolver.java
@@ -20,6 +20,7 @@ package org.apache.metron.stellar.dsl;
 
 
 public interface VariableResolver {
+  public static final String ALL_FIELDS = "_";
   Object resolve(String variable);
   boolean exists(String variable);
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/DataStructureFunctions.java
----------------------------------------------------------------------
diff --git a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/DataStructureFunctions.java b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/DataStructureFunctions.java
index eb1db48..bebe27d 100644
--- a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/DataStructureFunctions.java
+++ b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/DataStructureFunctions.java
@@ -27,6 +27,7 @@ import org.apache.metron.stellar.common.utils.SerDeUtils;
 import java.util.Collection;
 import java.util.LinkedHashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
 
 public class DataStructureFunctions {
@@ -211,6 +212,9 @@ public class DataStructureFunctions {
       if(o instanceof Collection) {
         return ((Collection)o).size();
       }
+      else if(o instanceof Map) {
+        return ((Map)o).size();
+      }
       else if(o instanceof String) {
         String val = (String) list.get(0);
         return val == null || val.isEmpty() ? 0 : val.length();

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/utils/ConcatMapTest.java
----------------------------------------------------------------------
diff --git a/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/utils/ConcatMapTest.java b/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/utils/ConcatMapTest.java
new file mode 100644
index 0000000..4c078e8
--- /dev/null
+++ b/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/utils/ConcatMapTest.java
@@ -0,0 +1,83 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.stellar.common.utils;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public class ConcatMapTest {
+
+  @Test
+  public void testToString() throws Exception {
+    Map<String, Object> v1 = new HashMap<>();
+    v1.put("k1", "v1");
+    Map<String, Object> v2 = new HashMap<>();
+    v2.put("k2", "v2");
+    v2.put("k3", null);
+    Map<String, Object> union = new HashMap<String, Object>() {{
+      putAll(v1);
+      put("k2", "v2");
+    }};
+    ConcatMap c = create(v1, v2);
+    Assert.assertEquals(c.toString(), union.toString());
+  }
+
+  private ConcatMap create(Map... ms) {
+    List<Map> l = new ArrayList<>();
+    for(Map m : ms) {
+      l.add(m);
+    }
+    return new ConcatMap(l);
+  }
+
+  private void assertKryoserializable(ConcatMap c) {
+    byte[] serialized = SerDeUtils.toBytes(c);
+    ConcatMap deserialized = SerDeUtils.fromBytes(serialized, ConcatMap.class);
+    Assert.assertEquals(deserialized, c);
+  }
+
+  @Test
+  public void testKryoSerialization() {
+    Map<String, Object> v1 = new HashMap<>();
+    v1.put("k1", "v1");
+    Map<String, Object> v2 = new HashMap<>();
+    v2.put("k2", "v2");
+    v2.put("k3", null);
+    {
+      //multi maps
+      ConcatMap c = create(v1, v2);
+      assertKryoserializable(c);
+    }
+    {
+      //single maps
+      ConcatMap c = create(v1);
+      assertKryoserializable(c);
+    }
+    {
+      //empty maps
+      ConcatMap c = create();
+      assertKryoserializable(c);
+    }
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/BasicStellarTest.java
----------------------------------------------------------------------
diff --git a/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/BasicStellarTest.java b/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/BasicStellarTest.java
index aa4462a..ea859c5 100644
--- a/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/BasicStellarTest.java
+++ b/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/BasicStellarTest.java
@@ -25,9 +25,11 @@ import org.apache.commons.lang3.StringUtils;
 import org.apache.metron.stellar.common.StellarProcessor;
 import org.apache.metron.stellar.dsl.Context;
 import org.apache.metron.stellar.dsl.DefaultVariableResolver;
+import org.apache.metron.stellar.dsl.MapVariableResolver;
 import org.apache.metron.stellar.dsl.ParseException;
 import org.apache.metron.stellar.dsl.Stellar;
 import org.apache.metron.stellar.dsl.StellarFunction;
+import org.apache.metron.stellar.dsl.VariableResolver;
 import org.apache.metron.stellar.dsl.functions.resolver.ClasspathFunctionResolver;
 import org.junit.Assert;
 import org.junit.Ignore;
@@ -957,4 +959,20 @@ public class BasicStellarTest {
     thrown.expectMessage("The rule 'TO_UPPER(protocol)' does not return a boolean value.");
     runPredicate("TO_UPPER(protocol)", new DefaultVariableResolver(v -> variableMap.get(v),v -> variableMap.containsKey(v)));
   }
+
+  @Test
+  public void all_fields_test() {
+    final Map<String, Object> varMap1 = new HashMap<String, Object>();
+    varMap1.put("field1", "val1");
+    final Map<String, Object> varMap2 = new HashMap<String, Object>();
+    varMap2.put("field2", "val2");
+    VariableResolver resolver = new MapVariableResolver(varMap1, varMap2);
+    Assert.assertTrue(runPredicate("MAP_GET('field1', _) == 'val1'", resolver));
+    Assert.assertTrue(runPredicate("MAP_GET('field2', _) == 'val2'", resolver));
+    Assert.assertTrue(runPredicate("LENGTH(_) == 2", resolver));
+    Map<String, Object> ret = (Map<String, Object>) run("_", resolver);
+    Assert.assertEquals(2, ret.size());
+    Assert.assertEquals("val1", ret.get("field1"));
+    Assert.assertEquals("val2", ret.get("field2"));
+  }
 }


[08/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-stellar/stellar-zeppelin/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-stellar/stellar-zeppelin/index.html b/site/current-book/metron-stellar/stellar-zeppelin/index.html
new file mode 100644
index 0000000..54bc800
--- /dev/null
+++ b/site/current-book/metron-stellar/stellar-zeppelin/index.html
@@ -0,0 +1,239 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-stellar/stellar-zeppelin/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Stellar Interpreter for Apache Zeppelin</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Stellar Interpreter for Apache Zeppelin</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Stellar Interpreter for Apache Zeppelin</h1>
+<p><a class="externalLink" href="https://zeppelin.apache.org/">Apache Zeppelin</a> is a web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.  This project provides a means to run the Stellar REPL directly within a Zeppelin Notebook.</p>
+<ul>
+
+<li><a href="#Prerequisites">Prerequisites</a></li>
+<li><a href="#Installation">Installation</a></li>
+<li><a href="#Usage">Usage</a></li>
+</ul>
+<div class="section">
+<h2><a name="Prerequisites"></a>Prerequisites</h2>
+<ul>
+
+<li><a class="externalLink" href="https://zeppelin.apache.org/">Apache Zeppelin</a> 0.7.3
+<p>This is tested with version 0.7.3.  Other versions may work, but are not supported.</p></li>
+</ul></div>
+<div class="section">
+<h2><a name="Installation"></a>Installation</h2>
+<p>Currently, you need to manually install the Stellar Interpreter in Zeppelin. In the future this step could be automated by the Metron Mpack.</p>
+<p>To install the Stellar Interpreter in your Apache Zeppelin installation, follow these instructions.  This is paraphrased from the <a class="externalLink" href="https://zeppelin.apache.org/docs/latest/development/writingzeppelininterpreter.html#install-your-interpreter-binary">Zeppelin docs</a>.</p>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Build and install Metron. Metron and its dependencies will be retrieved from your local Maven repository.</p>
+
+<div>
+<div>
+<pre class="source">cd $METRON_HOME
+mvn clean install -DskipTests
+</pre></div></div>
+</li>
+<li>
+
+<p>If you do not already have Zeppelin installed, <a class="externalLink" href="https://zeppelin.apache.org/download.html">download and unpack Apache Zeppelin</a>.  Then change directories to the root of your Zeppelin download.</p>
+
+<div>
+<div>
+<pre class="source">cd $ZEPPELIN_HOME
+</pre></div></div>
+</li>
+<li>
+
+<p>Use Zeppelin&#x2019;s installation utility to install the Stellar Interpreter.</p>
+<p>If Zeppelin was already installed, make sure that it is stopped before running this command.  Update the version, &#x2018;0.5.0&#x2019; in the example below, to whatever is appropriate for your environment.</p>
+
+<div>
+<div>
+<pre class="source">bin/install-interpreter.sh --name stellar --artifact org.apache.metron:stellar-zeppelin:0.5.0
+</pre></div></div>
+</li>
+<li>
+
+<p>Start Zeppelin.</p>
+
+<div>
+<div>
+<pre class="source">bin/zeppelin-daemon.sh start
+</pre></div></div>
+</li>
+<li>
+
+<p>Navigate to Zeppelin running at <a class="externalLink" href="http://localhost:8080/">http://localhost:8080/</a>.  The Stellar Interpreter should be ready for use with a basic set of functions.</p>
+</li>
+</ol></div>
+<div class="section">
+<h2><a name="Usage"></a>Usage</h2>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Create a new notebook.</p>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Click on &#x201c;Notebook&#x201d; &gt; &#x201c;Create new note&#x201d;.</p>
+</li>
+<li>
+
+<p>Set the default Interpreter to <tt>stellar</tt>.</p>
+<p>When creating the notebook, if you define <tt>stellar</tt> as the default interpreter, then there is no need to enter <tt>%stellar</tt> at the top of each code block.</p>
+<p>If <tt>stellar</tt> is not the default interpreter, then you must enter <tt>%stellar</tt> at the top of a code block containing Stellar code.</p>
+</li>
+</ol>
+</li>
+<li>
+
+<p>In the first block, add the following Stellar, then click Run.</p>
+
+<div>
+<div>
+<pre class="source">2 in [2,3,4]
+</pre></div></div>
+</li>
+<li>
+
+<p>In the next block, check which functions are available to you.</p>
+<p>When executing Stellar&#x2019;s magic functions, you must explicitly define which interpreter should be used in the code block.  If you define &#x2018;stellar&#x2019; as the default interpreter when creating a notebook, then this is only required when using Stellar&#x2019;s magic functions.</p>
+
+<div>
+<div>
+<pre class="source">%stellar
+
+%functions
+</pre></div></div>
+
+<p>You will <b>only</b> &#x2018;see&#x2019; the functions defined within <tt>stellar-common</tt> since that is the only library that we added to the interpreter.</p>
+</li>
+<li>
+
+<p>Add additional Stellar functions to your session.</p>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Go back to the Stellar interpreter configuration and add another dependency as follows.</p>
+
+<div>
+<div>
+<pre class="source">org.apache.metron:metron-statistics:0.5.0
+</pre></div></div>
+</li>
+<li>
+
+<p>Go back to your notebook and run <tt>%functions</tt> again.  You will now see the additional functions defined within the <tt>metron-statistics</tt> project.</p>
+</li>
+</ol>
+</li>
+<li>
+
+<p>Auto-completion is also available for Stellar expressions.</p>
+<p>In another block, type &#x2018;TO_&#x2019; then press the <kbd>CTRL</kbd> + <kbd>PERIOD</kbd> keys. This will trigger the auto-complete mechanism in Stellar and display a list of matching functions or variables.</p>
+</li>
+</ol></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/use-cases/forensic_clustering/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/use-cases/forensic_clustering/index.html b/site/current-book/use-cases/forensic_clustering/index.html
index 73a087e..f67a830 100644
--- a/site/current-book/use-cases/forensic_clustering/index.html
+++ b/site/current-book/use-cases/forensic_clustering/index.html
@@ -1,218 +1,111 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/use-cases/forensic_clustering/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Problem Statement</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Problem Statement</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Problem Statement</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                                    
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-down"></i>
-        Use-cases</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Forensic_clustering</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../use-cases/geographic_login_outliers/index.html" title="Geographic_login_outliers">
-          <i class="none"></i>
-        Geographic_login_outliers</a>
-            </li>
-              </ul>
-        </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-down"></span>Use-cases</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>Forensic_clustering</a></li>
+    <li><a href="../../use-cases/geographic_login_outliers/index.html" title="Geographic_login_outliers"><span class="none"></span>Geographic_login_outliers</a></li>
+    <li><a href="../../use-cases/typosquat_detection/index.html" title="Typosquat_detection"><span class="none"></span>Typosquat_detection</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Problem Statement</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Problem Statement</h1>
 <p><a name="Problem_Statement"></a></p>
-<p>Having a forensic hash, such as <a class="externalLink" href="https://github.com/trendmicro/tlsh">TLSH</a>, is a useful tool in cybersecurity. In short, the notion is that semantically similar documents should hash to a value which also similar. Contrast this with your standard cryptographic hashes, such as SHA and MD, where small deviations in the input data will yield large deviations in the hashes.</p>
-<p>The traditional use-case is to hash input documents or binaries and compare against a known blacklist of malicious hashes. A sufficiently similar hash will indicate a match. This will avoid malicious parties fuzzing input data to avoid detection.</p>
-<p>While this is interesting, it still requires metric-space searches in a blacklist. I envisioned a slightly more interesting streaming use-case of on-the-fly clustering of data. While the TLSH hashes created do not necessarily hash to precisely the same value on similar documents, more traditional non-forensic hashes <i>do</i> collide when sufficiently similar. Namely, the Hamming distance <a class="externalLink" href="https://en.wikipedia.org/wiki/Locality-sensitive_hashing#Bit_sampling_for_Hamming_distance">LSH</a> applied to the TLSH hash would give us a way to bin semantic hashes such that similar hashes (by hamming distance) have the same hash.</p>
+<p>Having a forensic hash, such as <a class="externalLink" href="https://github.com/trendmicro/tlsh">TLSH</a>, is a useful tool in cybersecurity. In short, the notion is that semantically similar documents should hash to a value which also similar.  Contrast this with your standard cryptographic hashes, such as SHA and MD, where small deviations in the input data will yield large deviations in the hashes.</p>
+<p>The traditional use-case is to hash input documents or binaries and compare against a known blacklist of malicious hashes.  A sufficiently similar hash will indicate a match.  This will avoid malicious parties fuzzing input data to avoid detection.</p>
+<p>While this is interesting, it still requires metric-space searches in a blacklist. I envisioned a slightly more interesting streaming use-case of on-the-fly clustering of data.  While the TLSH hashes created do not necessarily hash to precisely the same value on similar documents, more traditional non-forensic hashes <i>do</i> collide when sufficiently similar. Namely, the Hamming distance <a class="externalLink" href="https://en.wikipedia.org/wiki/Locality-sensitive_hashing#Bit_sampling_for_Hamming_distance">LSH</a> applied to the TLSH hash would give us a way to bin semantic hashes such that similar hashes (by hamming distance) have the same hash.</p>
 <p>Inspired by a good <a class="externalLink" href="https://github.com/fluenda/dataworks_summit_iot_botnet/blob/master/dws-fucs-lopresto.pdf">talk</a> by Andy LoPresto and Andre Fucs de Miranda from Apache NiFi, we will proceed to take logs from the Cowrie honeypot and compute TLSH hashes and semantic bins so that users can easily find similarly malicious activity to known threats in logs.</p>
 <p>Consider the following excerpts from the Cowrie logs the authors above have shared:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;eventid&quot;: &quot;cowrie.command.success&quot;
 , &quot;timestamp&quot;: &quot;2017-09-18T11:45:25.028091Z&quot;
 , &quot;message&quot;: &quot;Command found: /bin/busybox LSUCT&quot;
@@ -224,11 +117,12 @@
 , &quot;sensor&quot;: &quot;a927e8b28666&quot;
 }
 </pre></div></div>
+
 <p>and</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;eventid&quot;: &quot;cowrie.command.success&quot;
 , &quot;timestamp&quot;: &quot;2017-09-17T04:06:39.673206Z&quot;
 , &quot;message&quot;: &quot;Command found: /bin/busybox XUSRH&quot;
@@ -240,11 +134,13 @@
 , &quot;sensor&quot;: &quot;a927e8b28666&quot;
 }
 </pre></div></div>
-<p>You will note the <tt>/bin/busybox</tt> call with a random selection afterwards.<br />Excerpting from an analysis of an IOT exploit <a class="externalLink" href="https://isc.sans.edu/diary/21543">here</a>:</p>
 
-<div class="source">
-<div class="source">
-<pre>The use of the command &quot;busybox ECCHI&quot; appears to have two functions.
+<p>You will note the <tt>/bin/busybox</tt> call with a random selection afterwards.<br />
+Excerpting from an analysis of an IOT exploit <a class="externalLink" href="https://isc.sans.edu/diary/21543">here</a>:</p>
+
+<div>
+<div>
+<pre class="source">The use of the command &quot;busybox ECCHI&quot; appears to have two functions.
 First of all, cowrie, and more &quot;complete&quot; Linux distrubtions then
 commonly found on DVRs will respond with a help screen if a wrong module
 is used. So this way, &quot;ECCHI&quot; can be used to detect honeypots and
@@ -253,77 +149,67 @@ Secondly, the command is used as a market to indicate that the prior
 command finished. Later, the attacker adds &quot;/bin/busybox ECCHI&quot; at the
 end of each line, following the actual command to be executed.
 </pre></div></div>
-<p>We have a few options at our disposal:</p>
 
+<p>We have a few options at our disposal:</p>
 <ul>
-  
+
 <li>If we were merely filtering and alerting on the execution of <tt>/bin/busybox</tt> we would include false positives.</li>
-  
 <li>If we looked at <tt>/bin/busybox XUSRH</tt>, we&#x2019;d miss many attempts with a <i>different</i> value as <tt>XUSRH</tt> is able to be swapped out for another random sequence to foil overly strict rules.</li>
-  
 <li>If we looked for <tt>/bin/busybox *</tt> then we&#x2019;d capture this scenario well, but it&#x2019;d be nice to be able to not be specific to detecting the <tt>/bin/busybox</tt> style of exploits.</li>
 </ul>
-<p>Indeed, this is precisely what semantic hashing and binning allows us, the ability to group by semantic similarity without being too specific about what we mean of as &#x201c;semantic&#x201d; or &#x201c;similar&#x201d;. We want to cast a wide net, but not pull back every fish in the sea.</p>
-<p>For this demonstration, we will </p>
-
+<p>Indeed, this is precisely what semantic hashing and binning allows us, the ability to group by semantic similarity without being too specific about what we mean of as &#x201c;semantic&#x201d; or &#x201c;similar&#x201d;.  We want to cast a wide net, but not pull back every fish in the sea.</p>
+<p>For this demonstration, we will</p>
 <ul>
-  
+
 <li>ingest some 400 cowrie records</li>
-  
 <li>tag records from an IP blacklist for known malicious actors</li>
-  
 <li>use the alerts UI to investigate and find similar attacks.</li>
 </ul>
 <div class="section">
 <h2><a name="Preliminaries"></a>Preliminaries</h2>
 <p>We assume that the following environment variables are set:</p>
-
 <ul>
-  
+
 <li><tt>METRON_HOME</tt> - the home directory for metron</li>
-  
 <li><tt>ZOOKEEPER</tt> - The zookeeper quorum (comma separated with port specified: e.g. <tt>node1:2181</tt> for full-dev)</li>
-  
 <li><tt>BROKERLIST</tt> - The Kafka broker list (comma separated with port specified: e.g. <tt>node1:6667</tt> for full-dev)</li>
-  
 <li><tt>ES_HOST</tt> - The elasticsearch master (and port) e.g. <tt>node1:9200</tt> for full-dev.</li>
 </ul>
-<p>Also, this does not assume that you are using a kerberized cluster. If you are, then the parser start command will adjust slightly to include the security protocol.</p>
+<p>Also, this does not assume that you are using a kerberized cluster.  If you are, then the parser start command will adjust slightly to include the security protocol.</p>
 <p>Before editing configurations, be sure to pull the configs from zookeeper locally via</p>
 
-<div class="source">
-<div class="source">
-<pre>$METRON_HOME/bin/zk_load_configs.sh --mode PULL -z $ZOOKEEPER -o $METRON_HOME/config/zookeeper/ -f
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/zk_load_configs.sh --mode PULL -z $ZOOKEEPER -o $METRON_HOME/config/zookeeper/ -f
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Setting_up_the_Data"></a>Setting up the Data</h2>
 <p>First we must set up the cowrie log data in our cluster&#x2019;s access node.</p>
-
 <ul>
-  
+
 <li>Download the data from the github repository for the talk mentioned above <a class="externalLink" href="https://github.com/fluenda/dataworks_summit_iot_botnet/blob/master/180424243034750.tar.gz">here</a>. Ensure that&#x2019;s moved into your home directory on the metron node.</li>
-  
-<li>Create a directory called <tt>cowrie</tt> in ~ and untar the tarball into that  directory via:</li>
+<li>Create a directory called <tt>cowrie</tt> in ~ and untar the tarball into that directory via:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>mkdir ~/cowrie
+<div>
+<div>
+<pre class="source">mkdir ~/cowrie
 cd ~/cowrie
 tar xzvf ~/180424243034750.tar.gz
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Configuring_the_Parser"></a>Configuring the Parser</h2>
-<p>The Cowrie data is coming in as simple JSON blobs, so it&#x2019;s easy to parse. We really just need to adjust the timestamp and a few fields and we have valid data.</p>
-
+<p>The Cowrie data is coming in as simple JSON blobs, so it&#x2019;s easy to parse.  We really just need to adjust the timestamp and a few fields and we have valid data.</p>
 <ul>
-  
+
 <li>Create <tt>$METRON_HOME/config/zookeeper/parsers/cowrie.json</tt> with the following content:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;parserClassName&quot;:&quot;org.apache.metron.parsers.json.JSONMapParser&quot;,
   &quot;sensorTopic&quot;:&quot;cowrie&quot;,
   &quot;fieldTransformations&quot; : [
@@ -339,11 +225,12 @@ tar xzvf ~/180424243034750.tar.gz
 }
 
 </pre></div></div>
+
 <p>Before we start, we will want to install ES mappings so ES knows how to interpret our fields:</p>
 
-<div class="source">
-<div class="source">
-<pre>curl -XPUT 'http://$ES_HOST/cowrie*/_mapping/cowrie_doc' -d '
+<div>
+<div>
+<pre class="source">curl -XPUT 'http://$ES_HOST/cowrie*/_mapping/cowrie_doc' -d '
 {
         &quot;properties&quot; : {
           &quot;adapter:stellaradapter:begin:ts&quot; : {
@@ -487,36 +374,36 @@ tar xzvf ~/180424243034750.tar.gz
 </pre></div></div>
 
 <ul>
-  
+
 <li>Create the <tt>cowrie</tt> kafka topic via:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic cowrie --partitions 1 --replication-factor 1
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic cowrie --partitions 1 --replication-factor 1
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Import_the_Blacklist"></a>Import the Blacklist</h2>
-<p>Here, to build out a scenario, we will assume that we have a blacklist of known malicious hosts. For our purposes, we&#x2019;ll choose one particular host IP to be malicious.</p>
-
+<p>Here, to build out a scenario, we will assume that we have a blacklist of known malicious hosts.  For our purposes, we&#x2019;ll choose one particular host IP to be malicious.</p>
 <ul>
-  
+
 <li>Create <tt>~/blacklist.csv</tt> to contain the following:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>94.51.110.74
+<div>
+<div>
+<pre class="source">94.51.110.74
 </pre></div></div>
 
 <ul>
-  
+
 <li>Create <tt>~/blacklist_extractor.json</tt> to contain the following:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;config&quot; : {
     &quot;columns&quot; : {
        &quot;ip&quot; : 0
@@ -530,50 +417,43 @@ tar xzvf ~/180424243034750.tar.gz
 </pre></div></div>
 
 <ul>
-  
+
 <li>Import the data <tt>$METRON_HOME/bin/flatfile_loader.sh -i ~/blacklist.csv -t threatintel -c t -e ~/blacklist_extractor.json</tt></li>
 </ul>
 <p>This will create a new enrichment type &#x201c;blacklist&#x201d; with a single entry &#x201c;94.51.110.74&#x201d;.</p></div>
 <div class="section">
 <h2><a name="Configure_Enrichments"></a>Configure Enrichments</h2>
 <p>We will want to do the following:</p>
-
 <ul>
-  
+
 <li>Add enrichments to faciliate binning
-  
 <ul>
-    
-<li>Construct what we consider to be a sufficient representation of the thing we want to cluster. For our purposes, this is centered around the input command, so that would be:
-    
+
+<li>Construct what we consider to be a sufficient representation of the thing we want to cluster.  For our purposes, this is centered around the input command, so that would be:
 <ul>
-      
+
 <li>The <tt>message</tt> field</li>
-      
 <li>The <tt>input</tt> field</li>
-      
 <li>The <tt>isError</tt> field</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Compute the TLSH hash of this representation, called <tt>tlsh</tt></li>
-    
 <li>Compute the locality sensitive hash of the TLSH hash suitable for binning, called <tt>similarity_bin</tt></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Set up the threat intelligence to use the blacklist
-  
 <ul>
-    
+
 <li>Set an alert if the message is from an IP address in the threat intelligence blacklist.</li>
-    
-<li>Score blacklisted messages with <tt>10</tt>. In production, this would be more complex.</li>
-  </ul></li>
+<li>Score blacklisted messages with <tt>10</tt>.  In production, this would be more complex.</li>
+</ul>
+</li>
 </ul>
 <p>Now, we can create the enrichments thusly by creating <tt>$METRON_HOME/config/zookeeper/enrichments/cowrie.json</tt> with the following content:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;enrichment&quot;: {
     &quot;fieldMap&quot;: {
       &quot;stellar&quot; : {
@@ -615,23 +495,23 @@ tar xzvf ~/180424243034750.tar.gz
   }
 }
 </pre></div></div>
+
 <div class="section">
 <h3><a name="A_Note_About_Similarity_Hashes_and_TLSH"></a>A Note About Similarity Hashes and TLSH</h3>
-<p>Notice that we have specified a number of hash functions of <tt>16</tt> when constructing the similarity bin.<br />I arrived at that by trial and error, which is not always tenable, frankly. What is more sensible is likely to construct <i>multiple</i> similarity bins of size <tt>8</tt>, <tt>16</tt>, <tt>32</tt> at minimum.</p>
-
+<p>Notice that we have specified a number of hash functions of <tt>16</tt> when constructing the similarity bin.<br />
+I arrived at that by trial and error, which is not always tenable, frankly.  What is more sensible is likely to construct <i>multiple</i> similarity bins of size <tt>8</tt>, <tt>16</tt>, <tt>32</tt> at minimum.</p>
 <ul>
-  
+
 <li>The smaller the number of hashes, the more loose the notion of similarity (more possibly dissimilar things would get grouped together).</li>
-  
 <li>The larger the number of hashes, the more strict (similar things may not be grouped together).</li>
 </ul></div></div>
 <div class="section">
 <h2><a name="Create_the_Data_Loader"></a>Create the Data Loader</h2>
 <p>We want to pull a snapshot of the cowrie logs, so create <tt>~/load_data.sh</tt> with the following content:</p>
 
-<div class="source">
-<div class="source">
-<pre>COWRIE_HOME=~/cowrie
+<div>
+<div>
+<pre class="source">COWRIE_HOME=~/cowrie
 for i in cowrie.1626302-1636522.json cowrie.16879981-16892488.json cowrie.21312194-21331475.json cowrie.698260-710913.json cowrie.762933-772239.json cowrie.929866-939552.json cowrie.1246880-1248235.json cowrie.19285959-19295444.json cowrie.16542668-16581213.json cowrie.5849832-5871517.json cowrie.6607473-6609163.json;do
   echo $i
   cat $COWRIE_HOME/$i | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic cowrie
@@ -640,87 +520,80 @@ done
 </pre></div></div>
 
 <ul>
-  
+
 <li>Set the <tt>+x</tt> bit on the executable via:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>chmod +x ~/load_data.sh
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">chmod +x ~/load_data.sh
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Execute_Demonstration"></a>Execute Demonstration</h2>
 <p>From here, we&#x2019;ve set up our configuration and can push the configs:</p>
-
 <ul>
-  
+
 <li>Push the configs to zookeeper via</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -z $ZOOKEEPER -i $METRON_HOME/config/zookeeper/
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -z $ZOOKEEPER -i $METRON_HOME/config/zookeeper/
 </pre></div></div>
 
 <ul>
-  
+
 <li>Start the parser via:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>$METRON_HOME/bin/start_parser_topology.sh -k $BROKERLIST -z $ZOOKEEPER -s cowrie
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/start_parser_topology.sh -k $BROKERLIST -z $ZOOKEEPER -s cowrie
 </pre></div></div>
 
 <ul>
-  
+
 <li>Push cowrie data into the <tt>cowrie</tt> topic via</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>~/load_data.sh
+<div>
+<div>
+<pre class="source">~/load_data.sh
 </pre></div></div>
-<p>Once this data is loaded, we can use the Alerts UI, starting from known malicious actors, to find others doing similar things.</p>
 
+<p>Once this data is loaded, we can use the Alerts UI, starting from known malicious actors, to find others doing similar things.</p>
 <ul>
-  
+
 <li>
-<p>First we can look at the alerts directly and find an instance of our <tt>/bin/busybox</tt> activity: <img src="find_alerts.png" alt="Alerts" /></p></li>
-  
+
+<p>First we can look at the alerts directly and find an instance of our <tt>/bin/busybox</tt> activity: <img src="find_alerts.png" alt="Alerts" /></p>
+</li>
 <li>
-<p>We can now pivot and look for instances of messages with the same <tt>semantic_hash</tt> but who are <i>not</i> alerts: <img src="clustered.png" alt="Pivot" /></p></li>
+
+<p>We can now pivot and look for instances of messages with the same <tt>semantic_hash</tt> but who are <i>not</i> alerts: <img src="clustered.png" alt="Pivot" /></p>
+</li>
 </ul>
 <p>As you can see, we have found a few more malicious actors:</p>
-
 <ul>
-  
+
 <li>177.239.192.172</li>
-  
 <li>180.110.69.182</li>
-  
 <li>177.238.236.21</li>
-  
 <li>94.78.80.45</li>
 </ul>
-<p>Now we can look at <i>other</i> things that they&#x2019;re doing to build and refine our definition of what an alert is without resorting to hard-coding of rules. Note that nothing in our enrichments actually used the string <tt>busybox</tt>, so this is a more general purpose way of navigating similar things.</p></div>
-                  </div>
-            </div>
-          </div>
-
+<p>Now we can look at <i>other</i> things that they&#x2019;re doing to build and refine our definition of what an alert is without resorting to hard-coding of rules.  Note that nothing in our enrichments actually used the string <tt>busybox</tt>, so this is a more general purpose way of navigating similar things.</p></div>
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/use-cases/geographic_login_outliers/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/use-cases/geographic_login_outliers/index.html b/site/current-book/use-cases/geographic_login_outliers/index.html
index 4d0ff74..2c0441a 100644
--- a/site/current-book/use-cases/geographic_login_outliers/index.html
+++ b/site/current-book/use-cases/geographic_login_outliers/index.html
@@ -1,259 +1,148 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/use-cases/geographic_login_outliers/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Problem Statement</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Problem Statement</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Problem Statement</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                                    
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-down"></i>
-        Use-cases</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../use-cases/forensic_clustering/index.html" title="Forensic_clustering">
-          <i class="none"></i>
-        Forensic_clustering</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Geographic_login_outliers</a>
-          </li>
-              </ul>
-        </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-down"></span>Use-cases</a>
+    <ul class="nav nav-list">
+    <li><a href="../../use-cases/forensic_clustering/index.html" title="Forensic_clustering"><span class="none"></span>Forensic_clustering</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Geographic_login_outliers</a></li>
+    <li><a href="../../use-cases/typosquat_detection/index.html" title="Typosquat_detection"><span class="none"></span>Typosquat_detection</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Problem Statement</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Problem Statement</h1>
 <p><a name="Problem_Statement"></a></p>
-<p>One way to find anomalous behavior in a network is by inspecting user login behavior. In particular, if a user is logging in via vastly differing geographic locations in a short period of time, this may be evidence of malicious behavior.</p>
-<p>More formally, we can encode this potentially malicious event in terms of how far from the geographic centroid of the user&#x2019;s historic logins as compared to all users. For instance, if we track all users and the median distance from the central geographic location of all of their logins for the last 2 hours is 3 km and the standard deviation is 1 km, if we see a user logging in 1700 km from the central geographic location of their logins for the last 2 hours, then they MAY be exhibiting a deviation that we want to monitor since it would be hard to travel that distance in 4 hours. On the other hand, the user may have just used a VPN or proxy. Ultimately, this sort of analytic must be considered only one piece of evidence in addition to many others before we want to indicate an alert.</p>
+<p>One way to find anomalous behavior in a network is by inspecting user login behavior.  In particular, if a user is logging in via vastly differing geographic locations in a short period of time, this may be evidence of malicious behavior.</p>
+<p>More formally, we can encode this potentially malicious event in terms of how far from the geographic centroid of the user&#x2019;s historic logins as compared to all users.  For instance, if we track all users and the median distance from the central geographic location of all of their logins for the last 2 hours is 3 km and the standard deviation is 1 km, if we see a user logging in 1700 km from the central geographic location of their logins for the last 2 hours, then they MAY be exhibiting a deviation that we want to monitor since it would be hard to travel that distance in 4 hours.  On the other hand, the user may have just used a VPN or proxy.  Ultimately, this sort of analytic must be considered only one piece of evidence in addition to many others before we want to indicate an alert.</p>
 <p><a name="Demonstration_Design"></a></p>
 <h1>Demonstration Design</h1>
-<p>For the purposes of demonstration, we will construct synthetic data whereby 2 users are logging into a system rather quickly (once per second) from various hosts. Each user&#x2019;s locations share the same first 2 octets, but will choose the last 2 randomly. We will then inject a data point indicating <tt>user1</tt> is logging in via a russian IP address.</p>
+<p>For the purposes of demonstration, we will construct synthetic data whereby 2 users are logging into a system rather quickly (once per second) from various hosts.  Each user&#x2019;s locations share the same first 2 octets, but will choose the last 2 randomly.  We will then inject a data point indicating <tt>user1</tt> is logging in via a russian IP address.</p>
 <div class="section">
 <h2><a name="Preliminaries"></a>Preliminaries</h2>
 <p>We assume that the following environment variables are set:</p>
-
 <ul>
-  
+
 <li><tt>METRON_HOME</tt> - the home directory for metron</li>
-  
 <li><tt>ZOOKEEPER</tt> - The zookeeper quorum (comma separated with port specified: e.g. <tt>node1:2181</tt> for full-dev)</li>
-  
 <li><tt>BROKERLIST</tt> - The Kafka broker list (comma separated with port specified: e.g. <tt>node1:6667</tt> for full-dev)</li>
-  
 <li><tt>ES_HOST</tt> - The elasticsearch master (and port) e.g. <tt>node1:9200</tt> for full-dev.</li>
 </ul>
-<p>Also, this does not assume that you are using a kerberized cluster. If you are, then the parser start command will adjust slightly to include the security protocol.</p>
+<p>Also, this does not assume that you are using a kerberized cluster.  If you are, then the parser start command will adjust slightly to include the security protocol.</p>
 <p>Before editing configurations, be sure to pull the configs from zookeeper locally via</p>
 
-<div class="source">
-<div class="source">
-<pre>$METRON_HOME/bin/zk_load_configs.sh --mode PULL -z $ZOOKEEPER -o $METRON_HOME/config/zookeeper/ -f
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/zk_load_configs.sh --mode PULL -z $ZOOKEEPER -o $METRON_HOME/config/zookeeper/ -f
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Configure_the_Profiler"></a>Configure the Profiler</h2>
 <p>First, we&#x2019;ll configure the profiler to emit a profiler every 1 minute:</p>
-
 <ul>
-  
+
 <li>In Ambari, set the profiler period duration to <tt>1</tt> minute via the Profiler config section.</li>
-  
 <li>Adjust <tt>$METRON_HOME/config/zookeeper/global.json</tt> to adjust the capture duration:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre> &quot;profiler.client.period.duration&quot; : &quot;1&quot;,
+<div>
+<div>
+<pre class="source"> &quot;profiler.client.period.duration&quot; : &quot;1&quot;,
  &quot;profiler.client.period.duration.units&quot; : &quot;MINUTES&quot;
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Create_the_Data_Generator"></a>Create the Data Generator</h2>
-<p>We want to create a new sensor for our synthetic data called <tt>auth</tt>. To feed it, we need a synthetic data generator. In particular, we want a process which will feed authentication events per second for a set of users where the IPs are randomly chosen, but each user&#x2019;s login ip addresses share the same first 2 octets.</p>
+<p>We want to create a new sensor for our synthetic data called <tt>auth</tt>.  To feed it, we need a synthetic data generator.  In particular, we want a process which will feed authentication events per second for a set of users where the IPs are randomly chosen, but each user&#x2019;s login ip addresses share the same first 2 octets.</p>
 <p>Edit <tt>~/gen_data.py</tt> and paste the following into it:</p>
 
-<div class="source">
-<div class="source">
-<pre>#!/usr/bin/python
+<div>
+<div>
+<pre class="source">#!/usr/bin/python
 
 import random
 import sys
@@ -276,29 +165,26 @@ def main():
 
 if __name__ == '__main__':
   main()
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Create_the_auth_Parser"></a>Create the <tt>auth</tt> Parser</h2>
 <p>The message format for our simple synthetic data is a CSV with:</p>
-
 <ul>
-  
+
 <li>username</li>
-  
 <li>login ip address</li>
-  
 <li>timestamp</li>
 </ul>
 <p>We will need to parse this via our <tt>CSVParser</tt> and add the geohash of the login ip address.</p>
-
 <ul>
-  
+
 <li>To create this parser, edit <tt>$METRON_HOME/config/zookeeper/parsers/auth.json</tt> and paste the following:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;parserClassName&quot; : &quot;org.apache.metron.parsers.csv.CSVParser&quot;
  ,&quot;sensorTopic&quot; : &quot;auth&quot;
  ,&quot;parserConfig&quot; : {
@@ -321,29 +207,28 @@ if __name__ == '__main__':
 </pre></div></div>
 
 <ul>
-  
+
 <li>Create the kafka topic via:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic auth --partitions 1 --replication-factor 1
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic auth --partitions 1 --replication-factor 1
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Create_the_Profiles_for_Enrichment"></a>Create the Profiles for Enrichment</h2>
 <p>We will need to track 2 profiles to accomplish this task:</p>
-
 <ul>
-  
-<li><tt>locations_by_user</tt> - The geohashes of the locations the user has logged in from. This is a multiset of geohashes per user. Note that the multiset in this case is effectively a map of geohashes to occurrance counts.</li>
-  
+
+<li><tt>locations_by_user</tt> - The geohashes of the locations the user has logged in from.  This is a multiset of geohashes per user.  Note that the multiset in this case is effectively a map of geohashes to occurrance counts.</li>
 <li><tt>geo_distribution_from_centroid</tt> - The statistical distribution of the distance between a login location and the geographic centroid of the user&#x2019;s previous logins from the last 2 minutes. Note, in a real installation this would be a larger temporal lookback.</li>
 </ul>
 <p>We can represent these in the <tt>$METRON_HOME/config/zookeeper/profiler.json</tt> via the following:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;profiles&quot;: [
     {
       &quot;profile&quot;: &quot;geo_distribution_from_centroid&quot;,
@@ -371,37 +256,32 @@ if __name__ == '__main__':
     }
   ]
 }
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Enrich_authentication_Events"></a>Enrich authentication Events</h2>
 <p>We will need to enrich the authentication records in a couple of ways to use in the threat triage section as well as the profiles:</p>
-
 <ul>
-  
+
 <li><tt>geo_distance</tt>: representing the distance between the current geohash and the geographic centroid for the last 2 minutes.</li>
-  
 <li><tt>geo_centroid</tt>: representing the geographic centroid for the last 2 minutes</li>
 </ul>
 <p>Beyond that, we will need to determine if the authentication event is a geographic outlier by computing the following fields:</p>
-
 <ul>
-  
+
 <li><tt>dist_median</tt> : representing the median distance between a user&#x2019;s login location and the geographic centroid for the last 2 minutes (essentially the median of the <tt>geo_distance</tt> values across all users).</li>
-  
 <li><tt>dist_sd</tt> : representing the standard deviation of the distance between a user&#x2019;s login location and the geographic centroid for the last 2 minutes (essentially the standard deviation of the <tt>geo_distance</tt> values across all users).</li>
-  
 <li><tt>geo_outlier</tt> : whether <tt>geo_distance</tt> is more than 5 standard deviations from the median across all users.</li>
 </ul>
-<p>We also want to set up a triage rule associating a score and setting an alert if <tt>geo_outlier</tt> is true. In reality, this would be more complex as this metric is at best circumstantial and would need supporting evidence, but for simplicity we&#x2019;ll deal with the false positives.</p>
-
+<p>We also want to set up a triage rule associating a score and setting an alert if <tt>geo_outlier</tt> is true.  In reality, this would be more complex as this metric is at best circumstantial and would need supporting evidence, but for simplicity we&#x2019;ll deal with the false positives.</p>
 <ul>
-  
+
 <li>Edit <tt>$METRON_HOME/config/zookeeper/enrichments/auth.json</tt> and paste the following:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;enrichment&quot;: {
     &quot;fieldMap&quot;: {
       &quot;stellar&quot; : {
@@ -445,73 +325,73 @@ if __name__ == '__main__':
     }
   }
 }
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Execute_Demonstration"></a>Execute Demonstration</h2>
 <p>From here, we&#x2019;ve set up our configuration and can push the configs:</p>
-
 <ul>
-  
+
 <li>Push the configs to zookeeper via</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -z $ZOOKEEPER -i $METRON_HOME/config/zookeeper/
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -z $ZOOKEEPER -i $METRON_HOME/config/zookeeper/
 </pre></div></div>
 
 <ul>
-  
+
 <li>Start the parser via:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>$METRON_HOME/bin/start_parser_topology.sh -k $BROKERLIST -z $ZOOKEEPER -s auth
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/start_parser_topology.sh -k $BROKERLIST -z $ZOOKEEPER -s auth
 </pre></div></div>
 
 <ul>
-  
+
 <li>Push synthetic data into the <tt>auth</tt> topic via</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>python ~/gen_data.py |
+<div>
+<div>
+<pre class="source">python ~/gen_data.py |
 /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list $BROKERLIST --topic auth
 </pre></div></div>
 
 <ul>
-  
+
 <li>Wait for about <tt>5</tt> minutes and kill the previous command</li>
-  
 <li>Push a synthetic record indicating <tt>user1</tt> has logged in from a russian IP (<tt>109.252.227.173</tt>):</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>echo -e &quot;import time\nprint 'user1,109.252.227.173,'+str(int(time.time()))&quot; | python | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list $BROKERLIST --topic auth
+<div>
+<div>
+<pre class="source">echo -e &quot;import time\nprint 'user1,109.252.227.173,'+str(int(time.time()))&quot; | python | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list $BROKERLIST --topic auth
 </pre></div></div>
 
 <ul>
-  
+
 <li>Execute the following to search elasticsearch for our geographic login outliers:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>curl -XPOST &quot;http://$ES_HOST/auth*/_search?pretty&quot; -d '
+<div>
+<div>
+<pre class="source">curl -XPOST &quot;http://$ES_HOST/auth*/_search?pretty&quot; -d '
 {
   &quot;_source&quot; : [ &quot;is_alert&quot;, &quot;threat:triage:rules:0:reason&quot;, &quot;user&quot;, &quot;ip&quot;, &quot;geo_distance&quot; ],
   &quot;query&quot;: { &quot;exists&quot; : { &quot;field&quot; : &quot;threat:triage:rules:0:reason&quot; } }
 }
 '
 </pre></div></div>
+
 <p>You should see, among a few other false positive results, something like the following:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;_index&quot; : &quot;auth_index_2017.09.07.20&quot;,
     &quot;_type&quot; : &quot;auth_doc&quot;,
     &quot;_id&quot; : &quot;f5bdbf76-9d78-48cc-b21d-bc434c96e62e&quot;,
@@ -525,23 +405,17 @@ if __name__ == '__main__':
     }
 }
 </pre></div></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[21/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/Performance-tuning-guide.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/Performance-tuning-guide.html b/site/current-book/metron-platform/Performance-tuning-guide.html
index 941e2d7..00d2907 100644
--- a/site/current-book/metron-platform/Performance-tuning-guide.html
+++ b/site/current-book/metron-platform/Performance-tuning-guide.html
@@ -1,383 +1,676 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/Performance-tuning-guide.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Metron Performance Tuning Guide</title>
-    <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../css/site.css" />
     <link rel="stylesheet" href="../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Metron Performance Tuning Guide</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron Performance Tuning Guide</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Performance-tuning-guide</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Metron Performance Tuning Guide</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron Performance Tuning Guide</h1>
 <p><a name="Metron_Performance_Tuning_Guide"></a></p>
+<ul>
+
+<li><a href="#Overview">Overview</a></li>
+<li><a href="#General_Tuning_Suggestions">General Tuning Suggestions</a></li>
+<li><a href="#Component_Tuning_Levers">Component Tuning Levers</a></li>
+<li><a href="#Use_Case_Specific_Tuning_Suggestions">Use Case Specific Tuning Suggestions</a></li>
+<li><a href="#Debugging">Debugging</a></li>
+<li><a href="#Issues">Issues</a></li>
+<li><a href="#Reference">Reference</a></li>
+</ul>
 <div class="section">
 <h2><a name="Overview"></a>Overview</h2>
 <p>This document provides guidance from our experiences tuning the Apache Metron Storm topologies for maximum performance. You&#x2019;ll find suggestions for optimum configurations under a 1 gbps load along with some guidance around the tooling we used to monitor and assess our throughput.</p>
 <p>In the simplest terms, Metron is a streaming architecture created on top of Kafka and three main types of Storm topologies: parsers, enrichment, and indexing. Each parser has it&#x2019;s own topology and there is also a highly performant, specialized spout-only topology for streaming PCAP data to HDFS. We found that the architecture can be tuned almost exclusively through using a few primary Storm and Kafka parameters along with a few Metron-specific options. You can think of the data flow as being similar to water flowing through a pipe, and the majority of these options assist in tweaking the various pipe widths in the system.</p></div>
 <div class="section">
 <h2><a name="General_Tuning_Suggestions"></a>General Tuning Suggestions</h2>
-<p>Note that there is currently no method for specifying the number of tasks from the number of executors in Flux topologies (enrichment,  indexing). By default, the number of tasks will equal the number of executors. Logically, setting the number of tasks equal to the number of executors is sensible. Storm enforces num executors &lt;= num tasks. The reason you might set the number of tasks higher than the number of executors is for future performance tuning and rebalancing without the need to bring down your topologies. The number of tasks is fixed at topology startup time whereas the number of executors can be increased up to a maximum value equal to the number of tasks.</p>
-<p>When configuring Storm Kafka spouts, we found that the default values for poll.timeout.ms, offset.commit.period.ms, and max.uncommitted.offsets worked well in nearly all cases. As a general rule, it was optimal to set spout parallelism equal to the number of partitions used in your Kafka topic. Any greater parallelism will leave you with idle consumers since Kafka limits the max number of consumers to the number of partitions. This is important because Kafka has certain ordering guarantees for message delivery per partition that would not be possible if more than one consumer in a given consumer group were able to read from that partition.</p></div>
 <div class="section">
-<h2><a name="Component_Tuning_Levers"></a>Component Tuning Levers</h2>
+<h3><a name="Storm_Executors_vs._Tasks"></a>Storm Executors vs. Tasks</h3>
+<p>Note that there is currently no method for specifying the number of tasks from the number of executors in Flux topologies (enrichment, indexing). By default, the number of tasks will equal the number of executors. Logically, setting the number of tasks equal to the number of executors is sensible. Storm enforces num executors &lt;= num tasks. The reason you might set the number of tasks higher than the number of executors is for future performance tuning and rebalancing without the need to bring down your topologies. The number of tasks is fixed at topology startup time whereas the number of executors can be increased up to a maximum value equal to the number of tasks.</p></div>
+<div class="section">
+<h3><a name="Kafka_Spout_Configuration"></a>Kafka Spout Configuration</h3>
+<p>When configuring Storm Kafka spouts, we found that the default values for</p>
+<ul>
 
+<li><tt>poll.timeout.ms</tt></li>
+<li><tt>offset.commit.period.ms</tt></li>
+<li><tt>max.uncommitted.offsets</tt></li>
+</ul>
+<p>worked well in nearly all cases. As a general rule, it was optimal to set spout parallelism equal to the number of partitions used in your Kafka topic. Any greater parallelism will leave you with idle consumers since Kafka limits the max number of consumers to the number of partitions. This is important because Kafka has certain ordering guarantees for message delivery per partition that would not be possible if more than one consumer in a given consumer group were able to read from that partition.</p></div></div>
+<div class="section">
+<h2><a name="Sensor_Topology_Tuning_Suggestions"></a>Sensor Topology Tuning Suggestions</h2>
+<p>If you are using stellar field transformations in your sensors, by default, stellar expressions are not cached.  Sensors that use stellar field transformations by see a performance boost by turning on caching via setting the <tt>cacheConfig</tt> <a href="metron-parsers/index.html#parser_configuration">property</a>. This is beneficial if your transformations:</p>
 <ul>
-  
+
+<li>Are complex (e.g. <tt>ENRICHMENT_GET</tt> calls or other high latency calls)</li>
+<li>All Yield the same results for the same inputs ( caching is either off or applied to all transformations)
+<ul>
+
+<li>If any of your transformations are non-deterministic, caching should not be used as it will result in the likelihood of incorrect results being returned.</li>
+</ul>
+</li>
+</ul></div>
+<div class="section">
+<h2><a name="Component_Tuning_Levers"></a>Component Tuning Levers</h2>
+<div class="section">
+<h3><a name="High_Level_Overview"></a>High Level Overview</h3>
+<p>There are a number of levers that can be set while tuning a Metron cluster. The main services to interact with for performance tuning are: Kafka, Storm, HDFS, and indexing (Elasticsearch or Solr). For each service, here is a high level breakdown of the major knobs and levers that can be modified while tuning your cluster.</p>
+<ul>
+
 <li>Kafka
-  
 <ul>
-    
+
 <li>Number partitions</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Storm
-  
 <ul>
-    
+
 <li>Kafka spout
-    
 <ul>
-      
+
 <li>Polling frequency</li>
-      
 <li>Polling timeouts</li>
-      
 <li>Offset commit period</li>
-      
 <li>Max uncommitted offsets</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Number workers (OS processes)</li>
-    
 <li>Number executors (threads in a process)</li>
-    
 <li>Number ackers</li>
-    
 <li>Max spout pending</li>
-    
 <li>Spout and bolt parallelism</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>HDFS
-  
 <ul>
-    
+
 <li>Replication factor</li>
-  </ul></li>
 </ul>
+</li>
+</ul></div>
 <div class="section">
 <h3><a name="Kafka_Tuning"></a>Kafka Tuning</h3>
 <p>The main lever you&#x2019;re going to work with when tuning Kafka throughput will be the number of partitions. A handy method for deciding how many partitions to use is to first calculate the throughput for a single producer (p) and a single consumer (c), and then use that with the desired throughput (t) to roughly estimate the number of partitions to use. You would want at least max(t/p, t/c) partitions to attain the desired throughput. See <a class="externalLink" href="https://www.confluent.io/blog/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/">https://www.confluent.io/blog/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/</a> for more details.</p></div>
 <div class="section">
 <h3><a name="Storm_Tuning"></a>Storm Tuning</h3>
+<div class="section">
+<h4><a name="Overview"></a>Overview</h4>
 <p>There are quite a few options you will be confronted with when tuning your Storm topologies and this is largely trial and error. As a general rule of thumb, we recommend starting with the defaults and smaller numbers in terms of parallelism while iteratively working up until the desired performance is achieved. You will find the offset lag tool indispensable while verifying your settings.</p>
 <p>We won&#x2019;t go into a full discussion about Storm&#x2019;s architecture - see references section for more info - but there are some general rules of thumb that should be followed. It&#x2019;s first important to understand the ways you can impact parallelism in a Storm topology.</p>
-
 <ul>
-  
+
 <li>num tasks</li>
-  
 <li>num executors (parallelism hint)</li>
-  
 <li>num workers</li>
 </ul>
-<p>Tasks are instances of a given spout or bolt, executors are threads in a process, and workers are jvm processes. You&#x2019;ll want the number of tasks as a multiple of the number of executors, the number of executors as multiple of the number of workers, and the number of workers as a multiple of the number of machines. The main reason for this approach is  that it will give a uniform distribution of work to each machine and jvm process. More often than not, your number of tasks will be equal to the number of executors, which  is the default in Storm. Flux does not actually provide a way to independently set number of tasks, so for enrichments and indexing which use Flux, num tasks will always equal  num executors.</p>
-<p>You can change the number of workers via the property <tt>topology.workers</tt></p>
+<p>Tasks are instances of a given spout or bolt, executors are threads in a process, and workers are jvm processes. You&#x2019;ll want the number of tasks as a multiple of the number of executors, the number of executors as multiple of the number of workers, and the number of workers as a multiple of the number of machines. The main reason for this approach is that it will give a uniform distribution of work to each machine and jvm process. More often than not, your number of tasks will be equal to the number of executors, which is the default in Storm. Flux does not actually provide a way to independently set number of tasks, so for enrichments and indexing, which use Flux, num tasks will always equal num executors.</p>
+<p>You can change the number of workers via the Storm property <tt>topology.workers</tt></p>
 <p><b>Other Storm Settings</b></p>
 
-<div class="source">
-<div class="source">
-<pre>topology.max.spout.pending
+<div>
+<div>
+<pre class="source">topology.max.spout.pending
 </pre></div></div>
+
 <p>This is the maximum number of tuples that can be in flight (ie, not yet acked) at any given time within your topology. You set this as a form of backpressure to ensure you don&#x2019;t flood your topology.</p>
 
-<div class="source">
-<div class="source">
-<pre>topology.ackers.executors
+<div>
+<div>
+<pre class="source">topology.ackers.executors
 </pre></div></div>
+
 <p>This specifies how many threads should be dedicated to tuple acking. We found that setting this equal to the number of partitions in your inbound Kafka topic worked well.</p>
 <p><b>spout-config.json</b></p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
     ...
     &quot;spout.pollTimeoutMs&quot; : 200,
     &quot;spout.maxUncommittedOffsets&quot; : 10000000,
     &quot;spout.offsetCommitPeriodMs&quot; : 30000
 }
 </pre></div></div>
-<p>These are the spout recommended defaults from Storm and are currently the defaults provided in the Kafka spout itself. In fact, if you find the recommended defaults work fine for you, then you can omit these settings altogether.</p></div></div>
+
+<p>Above is a snippet for configuring parsers. These are the spout recommended defaults from Storm and are currently the defaults provided in the Kafka spout itself. In fact, if you find the recommended defaults work fine for you, then you can omit these settings altogether.</p></div>
+<div class="section">
+<h4><a name="Where_to_Find_Tuning_Properties"></a>Where to Find Tuning Properties</h4>
+<p><b>Important:</b> The parser topologies are deployed via a builder pattern that takes parameters from the CLI as set via Ambari. The enrichment and indexing topologies are configured using a Storm Flux file, a configuration properties file, and Ambari. Here is a setting materialization summary for each of the topology types:</p>
+<ul>
+
+<li>Parsers
+<ul>
+
+<li>Management UI -&gt; parser json config and CLI -&gt; Storm</li>
+</ul>
+</li>
+<li>Enrichment
+<ul>
+
+<li>Ambari UI -&gt; properties file -&gt; Flux -&gt; Storm</li>
+</ul>
+</li>
+<li>Indexing
+<ul>
+
+<li>Ambari UI -&gt; properties file -&gt; Flux -&gt; Storm</li>
+</ul>
+</li>
+</ul>
+<p><b>Parsers</b></p>
+<p>This is a mapping of the various performance tuning properties for parsers and how they are materialized.</p>
+<p>See more detail on starting parsers <a class="externalLink" href="https://github.com/apache/metron/blob/master/metron-platform/metron-parsers/README.md#starting-the-parser-topology">here</a></p>
+<table border="0" class="table table-striped">
+<thead>
+
+<tr class="a">
+<th> Category                    </th>
+<th> Management UI Property Name                </th>
+<th> JSON Config File Property Name     </th>
+<th> CLI Option                                                                                     </th>
+<th> Storm Property Name             </th>
+<th>  Notes                                                                        </th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td> Storm topology config       </td>
+<td> Num Workers                                </td>
+<td> n/a                                </td>
+<td> -nw,&#x2013;num_workers &lt;NUM_WORKERS&gt;                                                                </td>
+<td> topology.workers                </td>
+<td>                                                                               </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> Num Ackers                                 </td>
+<td> n/a                                </td>
+<td> -na,&#x2013;num_ackers &lt;NUM_ACKERS&gt;                                                                  </td>
+<td> topology.acker.executors        </td>
+<td>                                                                               </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> Storm Config                               </td>
+<td> topology.max.spout.pending         </td>
+<td> -e,&#x2013;extra_topology_options &lt;JSON_FILE&gt;, e.g. { &#x201c;topology.max.spout.pending&#x201d; : NUM }           </td>
+<td> topology.max.spout.pending      </td>
+<td> Put property in JSON format in a file named <tt>storm-&lt;MY_PARSER&gt;-config.json</tt>   </td></tr>
+<tr class="a">
+<td> Kafka spout                 </td>
+<td> Spout Parallelism                          </td>
+<td> n/a                                </td>
+<td> -sp,&#x2013;spout_p &lt;SPOUT_PARALLELISM_HINT&gt;                                                         </td>
+<td> n/a                             </td>
+<td>                                                                               </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> Spout Num Tasks                            </td>
+<td> n/a                                </td>
+<td> -snt,&#x2013;spout_num_tasks &lt;NUM_TASKS&gt;                                                             </td>
+<td> n/a                             </td>
+<td>                                                                               </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> Spout Config                               </td>
+<td> spout.pollTimeoutMs                </td>
+<td> -esc,&#x2013;extra_kafka_spout_config &lt;JSON_FILE&gt;, e.g. { &#x201c;spout.pollTimeoutMs&#x201d; : 200 }              </td>
+<td> n/a                             </td>
+<td> Put property in JSON format in a file named <tt>spout-&lt;MY_PARSER&gt;-config.json</tt>   </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> Spout Config                               </td>
+<td> spout.maxUncommittedOffsets        </td>
+<td> -esc,&#x2013;extra_kafka_spout_config &lt;JSON_FILE&gt;, e.g. { &#x201c;spout.maxUncommittedOffsets&#x201d; : 10000000 } </td>
+<td> n/a                             </td>
+<td> Put property in JSON format in a file named <tt>spout-&lt;MY_PARSER&gt;-config.json</tt>   </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> Spout Config                               </td>
+<td> spout.offsetCommitPeriodMs         </td>
+<td> -esc,&#x2013;extra_kafka_spout_config &lt;JSON_FILE&gt;, e.g. { &#x201c;spout.offsetCommitPeriodMs&#x201d; : 30000 }     </td>
+<td> n/a                             </td>
+<td> Put property in JSON format in a file named <tt>spout-&lt;MY_PARSER&gt;-config.json</tt>   </td></tr>
+<tr class="b">
+<td> Parser bolt                 </td>
+<td> Parser Num Tasks                           </td>
+<td> n/a                                </td>
+<td> -pnt,&#x2013;parser_num_tasks &lt;NUM_TASKS&gt;                                                            </td>
+<td> n/a                             </td>
+<td>                                                                               </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> Parser Parallelism                         </td>
+<td> n/a                                </td>
+<td> -pp,&#x2013;parser_p &lt;PARALLELISM_HINT&gt;                                                              </td>
+<td> n/a                             </td>
+<td>                                                                               </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> Parser Parallelism                         </td>
+<td> n/a                                </td>
+<td> -pp,&#x2013;parser_p &lt;PARALLELISM_HINT&gt;                                                              </td>
+<td> n/a                             </td>
+<td>                                                                               </td></tr>
+</tbody>
+</table>
+<p><b>Enrichment</b></p>
+<p>This is a mapping of the various performance tuning properties for enrichments and how they are materialized.</p>
+<p>Flux file found here - $METRON_HOME/flux/enrichment/remote.yaml</p>
+<p><i>Note 1:</i> Changes to Flux file properties that are managed by Ambari will render Ambari unable to further manage the property.</p>
+<p><i>Note 2:</i> Many of these settings will be irrelevant in the alternate non-split-join topology</p>
+<table border="0" class="table table-striped">
+<thead>
+
+<tr class="a">
+<th> Category                    </th>
+<th> Ambari Property Name                       </th>
+<th> enrichment.properties property                         </th>
+<th> Flux Property                                          </th>
+<th> Flux Section Location               </th>
+<th> Storm Property Name             </th>
+<th> Notes                                  </th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td> Storm topology config       </td>
+<td> enrichment_workers                         </td>
+<td> enrichment.workers                                     </td>
+<td> topology.workers                                       </td>
+<td> line 18, config                     </td>
+<td> topology.workers                </td>
+<td>                                        </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> enrichment_acker_executors                 </td>
+<td> enrichment.acker.executors                             </td>
+<td> topology.acker.executors                               </td>
+<td> line 18, config                     </td>
+<td> topology.acker.executors        </td>
+<td>                                        </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> enrichment_topology_max_spout_pending      </td>
+<td> topology.max.spout.pending                             </td>
+<td> topology.max.spout.pending                             </td>
+<td> line 18, config                     </td>
+<td> topology.max.spout.pending      </td>
+<td>                                        </td></tr>
+<tr class="a">
+<td> Kafka spout                 </td>
+<td> enrichment_kafka_spout_parallelism         </td>
+<td> kafka.spout.parallelism                                </td>
+<td> parallelism                                            </td>
+<td> line 245, id: kafkaSpout            </td>
+<td> n/a                             </td>
+<td>                                        </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> session.timeout.ms                                     </td>
+<td> session.timeout.ms                                     </td>
+<td> line 201, id: kafkaProps            </td>
+<td> n/a                             </td>
+<td> Kafka consumer client property         </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> enable.auto.commit                                     </td>
+<td> enable.auto.commit                                     </td>
+<td> line 201, id: kafkaProps            </td>
+<td> n/a                             </td>
+<td> Kafka consumer client property         </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> n/a                                                    </td>
+<td> setPollTimeoutMs                                       </td>
+<td> line 230, id: kafkaConfig           </td>
+<td> n/a                             </td>
+<td> Kafka consumer client property         </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> n/a                                                    </td>
+<td> setMaxUncommittedOffsets                               </td>
+<td> line 230, id: kafkaConfig           </td>
+<td> n/a                             </td>
+<td> Kafka consumer client property         </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> n/a                                                    </td>
+<td> setOffsetCommitPeriodMs                                </td>
+<td> line 230, id: kafkaConfig           </td>
+<td> n/a                             </td>
+<td> Kafka consumer client property         </td></tr>
+<tr class="a">
+<td> Enrichment splitter         </td>
+<td> enrichment_split_parallelism               </td>
+<td> enrichment.split.parallelism                           </td>
+<td> parallelism                                            </td>
+<td> line 253, id: enrichmentSplitBolt   </td>
+<td> n/a                             </td>
+<td>                                        </td></tr>
+<tr class="b">
+<td> Enrichment joiner           </td>
+<td> enrichment_join_parallelism                </td>
+<td> enrichment.join.parallelism                            </td>
+<td> parallelism                                            </td>
+<td> line 316, id: enrichmentJoinBolt    </td>
+<td> n/a                             </td>
+<td>                                        </td></tr>
+<tr class="a">
+<td> Threat intel splitter       </td>
+<td> threat_intel_split_parallelism             </td>
+<td> threat.intel.split.parallelism                         </td>
+<td> parallelism                                            </td>
+<td> line 338, id: threatIntelSplitBolt  </td>
+<td> n/a                             </td>
+<td>                                        </td></tr>
+<tr class="b">
+<td> Threat intel joiner         </td>
+<td> threat_intel_join_parallelism              </td>
+<td> threat.intel.join.parallelism                          </td>
+<td> parallelism                                            </td>
+<td> line 376, id: threatIntelJoinBolt   </td>
+<td> n/a                             </td>
+<td>                                        </td></tr>
+<tr class="a">
+<td> Output bolt                 </td>
+<td> kafka_writer_parallelism                   </td>
+<td> kafka.writer.parallelism                               </td>
+<td> parallelism                                            </td>
+<td> line 397, id: outputBolt            </td>
+<td> n/a                             </td>
+<td>                                        </td></tr>
+</tbody>
+</table>
+<p>When adding Kafka spout properties, there are 3 ways you&#x2019;ll do this.</p>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Ambari: If they are properties managed by Ambari (noted in the table under &#x2018;Ambari Property Name&#x2019;), look for the setting in Ambari.</p>
+</li>
+<li>
+
+<p>Flux -&gt; kafkaProps: add a new key/value to the kafkaProps section HashMap on line 201. For example, if you want to set the Kafka Spout consumer&#x2019;s session.timeout.ms to 30 seconds, you would add the following:</p>
+
+<div>
+<div>
+<pre class="source">       -   name: &quot;put&quot;
+           args:
+               - &quot;session.timeout.ms&quot;
+               - 30000
+</pre></div></div>
+</li>
+<li>
+
+<p>Flux -&gt; kafkaConfig: add a new setter to the kafkaConfig section on line 230. For example, if you want to set the Kafka Spout consumer&#x2019;s poll timeout to 200 milliseconds, you would add the following under <tt>configMethods</tt>:</p>
+
+<div>
+<div>
+<pre class="source">         -   name: &quot;setPollTimeoutMs&quot;
+             args:
+                 - 200
+</pre></div></div>
+</li>
+</ol>
+<p><b>Indexing (Batch)</b></p>
+<p>This is a mapping of the various performance tuning properties for indexing and how they are materialized.</p>
+<p>Flux file can be found here - $METRON_HOME/flux/indexing/batch/remote.yaml.</p>
+<p>Note: Changes to Flux file properties that are managed by Ambari will render Ambari unable to further manage the property.</p>
+<table border="0" class="table table-striped">
+<thead>
+
+<tr class="a">
+<th> Category                    </th>
+<th> Ambari Property Name                       </th>
+<th> hdfs.properties property                               </th>
+<th> Flux Property                                          </th>
+<th> Flux Section Location               </th>
+<th> Storm Property Name             </th>
+<th> Notes                                  </th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td> Storm topology config       </td>
+<td> enrichment_workers                         </td>
+<td> enrichment.workers                                     </td>
+<td> topology.workers                                       </td>
+<td> line 19, config                     </td>
+<td> topology.workers                </td>
+<td>                                        </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> enrichment_acker_executors                 </td>
+<td> enrichment.acker.executors                             </td>
+<td> topology.acker.executors                               </td>
+<td> line 19, config                     </td>
+<td> topology.acker.executors        </td>
+<td>                                        </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> enrichment_topology_max_spout_pending      </td>
+<td> topology.max.spout.pending                             </td>
+<td> topology.max.spout.pending                             </td>
+<td> line 19, config                     </td>
+<td> topology.max.spout.pending      </td>
+<td>                                        </td></tr>
+<tr class="a">
+<td> Kafka spout                 </td>
+<td> batch_indexing_kafka_spout_parallelism     </td>
+<td> kafka.spout.parallelism                                </td>
+<td> parallelism                                            </td>
+<td> line 123, id: kafkaSpout            </td>
+<td> n/a                             </td>
+<td>                                        </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> session.timeout.ms                                     </td>
+<td> session.timeout.ms                                     </td>
+<td> line 80, id: kafkaProps             </td>
+<td> n/a                             </td>
+<td> Kafka consumer client property         </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> enable.auto.commit                                     </td>
+<td> enable.auto.commit                                     </td>
+<td> line 80, id: kafkaProps             </td>
+<td> n/a                             </td>
+<td> Kafka consumer client property         </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> n/a                                                    </td>
+<td> setPollTimeoutMs                                       </td>
+<td> line 108, id: kafkaConfig           </td>
+<td> n/a                             </td>
+<td> Kafka consumer client property         </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> n/a                                                    </td>
+<td> setMaxUncommittedOffsets                               </td>
+<td> line 108, id: kafkaConfig           </td>
+<td> n/a                             </td>
+<td> Kafka consumer client property         </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> n/a                                                    </td>
+<td> setOffsetCommitPeriodMs                                </td>
+<td> line 108, id: kafkaConfig           </td>
+<td> n/a                             </td>
+<td> Kafka consumer client property         </td></tr>
+<tr class="a">
+<td> Output bolt                 </td>
+<td> hdfs_writer_parallelism                    </td>
+<td> hdfs.writer.parallelism                                </td>
+<td> parallelism                                            </td>
+<td> line 133, id: hdfsIndexingBolt      </td>
+<td> n/a                             </td>
+<td>                                        </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> n/a                                        </td>
+<td> n/a                                                    </td>
+<td> hdfsSyncPolicy (see notes below)                       </td>
+<td> line 47, id: hdfsWriter             </td>
+<td> n/a                             </td>
+<td> See notes below about adding this prop </td></tr>
+<tr class="a">
+<td>                             </td>
+<td> bolt_hdfs_rotation_policy_units            </td>
+<td> bolt.hdfs.rotation.policy.units                        </td>
+<td> constructorArgs                                        </td>
+<td> line 41, id: hdfsRotationPolicy     </td>
+<td> n/a                             </td>
+<td>                                        </td></tr>
+<tr class="b">
+<td>                             </td>
+<td> bolt_hdfs_rotation_policy_count            </td>
+<td> bolt.hdfs.rotation.policy.count                        </td>
+<td> constructorArgs                                        </td>
+<td> line 41, id: hdfsRotationPolicy     </td>
+<td> n/a                             </td>
+<td>                                        </td></tr>
+</tbody>
+</table>
+<p><i>Note</i>: HDFS sync policy is not currently managed via Ambari. You will need to modify the Flux file directly to accommodate this setting. e.g.</p>
+<p>Add a new setter to the hdfsWriter around line 56. Lines 53-55 provided for context.</p>
+
+<div>
+<div>
+<pre class="source"> 53             -   name: &quot;withRotationPolicy&quot;
+ 54                 args:
+ 55                     - ref: &quot;hdfsRotationPolicy
+ 56             -   name: &quot;withSyncPolicy&quot;
+ 57                 args:
+ 58                     - ref: &quot;hdfsSyncPolicy
+</pre></div></div>
+
+<p>Add an hdfsSyncPolicy after the hdfsRotationPolicy that appears on line 41. e.g.</p>
+
+<div>
+<div>
+<pre class="source"> 41     -   id: &quot;hdfsRotationPolicy&quot;
+...
+ 45           - &quot;${bolt.hdfs.rotation.policy.units}&quot;
+ 46
+ 47     -   id: &quot;hdfsSyncPolicy&quot;
+ 48         className: &quot;org.apache.storm.hdfs.bolt.sync.CountSyncPolicy&quot;
+ 49         constructorArgs:
+ 50           -  100000
+</pre></div></div>
+</div></div></div>
 <div class="section">
 <h2><a name="Use_Case_Specific_Tuning_Suggestions"></a>Use Case Specific Tuning Suggestions</h2>
 <p>The below discussion outlines a specific tuning exercise we went through for driving 1 Gbps of traffic through a Metron cluster running with 4 Kafka brokers and 4 Storm Supervisors.</p>
 <p>General machine specs</p>
-
 <ul>
-  
+
 <li>10 Gb network cards</li>
-  
 <li>256 GB memory</li>
-  
 <li>12 disks</li>
-  
 <li>32 cores</li>
 </ul>
 <div class="section">
@@ -386,74 +679,75 @@
 <div class="section">
 <h4><a name="Tooling"></a>Tooling</h4>
 <p>Kafka</p>
-
 <ul>
-  
+
 <li>consumer group offset lag viewer</li>
-  
 <li>There is a GUI tool to make creating, modifying, and generally managing your Kafka topics a bit easier - see <a class="externalLink" href="https://github.com/yahoo/kafka-manager">https://github.com/yahoo/kafka-manager</a></li>
-  
 <li>console consumer - useful for quickly verifying topic contents</li>
 </ul>
 <p>Storm</p>
-
 <ul>
-  
+
 <li>Storm UI - <a class="externalLink" href="http://www.malinga.me/reading-and-understanding-the-storm-ui-storm-ui-explained/">http://www.malinga.me/reading-and-understanding-the-storm-ui-storm-ui-explained/</a></li>
 </ul></div>
 <div class="section">
 <h4><a name="Example_-_Viewing_Kafka_Offset_Lags"></a>Example - Viewing Kafka Offset Lags</h4>
 <p>First we need to setup some environment variables</p>
 
-<div class="source">
-<div class="source">
-<pre>export BROKERLIST=&lt;your broker comma-delimated list of host:ports&gt;
+<div>
+<div>
+<pre class="source">export BROKERLIST=&lt;your broker comma-delimated list of host:ports&gt;
 export ZOOKEEPER=&lt;your zookeeper comma-delimated list of host:ports&gt;
 export KAFKA_HOME=&lt;kafka home dir&gt;
 export METRON_HOME=&lt;your metron home&gt;
 export HDP_HOME=&lt;your HDP home&gt;
 </pre></div></div>
+
 <p>If you have Kerberos enabled, setup the security protocol</p>
 
-<div class="source">
-<div class="source">
-<pre>$ cat /tmp/consumergroup.config
+<div>
+<div>
+<pre class="source">$ cat /tmp/consumergroup.config
 security.protocol=SASL_PLAINTEXT
 </pre></div></div>
+
 <p>Now run the following command for a running topology&#x2019;s consumer group. In this example we are using enrichments.</p>
 
-<div class="source">
-<div class="source">
-<pre>${KAFKA_HOME}/bin/kafka-consumer-groups.sh \
+<div>
+<div>
+<pre class="source">${KAFKA_HOME}/bin/kafka-consumer-groups.sh \
     --command-config=/tmp/consumergroup.config \
     --describe \
     --group enrichments \
     --bootstrap-server $BROKERLIST \
     --new-consumer
 </pre></div></div>
+
 <p>This will return a table with the following output depicting offsets for all partitions and consumers associated with the specified consumer group:</p>
 
-<div class="source">
-<div class="source">
-<pre>GROUP                          TOPIC              PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             OWNER
+<div>
+<div>
+<pre class="source">GROUP                          TOPIC              PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             OWNER
 enrichments                    enrichments        9          29746066        29746067        1               consumer-2_/xxx.xxx.xxx.xxx
 enrichments                    enrichments        3          29754325        29754326        1               consumer-1_/xxx.xxx.xxx.xxx
 enrichments                    enrichments        43         29754331        29754332        1               consumer-6_/xxx.xxx.xxx.xxx
 ...
 </pre></div></div>
+
 <p><i>Note</i>: You won&#x2019;t see any output until a topology is actually running because the consumer groups only exist while consumers in the spouts are up and running.</p>
 <p>The primary column we&#x2019;re concerned with paying attention to is the LAG column, which is the current delta calculation between the current and end offset for the partition. This tells us how close we are to keeping up with incoming data. And, as we found through multiple trials, whether there are any problems with specific consumers getting stuck.</p>
 <p>Taking this one step further, it&#x2019;s probably more useful if we can watch the offsets and lags change over time. In order to do this we&#x2019;ll add a &#x201c;watch&#x201d; command and set the refresh rate to 10 seconds.</p>
 
-<div class="source">
-<div class="source">
-<pre>watch -n 10 -d ${KAFKA_HOME}/bin/kafka-consumer-groups.sh \
+<div>
+<div>
+<pre class="source">watch -n 10 -d ${KAFKA_HOME}/bin/kafka-consumer-groups.sh \
     --command-config=/tmp/consumergroup.config \
     --describe \
     --group enrichments \
     --bootstrap-server $BROKERLIST \
     --new-consumer
 </pre></div></div>
+
 <p>Every 10 seconds the command will re-run and the screen will be refreshed with new information. The most useful bit is that the watch command will highlight the differences from the current output and the last output screens.</p></div></div>
 <div class="section">
 <h3><a name="Parser_Tuning"></a>Parser Tuning</h3>
@@ -461,31 +755,33 @@ enrichments                    enrichments        43         29754331        297
 <p>We started with a single partition for the inbound Kafka topics and eventually worked our way up to 48. And We&#x2019;re using the following pending value, as shown below. The default is &#x2018;null&#x2019; which would result in no limit.</p>
 <p><b>storm-bro.config</b></p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
     ...
     &quot;topology.max.spout.pending&quot; : 2000
     ...
 }
 </pre></div></div>
+
 <p>And the following default spout settings. Again, this can be ommitted entirely since we are using the defaults.</p>
 <p><b>spout-bro.config</b></p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
     ...
     &quot;spout.pollTimeoutMs&quot; : 200,
     &quot;spout.maxUncommittedOffsets&quot; : 10000000,
     &quot;spout.offsetCommitPeriodMs&quot; : 30000
 }
 </pre></div></div>
+
 <p>And we ran our bro parser topology with the following options. We did not need to fully match the number of Kafka partitions with our parallelism in this case, though you could certainly do so if necessary. Notice that we only needed 1 worker.</p>
 
-<div class="source">
-<div class="source">
-<pre>/usr/metron/0.4.2/bin/start_parser_topology.sh \
+<div>
+<div>
+<pre class="source">/usr/metron/0.5.0/bin/start_parser_topology.sh \
     -e ~metron/.storm/storm-bro.config \
     -esc ~/.storm/spout-bro.config \
     -k $BROKERLIST \
@@ -499,11 +795,12 @@ enrichments                    enrichments        43         29754331        297
     -sp 24 \
     -z $ZOOKEEPER \
 </pre></div></div>
+
 <p>From the usage docs, here are the options we&#x2019;ve used. The full reference can be found <a href="../metron-platform/metron-parsers/index.html#Starting_the_Parser_Topology">here</a>.</p>
 
-<div class="source">
-<div class="source">
-<pre>usage: start_parser_topology.sh
+<div>
+<div>
+<pre class="source">usage: start_parser_topology.sh
  -e,--extra_topology_options &lt;JSON_FILE&gt;               Extra options in the form
                                                        of a JSON file with a map
                                                        for content.
@@ -529,24 +826,26 @@ enrichments                    enrichments        43         29754331        297
  -sp,--spout_p &lt;SPOUT_PARALLELISM_HINT&gt;                Spout Parallelism Hint
  -z,--zk &lt;ZK_QUORUM&gt;                                   Zookeeper Quroum URL
                                                        (zk1:2181,zk2:2181,...
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Enrichment_Tuning"></a>Enrichment Tuning</h3>
 <p>We landed on the same number of partitions for enrichemnt and indexing as we did for bro - 48.</p>
 <p>For configuring Storm, there is a flux file and properties file that we modified. Here are the settings we changed for bro in Flux. Note that the main Metron-specific option we&#x2019;ve changed to accomodate the desired rate of data throughput is max cache size in the join bolts. More information on Flux can be found here - <a class="externalLink" href="http://storm.apache.org/releases/1.0.1/flux.html">http://storm.apache.org/releases/1.0.1/flux.html</a></p>
 <p><b>General storm settings</b></p>
 
-<div class="source">
-<div class="source">
-<pre>topology.workers: 8
+<div>
+<div>
+<pre class="source">topology.workers: 8
 topology.acker.executors: 48
 topology.max.spout.pending: 2000
 </pre></div></div>
+
 <p><b>Spout and Bolt Settings</b></p>
 
-<div class="source">
-<div class="source">
-<pre>kafkaSpout
+<div>
+<div>
+<pre class="source">kafkaSpout
     parallelism=48
     session.timeout.ms=29999
     enable.auto.commit=false
@@ -567,15 +866,16 @@ threatIntelJoinBolt
     withMaxTimeRetain=10
 outputBolt
     parallelism=48
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
-<h3><a name="Indexing_HDFS_Tuning"></a>Indexing (HDFS) Tuning</h3>
+<h3><a name="Indexing_.28HDFS.29_Tuning"></a>Indexing (HDFS) Tuning</h3>
 <p>There are 48 partitions set for the indexing partition, per the enrichment exercise above.</p>
 <p>These are the batch size settings for the bro index</p>
 
-<div class="source">
-<div class="source">
-<pre>cat ${METRON_HOME}/config/zookeeper/indexing/bro.json
+<div>
+<div>
+<pre class="source">cat ${METRON_HOME}/config/zookeeper/indexing/bro.json
 {
   &quot;hdfs&quot; : {
     &quot;index&quot;: &quot;bro&quot;,
@@ -584,20 +884,22 @@ outputBolt
   }...
 }
 </pre></div></div>
+
 <p>And here are the settings we used for the indexing topology</p>
 <p><b>General storm settings</b></p>
 
-<div class="source">
-<div class="source">
-<pre>topology.workers: 4
+<div>
+<div>
+<pre class="source">topology.workers: 4
 topology.acker.executors: 24
 topology.max.spout.pending: 2000
 </pre></div></div>
+
 <p><b>Spout and Bolt Settings</b></p>
 
-<div class="source">
-<div class="source">
-<pre>hdfsSyncPolicy
+<div>
+<div>
+<pre class="source">hdfsSyncPolicy
     org.apache.storm.hdfs.bolt.sync.CountSyncPolicy
     constructor arg=100000
 hdfsRotationPolicy
@@ -612,22 +914,24 @@ kafkaSpout
     setOffsetCommitPeriodMs=30000
 hdfsIndexingBolt
     parallelism: 24
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="PCAP_Tuning"></a>PCAP Tuning</h3>
 <p>PCAP is a specialized topology that is a Spout-only topology. Both Kafka topic consumption and HDFS writing is done within a spout to avoid the additional network hop required if using an additional bolt.</p>
 <p><b>General Storm topology properties</b></p>
 
-<div class="source">
-<div class="source">
-<pre>topology.workers=16
+<div>
+<div>
+<pre class="source">topology.workers=16
 topology.ackers.executors: 0
 </pre></div></div>
+
 <p><b>Spout and Bolt properties</b></p>
 
-<div class="source">
-<div class="source">
-<pre>kafkaSpout
+<div>
+<div>
+<pre class="source">kafkaSpout
     parallelism: 128
     poll.timeout.ms=100
     offset.commit.period.ms=30000
@@ -649,54 +953,117 @@ writerConfig
     withHDFSConfig
         io.file.buffer.size=1000000
         dfs.blocksize=1073741824
-</pre></div></div></div></div>
+</pre></div></div>
+</div></div>
+<div class="section">
+<h2><a name="Debugging"></a>Debugging</h2>
+<p>Set the following env vars accordingly for your cluster. This is how we would configure it for the Metron full dev development environment.</p>
+
+<div>
+<div>
+<pre class="source">export HDP_HOME=/usr/hdp/current
+export KAFKA_HOME=$HDP_HOME/kafka-broker
+export STORM_UI=http://node1:8744
+export ELASTIC=http://node1:9200
+export ZOOKEEPER=node1:2181
+export METRON_VERSION=0.5.0
+export METRON_HOME=/usr/metron/${METRON_VERSION}
+</pre></div></div>
+
+<p>Note that the output from Storm will be a flattened blob of JSON. In order to pretty print for readability, you can pipe it through a JSON formatter, e.g.</p>
+
+<div>
+<div>
+<pre class="source">[some Storm curl command] | python -m json.tool
+</pre></div></div>
+
+<p><b>Getting Storm Configuration Details</b></p>
+<p>Storm has a useful REST API you can use to get full details about your running topologies. This is generally more convenient and complete for troubleshooting performance problems than going to the Storm UI alone. See Storm&#x2019;s <a class="externalLink" href="http://storm.apache.org/releases/1.1.0/STORM-UI-REST-API.html">REST API docs</a> for more details.</p>
+
+<div>
+<div>
+<pre class="source"># get Storm cluster summary info including version
+curl -XGET ${STORM_UI}'/api/v1/cluster/summary'
+</pre></div></div>
+
+<div>
+<div>
+<pre class="source"># get overall Storm cluster configuration
+curl -XGET ${STORM_UI}'/api/v1/cluster/configuration'
+</pre></div></div>
+
+<div>
+<div>
+<pre class="source"># get list of topologies and brief summary detail
+curl -XGET ${STORM_UI}'/api/v1/topology/summary'
+</pre></div></div>
+
+<div>
+<div>
+<pre class="source"># get all topology runtime settings. Plugin the ID for your topology, which you can get from the topology summary command or from the Storm UI. Passing sys=1 will also return system stats.
+curl -XGET ${STORM_UI}'/api/v1/topology/:id?sys=1&#x200b;'
+</pre></div></div>
+
+<p><b>Getting Kafka Configuration Details</b></p>
+
+<div>
+<div>
+<pre class="source"># Get list of Kafka topics
+${HDP_HOME}/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --list
+</pre></div></div>
+
+<div>
+<div>
+<pre class="source"># Get Kafka topic details - plugin the desired topic name in place of &quot;enrichments&quot;
+${HDP_HOME}/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --topic enrichments --describe
+</pre></div></div>
+
+<p><b>Getting Metron Topology Zookeeper Configuration</b></p>
+
+<div>
+<div>
+<pre class="source"># Provides a full listing of all Metron parser, enrichment, and indexing topology configuration
+$METRON_HOME/bin/zk_load_configs.sh -m DUMP -z $ZOOKEEPER
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Issues"></a>Issues</h2>
 <p><b>Error</b></p>
 
-<div class="source">
-<div class="source">
-<pre>org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned
+<div>
+<div>
+<pre class="source">org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned
 the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms,
 which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the
 session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records
 </pre></div></div>
+
 <p><b>Suggestions</b></p>
 <p>This implies that the spout hasn&#x2019;t been given enough time between polls before committing the offsets. In other words, the amount of time taken to process the messages is greater than the timeout window. In order to fix this, you can improve message throughput by modifying the options outlined above, increasing the poll timeout, or both.</p></div>
 <div class="section">
 <h2><a name="Reference"></a>Reference</h2>
-
 <ul>
-  
-<li><a class="externalLink" href="http://storm.apache.org/releases/1.0.1/flux.html">http://storm.apache.org/releases/1.0.1/flux.html</a></li>
-  
+
+<li><a href="metron-enrichment/Performance.html">Enrichment Performance</a></li>
+<li><a class="externalLink" href="http://storm.apache.org/releases/1.1.0/flux.html">http://storm.apache.org/releases/1.1.0/flux.html</a></li>
 <li><a class="externalLink" href="https://stackoverflow.com/questions/17257448/what-is-the-task-in-storm-parallelism">https://stackoverflow.com/questions/17257448/what-is-the-task-in-storm-parallelism</a></li>
-  
 <li><a class="externalLink" href="http://storm.apache.org/releases/current/Understanding-the-parallelism-of-a-Storm-topology.html">http://storm.apache.org/releases/current/Understanding-the-parallelism-of-a-Storm-topology.html</a></li>
-  
 <li><a class="externalLink" href="http://www.malinga.me/reading-and-understanding-the-storm-ui-storm-ui-explained/">http://www.malinga.me/reading-and-understanding-the-storm-ui-storm-ui-explained/</a></li>
-  
 <li><a class="externalLink" href="https://www.confluent.io/blog/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/">https://www.confluent.io/blog/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/</a></li>
-  
 <li><a class="externalLink" href="https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_storm-component-guide/content/storm-kafkaspout-perf.html">https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_storm-component-guide/content/storm-kafkaspout-perf.html</a></li>
+<li><a class="externalLink" href="http://storm.apache.org/releases/1.1.0/STORM-UI-REST-API.html">http://storm.apache.org/releases/1.1.0/STORM-UI-REST-API.html</a></li>
 </ul></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/index.html b/site/current-book/metron-platform/index.html
index ee219ed..7819640 100644
--- a/site/current-book/metron-platform/index.html
+++ b/site/current-book/metron-platform/index.html
@@ -1,271 +1,93 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Current Build</title>
-    <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../css/site.css" />
     <link rel="stylesheet" href="../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Current Build</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Current Build</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                        
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Platform</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <!-- Licensed to the Apache Software Foundation (ASF) under one or more
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
@@ -278,33 +100,29 @@ Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
-limitations under the License. --><h1>Current Build</h1>
+limitations under the License.
+ -->
+<h1>Current Build</h1>
 <p><a name="Current_Build"></a></p>
-<p>The latest build of metron-platform is 0.4.2.</p>
+<p>The latest build of metron-platform is 0.5.0.</p>
 <p>We are still in the process of merging/porting additional features from our production code base into this open source release. This release will be followed by a number of additional beta releases until the port is complete. We will also work on getting additional documentation and user/developer guides to the community as soon as we can. At this time we offer no support for the beta software, but will try to respond to requests as promptly as we can.</p>
 <p><a name="metron-platform"></a></p>
 <h1>metron-platform</h1>
-<p>Extensible set of Storm topologies and topology attributes for streaming, enriching, indexing, and storing telemetry in Hadoop. General information on Metron is available at <a class="externalLink" href="https://metron.apache.org/">https://metron.apache.org/</a></p>
+<p>Extensible set of Storm topologies and topology attributes for streaming, enriching, indexing, and storing telemetry in Hadoop.  General information on Metron is available at <a class="externalLink" href="https://metron.apache.org/">https://metron.apache.org/</a></p>
 <p><a name="Documentation"></a></p>
 <h1>Documentation</h1>
 <p>Please see documentation within each individual module for description and usage instructions. Sample topologies are provided under Metron_Topologies to get you started with the framework. We pre-assume knowledge of Hadoop, Storm, Kafka, and HBase.</p>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[04/50] [abbrv] metron git commit: METRON-1594: KafkaWriter is asynchronous and may lose data on node failure (mmiklavc via mmiklavc) closes apache/metron#1045

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-indexing/src/main/flux/indexing/batch/remote.yaml
----------------------------------------------------------------------
diff --git a/metron-platform/metron-indexing/src/main/flux/indexing/batch/remote.yaml b/metron-platform/metron-indexing/src/main/flux/indexing/batch/remote.yaml
index 3087dd9..919654c 100644
--- a/metron-platform/metron-indexing/src/main/flux/indexing/batch/remote.yaml
+++ b/metron-platform/metron-indexing/src/main/flux/indexing/batch/remote.yaml
@@ -134,6 +134,7 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "INDEXING"
         configMethods:
             -   name: "withBulkMessageWriter"
                 args:
@@ -147,8 +148,9 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "INDEXING"
         configMethods:
-            -   name: "withMessageWriter"
+            -   name: "withBulkMessageWriter"
                 args:
                     - ref: "kafkaWriter"
 

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-indexing/src/main/flux/indexing/random_access/remote.yaml
----------------------------------------------------------------------
diff --git a/metron-platform/metron-indexing/src/main/flux/indexing/random_access/remote.yaml b/metron-platform/metron-indexing/src/main/flux/indexing/random_access/remote.yaml
index 429ba45..9aa2b24 100644
--- a/metron-platform/metron-indexing/src/main/flux/indexing/random_access/remote.yaml
+++ b/metron-platform/metron-indexing/src/main/flux/indexing/random_access/remote.yaml
@@ -106,6 +106,7 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "INDEXING"
         configMethods:
             -   name: "withBulkMessageWriter"
                 args:
@@ -119,8 +120,9 @@ bolts:
         className: "org.apache.metron.writer.bolt.BulkMessageWriterBolt"
         constructorArgs:
             - "${kafka.zk}"
+            - "INDEXING"
         configMethods:
-            -   name: "withMessageWriter"
+            -   name: "withBulkMessageWriter"
                 args:
                     - ref: "kafkaWriter"
 

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-parsers/README.md
----------------------------------------------------------------------
diff --git a/metron-platform/metron-parsers/README.md b/metron-platform/metron-parsers/README.md
index e8b2896..8254baf 100644
--- a/metron-platform/metron-parsers/README.md
+++ b/metron-platform/metron-parsers/README.md
@@ -161,7 +161,11 @@ Example Stellar Filter which includes messages which contain a the `field1` fiel
 then it is assumed to be a regex and will match any topic matching the pattern (e.g. `/bro.*/` would match `bro_cust0`, `bro_cust1` and `bro_cust2`)
 * `readMetadata` : Boolean indicating whether to read metadata or not (`false` by default).  See below for a discussion about metadata.
 * `mergeMetadata` : Boolean indicating whether to merge metadata with the message or not (`false` by default).  See below for a discussion about metadata.
-* `parserConfig` : A JSON Map representing the parser implementation specific configuration.
+* `parserConfig` : A JSON Map representing the parser implementation specific configuration. Also include batch sizing and timeout for writer configuration here.
+  * `batchSize` : Integer indicating number of records to batch together before sending to the writer. (default to `15`)
+  * `batchTimeout` : The timeout after which a batch will be flushed even if batchSize has not been met.  Optional.
+    If unspecified, or set to `0`, it defaults to a system-determined duration which is a fraction of the Storm
+    parameter `topology.message.timeout.secs`.  Ignored if batchSize is `1`, since this disables batching.
 * `fieldTransformations` : An array of complex objects representing the transformations to be done on the message generated from the parser before writing out to the kafka topic.
 * `spoutParallelism` : The kafka spout parallelism (default to `1`).  This can be overridden on the command line.
 * `spoutNumTasks` : The number of tasks for the spout (default to `1`). This can be overridden on the command line.

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java b/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java
index dd59355..0e9b48f 100644
--- a/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java
+++ b/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java
@@ -19,6 +19,7 @@ package org.apache.metron.parsers.bolt;
 
 import static org.apache.metron.common.Constants.METADATA_PREFIX;
 
+import com.github.benmanes.caffeine.cache.Cache;
 import java.io.IOException;
 import java.io.Serializable;
 import java.lang.invoke.MethodHandles;
@@ -30,15 +31,15 @@ import java.util.Map;
 import java.util.Optional;
 import java.util.Set;
 import java.util.UUID;
+import java.util.function.Function;
 import java.util.stream.Collectors;
-
-import com.github.benmanes.caffeine.cache.Cache;
 import org.apache.commons.lang3.StringUtils;
 import org.apache.metron.common.Constants;
 import org.apache.metron.common.bolt.ConfiguredParserBolt;
 import org.apache.metron.common.configuration.FieldTransformer;
 import org.apache.metron.common.configuration.FieldValidator;
 import org.apache.metron.common.configuration.SensorParserConfig;
+import org.apache.metron.common.configuration.writer.WriterConfiguration;
 import org.apache.metron.common.error.MetronError;
 import org.apache.metron.common.message.MessageGetStrategy;
 import org.apache.metron.common.message.MessageGetters;
@@ -50,11 +51,15 @@ import org.apache.metron.parsers.interfaces.MessageParser;
 import org.apache.metron.stellar.common.CachingStellarProcessor;
 import org.apache.metron.stellar.dsl.Context;
 import org.apache.metron.stellar.dsl.StellarFunctions;
+import org.apache.metron.writer.WriterToBulkWriter;
+import org.apache.metron.writer.bolt.BatchTimeoutHelper;
+import org.apache.storm.Config;
 import org.apache.storm.task.OutputCollector;
 import org.apache.storm.task.TopologyContext;
 import org.apache.storm.topology.OutputFieldsDeclarer;
 import org.apache.storm.tuple.Fields;
 import org.apache.storm.tuple.Tuple;
+import org.apache.storm.utils.TupleUtils;
 import org.json.simple.JSONObject;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -71,6 +76,10 @@ public class ParserBolt extends ConfiguredParserBolt implements Serializable {
   private Context stellarContext;
   private transient MessageGetStrategy messageGetStrategy;
   private transient Cache<CachingStellarProcessor.Key, Object> cache;
+  private int requestedTickFreqSecs;
+  private int defaultBatchTimeout;
+  private int batchTimeoutDivisor = 1;
+
   public ParserBolt( String zookeeperUrl
                    , String sensorType
                    , MessageParser<JSONObject> parser
@@ -88,10 +97,85 @@ public class ParserBolt extends ConfiguredParserBolt implements Serializable {
     return this;
   }
 
+  /**
+   * If this ParserBolt is in a topology where it is daisy-chained with
+   * other queuing Writers, then the max amount of time it takes for a tuple
+   * to clear the whole topology is the sum of all the batchTimeouts for all the
+   * daisy-chained Writers.  In the common case where each Writer is using the default
+   * batchTimeout, it is then necessary to divide that batchTimeout by the number of
+   * daisy-chained Writers.  There are no examples of daisy-chained batching Writers
+   * in the current Metron topologies, but the feature is available as a "fluent"-style
+   * mutator if needed.  It would be used in the parser topology builder.
+   * Default value, if not otherwise set, is 1.
+   *
+   * If non-default batchTimeouts are configured for some components, the administrator
+   * may want to take this behavior into account.
+   *
+   * @param batchTimeoutDivisor
+   * @return BulkMessageWriterBolt
+   */
+  public ParserBolt withBatchTimeoutDivisor(int batchTimeoutDivisor) {
+    if (batchTimeoutDivisor <= 0) {
+      throw new IllegalArgumentException(String.format("batchTimeoutDivisor must be positive. Value provided was %s", batchTimeoutDivisor));
+    }
+    this.batchTimeoutDivisor = batchTimeoutDivisor;
+    return this;
+  }
+
+  /**
+   * Used only for unit testing
+   * @param defaultBatchTimeout
+   */
+  protected void setDefaultBatchTimeout(int defaultBatchTimeout) {
+    this.defaultBatchTimeout = defaultBatchTimeout;
+  }
+
+  /**
+   * Used only for unit testing
+   */
+  public int getDefaultBatchTimeout() {
+    return defaultBatchTimeout;
+  }
+
   public MessageParser<JSONObject> getParser() {
     return parser;
   }
 
+  /**
+   * This method is called by TopologyBuilder.createTopology() to obtain topology and
+   * bolt specific configuration parameters.  We use it primarily to configure how often
+   * a tick tuple will be sent to our bolt.
+   * @return conf topology and bolt specific configuration parameters
+   */
+  @Override
+  public Map<String, Object> getComponentConfiguration() {
+    // This is called long before prepare(), so do some of the same stuff as prepare() does,
+    // to get the valid WriterConfiguration.  But don't store any non-serializable objects,
+    // else Storm will throw a runtime error.
+    Function<WriterConfiguration, WriterConfiguration> configurationXform;
+    if(writer.isWriterToBulkWriter()) {
+      configurationXform = WriterToBulkWriter.TRANSFORMATION;
+    }
+    else {
+      configurationXform = x -> x;
+    }
+    WriterConfiguration writerconf = configurationXform
+        .apply(getConfigurationStrategy().createWriterConfig(writer.getBulkMessageWriter(), getConfigurations()));
+
+    BatchTimeoutHelper timeoutHelper = new BatchTimeoutHelper(writerconf::getAllConfiguredTimeouts, batchTimeoutDivisor);
+    this.requestedTickFreqSecs = timeoutHelper.getRecommendedTickInterval();
+    //And while we've got BatchTimeoutHelper handy, capture the defaultBatchTimeout for writerComponent.
+    this.defaultBatchTimeout = timeoutHelper.getDefaultBatchTimeout();
+
+    Map<String, Object> conf = super.getComponentConfiguration();
+    if (conf == null) {
+      conf = new HashMap<String, Object>();
+    }
+    conf.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, requestedTickFreqSecs);
+    LOG.info("Requesting " + Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS + " set to " + Integer.toString(requestedTickFreqSecs));
+    return conf;
+  }
+
   @SuppressWarnings("unchecked")
   @Override
   public void prepare(Map stormConf, TopologyContext context, OutputCollector collector) {
@@ -114,6 +198,15 @@ public class ParserBolt extends ConfiguredParserBolt implements Serializable {
     parser.init();
 
     writer.init(stormConf, context, collector, getConfigurations());
+    if (defaultBatchTimeout == 0) {
+      //This means getComponentConfiguration was never called to initialize defaultBatchTimeout,
+      //probably because we are in a unit test scenario.  So calculate it here.
+      WriterConfiguration writerConfig = getConfigurationStrategy()
+          .createWriterConfig(writer.getBulkMessageWriter(), getConfigurations());
+      BatchTimeoutHelper timeoutHelper = new BatchTimeoutHelper(writerConfig::getAllConfiguredTimeouts, batchTimeoutDivisor);
+      defaultBatchTimeout = timeoutHelper.getDefaultBatchTimeout();
+    }
+    writer.setDefaultBatchTimeout(defaultBatchTimeout);
 
     SensorParserConfig config = getSensorParserConfig();
     if(config != null) {
@@ -173,6 +266,17 @@ public class ParserBolt extends ConfiguredParserBolt implements Serializable {
   @SuppressWarnings("unchecked")
   @Override
   public void execute(Tuple tuple) {
+    if (TupleUtils.isTick(tuple)) {
+      try {
+        writer.flush(getConfigurations(), messageGetStrategy);
+      } catch (Exception e) {
+        throw new RuntimeException(
+            "This should have been caught in the writerHandler.  If you see this, file a JIRA", e);
+      } finally {
+        collector.ack(tuple);
+      }
+      return;
+    }
     byte[] originalMessage = (byte[]) messageGetStrategy.get(tuple);
     SensorParserConfig sensorParserConfig = getSensorParserConfig();
     try {

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/WriterHandler.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/WriterHandler.java b/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/WriterHandler.java
index 2192942..3916dea 100644
--- a/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/WriterHandler.java
+++ b/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/WriterHandler.java
@@ -18,31 +18,37 @@
 
 package org.apache.metron.parsers.bolt;
 
-import org.apache.metron.common.message.MessageGetStrategy;
-import org.apache.storm.task.OutputCollector;
-import org.apache.storm.task.TopologyContext;
-import org.apache.storm.tuple.Tuple;
+import java.io.Serializable;
+import java.lang.invoke.MethodHandles;
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.function.Function;
 import org.apache.metron.common.configuration.ParserConfigurations;
-import org.apache.metron.common.configuration.writer.ParserWriterConfiguration;
+import org.apache.metron.common.configuration.writer.ConfigurationStrategy;
+import org.apache.metron.common.configuration.writer.ConfigurationsStrategies;
 import org.apache.metron.common.configuration.writer.SingleBatchConfigurationFacade;
 import org.apache.metron.common.configuration.writer.WriterConfiguration;
+import org.apache.metron.common.message.MessageGetStrategy;
 import org.apache.metron.common.writer.BulkMessageWriter;
 import org.apache.metron.common.writer.MessageWriter;
 import org.apache.metron.writer.BulkWriterComponent;
 import org.apache.metron.writer.WriterToBulkWriter;
+import org.apache.storm.task.OutputCollector;
+import org.apache.storm.task.TopologyContext;
+import org.apache.storm.tuple.Tuple;
 import org.json.simple.JSONObject;
-
-import java.io.Serializable;
-import java.util.Collection;
-import java.util.HashSet;
-import java.util.Map;
-import java.util.function.Function;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 public class WriterHandler implements Serializable {
+  private static final Logger LOG = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
   private BulkMessageWriter<JSONObject> messageWriter;
   private transient BulkWriterComponent<JSONObject> writerComponent;
   private transient Function<ParserConfigurations, WriterConfiguration> writerTransformer;
   private boolean isBulk = false;
+  private ConfigurationStrategy configStrategy = ConfigurationsStrategies.PARSERS;
+
   public WriterHandler(MessageWriter<JSONObject> writer) {
     isBulk = false;
     messageWriter = new WriterToBulkWriter<>(writer);
@@ -53,17 +59,24 @@ public class WriterHandler implements Serializable {
     messageWriter = writer;
   }
 
-
   public boolean handleAck() {
     return isBulk;
   }
 
+  public boolean isWriterToBulkWriter() {
+    return messageWriter instanceof  WriterToBulkWriter;
+  }
+
+  public BulkMessageWriter getBulkMessageWriter() {
+    return messageWriter;
+  }
+
   public void init(Map stormConf, TopologyContext topologyContext, OutputCollector collector, ParserConfigurations configurations) {
     if(isBulk) {
-      writerTransformer = config -> new ParserWriterConfiguration(config);
+      writerTransformer = config -> configStrategy.createWriterConfig(messageWriter, config);
     }
     else {
-      writerTransformer = config -> new SingleBatchConfigurationFacade(new ParserWriterConfiguration(config));
+      writerTransformer = config -> new SingleBatchConfigurationFacade(configStrategy.createWriterConfig(messageWriter, config));
     }
     try {
       messageWriter.init(stormConf, topologyContext, writerTransformer.apply(configurations));
@@ -87,7 +100,29 @@ public class WriterHandler implements Serializable {
     writerComponent.write(sensorType, tuple, message, messageWriter, writerTransformer.apply(configurations), messageGetStrategy);
   }
 
+  public void flush(ParserConfigurations configurations, MessageGetStrategy messageGetStrategy)
+      throws Exception {
+    if (!(messageWriter instanceof WriterToBulkWriter)) {
+      //WriterToBulkWriter doesn't allow batching, so no need to flush on Tick.
+      LOG.debug("Flushing message queues older than their batchTimeouts");
+      writerComponent.flushTimeouts(messageWriter, writerTransformer.apply(configurations),
+          messageGetStrategy);
+    }
+  }
+
   public void errorAll(String sensorType, Throwable e, MessageGetStrategy messageGetStrategy) {
     writerComponent.errorAll(sensorType, e, messageGetStrategy);
   }
+
+  /**
+   * Sets batch timeout on the underlying component
+   * @param defaultBatchTimeout
+   */
+  public void setDefaultBatchTimeout(int defaultBatchTimeout) {
+    if (writerComponent == null) {
+      throw new UnsupportedOperationException("Must call init prior to calling this method.");
+    }
+    writerComponent.setDefaultBatchTimeout(defaultBatchTimeout);
+  }
+
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/bolt/ParserBoltTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/bolt/ParserBoltTest.java b/metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/bolt/ParserBoltTest.java
index 6439b2b..15ce735 100644
--- a/metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/bolt/ParserBoltTest.java
+++ b/metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/bolt/ParserBoltTest.java
@@ -17,46 +17,59 @@
  */
 package org.apache.metron.parsers.bolt;
 
-import org.apache.curator.framework.CuratorFramework;
-import org.apache.metron.common.Constants;
-import org.apache.metron.common.configuration.*;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.eq;
+import static org.mockito.Mockito.argThat;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
 
-import org.apache.metron.common.error.MetronError;
-import org.apache.metron.common.zookeeper.configurations.ConfigurationsUpdater;
-import org.apache.metron.test.error.MetronErrorJSONMatcher;
-import org.apache.storm.task.OutputCollector;
-import org.apache.storm.task.TopologyContext;
-import org.apache.storm.tuple.Tuple;
 import com.google.common.collect.ImmutableList;
-import org.apache.metron.common.configuration.writer.ParserWriterConfiguration;
-import org.apache.metron.common.configuration.writer.WriterConfiguration;
-import org.apache.metron.stellar.dsl.Context;
-import org.apache.metron.common.writer.BulkMessageWriter;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
 import org.adrianwalker.multilinestring.Multiline;
+import org.apache.curator.framework.CuratorFramework;
 import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.metron.common.Constants;
+import org.apache.metron.common.configuration.ConfigurationType;
+import org.apache.metron.common.configuration.IndexingConfigurations;
 import org.apache.metron.common.configuration.ParserConfigurations;
 import org.apache.metron.common.configuration.SensorParserConfig;
+import org.apache.metron.common.configuration.writer.ParserWriterConfiguration;
+import org.apache.metron.common.configuration.writer.WriterConfiguration;
+import org.apache.metron.common.error.MetronError;
+import org.apache.metron.common.writer.BulkMessageWriter;
 import org.apache.metron.common.writer.BulkWriterResponse;
+import org.apache.metron.common.writer.MessageWriter;
+import org.apache.metron.common.zookeeper.configurations.ConfigurationsUpdater;
 import org.apache.metron.parsers.BasicParser;
-import org.apache.metron.test.bolt.BaseBoltTest;
 import org.apache.metron.parsers.interfaces.MessageFilter;
 import org.apache.metron.parsers.interfaces.MessageParser;
-import org.apache.metron.common.writer.MessageWriter;
+import org.apache.metron.stellar.dsl.Context;
+import org.apache.metron.test.bolt.BaseBoltTest;
+import org.apache.metron.test.error.MetronErrorJSONMatcher;
+import org.apache.storm.task.OutputCollector;
+import org.apache.storm.task.TopologyContext;
+import org.apache.storm.tuple.Tuple;
 import org.json.simple.JSONObject;
 import org.json.simple.parser.JSONParser;
 import org.junit.Assert;
 import org.junit.Test;
 import org.mockito.Mock;
 
-import java.io.IOException;
-import java.util.*;
-import java.util.stream.Collectors;
-import java.util.stream.Stream;
-
-import static org.mockito.Matchers.any;
-import static org.mockito.Matchers.eq;
-import static org.mockito.Mockito.*;
-
 public class ParserBoltTest extends BaseBoltTest {
 
   @Mock
@@ -285,33 +298,6 @@ public class ParserBoltTest extends BaseBoltTest {
     parserBolt.execute(tuple);
     verify(outputCollector, times(1)).reportError(any(Throwable.class));
   }
-@Test
-public void testImplicitBatchOfOne() throws Exception {
-
-  String sensorType = "yaf";
-
-  ParserBolt parserBolt = new ParserBolt("zookeeperUrl", sensorType, parser, new WriterHandler(batchWriter)) {
-    @Override
-    protected ConfigurationsUpdater<ParserConfigurations> createUpdater() {
-      return ParserBoltTest.createUpdater();
-    }
-  };
-
-  parserBolt.setCuratorFramework(client);
-  parserBolt.setZKCache(cache);
-  parserBolt.prepare(new HashMap(), topologyContext, outputCollector);
-  verify(parser, times(1)).init();
-  verify(batchWriter, times(1)).init(any(), any(), any());
-  when(parser.validate(any())).thenReturn(true);
-  when(parser.parseOptional(any())).thenReturn(Optional.of(ImmutableList.of(new JSONObject())));
-  when(filter.emitTuple(any(), any(Context.class))).thenReturn(true);
-  BulkWriterResponse response = new BulkWriterResponse();
-  response.addSuccess(t1);
-  when(batchWriter.write(eq(sensorType), any(WriterConfiguration.class), eq(Collections.singleton(t1)), any())).thenReturn(response);
-  parserBolt.withMessageFilter(filter);
-  parserBolt.execute(t1);
-  verify(outputCollector, times(1)).ack(t1);
-}
 
   /**
    {
@@ -343,7 +329,7 @@ public void testImplicitBatchOfOne() throws Exception {
       }
       @Override
       protected ConfigurationsUpdater<ParserConfigurations> createUpdater() {
-        return ParserBoltTest.createUpdater();
+        return ParserBoltTest.createUpdater(Optional.of(1));
       }
     };
 
@@ -459,7 +445,7 @@ public void testImplicitBatchOfOne() throws Exception {
 
       @Override
       protected ConfigurationsUpdater<ParserConfigurations> createUpdater() {
-        return ParserBoltTest.createUpdater();
+        return ParserBoltTest.createUpdater(Optional.of(1));
       }
     };
 
@@ -474,13 +460,14 @@ public void testImplicitBatchOfOne() throws Exception {
   }
 
   @Test
-  public void testBatchOfOne() throws Exception {
+  public void testDefaultBatchSize() throws Exception {
 
     String sensorType = "yaf";
 
     ParserBolt parserBolt = new ParserBolt("zookeeperUrl", sensorType, parser, new WriterHandler(batchWriter)) {
       @Override
       protected ConfigurationsUpdater<ParserConfigurations> createUpdater() {
+        // this uses default batch size
         return ParserBoltTest.createUpdater();
       }
     };
@@ -494,12 +481,93 @@ public void testImplicitBatchOfOne() throws Exception {
     when(parser.parseOptional(any())).thenReturn(Optional.of(ImmutableList.of(new JSONObject())));
     when(filter.emitTuple(any(), any(Context.class))).thenReturn(true);
     BulkWriterResponse response = new BulkWriterResponse();
+    Tuple[] uniqueTuples = new Tuple[ParserConfigurations.DEFAULT_KAFKA_BATCH_SIZE];
+    for (int i=0; i < uniqueTuples.length; i++) {
+      uniqueTuples[i] = mock(Tuple.class);
+      response.addSuccess(uniqueTuples[i]);
+    }
+    when(batchWriter.write(eq(sensorType), any(WriterConfiguration.class), eq(new HashSet<>(Arrays.asList(uniqueTuples))), any())).thenReturn(response);
+    parserBolt.withMessageFilter(filter);
+    for (Tuple tuple : uniqueTuples) {
+      parserBolt.execute(tuple);
+    }
+    for (Tuple uniqueTuple : uniqueTuples) {
+      verify(outputCollector, times(1)).ack(uniqueTuple);
+    }
+  }
+
+  @Test
+  public void testLessRecordsThanDefaultBatchSize() throws Exception {
+
+    String sensorType = "yaf";
+
+    ParserBolt parserBolt = new ParserBolt("zookeeperUrl", sensorType, parser, new WriterHandler(batchWriter)) {
+      @Override
+      protected ConfigurationsUpdater<ParserConfigurations> createUpdater() {
+        // this uses default batch size
+        return ParserBoltTest.createUpdater();
+      }
+    };
+
+    parserBolt.setCuratorFramework(client);
+    parserBolt.setZKCache(cache);
+    parserBolt.prepare(new HashMap(), topologyContext, outputCollector);
+    verify(parser, times(1)).init();
+    verify(batchWriter, times(1)).init(any(), any(), any());
+    when(parser.validate(any())).thenReturn(true);
+    when(parser.parseOptional(any())).thenReturn(Optional.of(ImmutableList.of(new JSONObject())));
+    when(filter.emitTuple(any(), any(Context.class))).thenReturn(true);
+    int oneLessThanDefaultBatchSize = ParserConfigurations.DEFAULT_KAFKA_BATCH_SIZE - 1;
+    BulkWriterResponse response = new BulkWriterResponse();
+    Tuple[] uniqueTuples = new Tuple[oneLessThanDefaultBatchSize];
+    for (int i=0; i < uniqueTuples.length; i++) {
+      uniqueTuples[i] = mock(Tuple.class);
+      response.addSuccess(uniqueTuples[i]);
+    }
+    parserBolt.withMessageFilter(filter);
+    for (Tuple tuple : uniqueTuples) {
+      parserBolt.execute(tuple);
+    }
+    // should have no acking yet - batch size not fulfilled
+    verify(outputCollector, never()).ack(any(Tuple.class));
+    response.addSuccess(t1); // used to achieve count in final verify
+    Iterable<Tuple> tuples = new HashSet(Arrays.asList(uniqueTuples)) {{
+      add(t1);
+    }};
+    when(batchWriter.write(eq(sensorType), any(WriterConfiguration.class), eq(tuples), any())).thenReturn(response);
+    // meet batch size requirement and now it should ack
+    parserBolt.execute(t1);
+    verify(outputCollector, times(ParserConfigurations.DEFAULT_KAFKA_BATCH_SIZE)).ack(any(Tuple.class));
+  }
+
+  @Test
+  public void testBatchOfOne() throws Exception {
+
+    String sensorType = "yaf";
+
+    ParserBolt parserBolt = new ParserBolt("zookeeperUrl", sensorType, parser, new WriterHandler(batchWriter)) {
+      @Override
+      protected ConfigurationsUpdater<ParserConfigurations> createUpdater() {
+        return ParserBoltTest.createUpdater(Optional.of(1));
+      }
+    };
+
+    parserBolt.setCuratorFramework(client);
+    parserBolt.setZKCache(cache);
+    parserBolt.prepare(new HashMap(), topologyContext, outputCollector);
+    verify(parser, times(1)).init();
+    verify(batchWriter, times(1)).init(any(), any(), any());
+    when(parser.validate(any())).thenReturn(true);
+    when(parser.parseOptional(any())).thenReturn(Optional.of(ImmutableList.of(new JSONObject())));
+    when(filter.emitTuple(any(), any(Context.class))).thenReturn(true);
+    BulkWriterResponse response = new BulkWriterResponse();
     response.addSuccess(t1);
     when(batchWriter.write(eq(sensorType), any(WriterConfiguration.class), eq(Collections.singleton(t1)), any())).thenReturn(response);
     parserBolt.withMessageFilter(filter);
     parserBolt.execute(t1);
     verify(outputCollector, times(1)).ack(t1);
   }
+
   @Test
   public void testBatchOfFive() throws Exception {
 

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-parsers/src/test/java/org/apache/metron/writers/integration/WriterBoltIntegrationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-parsers/src/test/java/org/apache/metron/writers/integration/WriterBoltIntegrationTest.java b/metron-platform/metron-parsers/src/test/java/org/apache/metron/writers/integration/WriterBoltIntegrationTest.java
index d565147..dfadfdc 100644
--- a/metron-platform/metron-parsers/src/test/java/org/apache/metron/writers/integration/WriterBoltIntegrationTest.java
+++ b/metron-platform/metron-parsers/src/test/java/org/apache/metron/writers/integration/WriterBoltIntegrationTest.java
@@ -94,6 +94,7 @@ public class WriterBoltIntegrationTest extends BaseIntegrationTest {
    *    "outputTopic": "output",
    *    "errorTopic": "parser_error",
    *    "parserConfig": {
+   *        "batchSize" : 1,
    *        "columns" : {
    *            "action" : 0,
    *            "dummy" : 1
@@ -246,7 +247,10 @@ public class WriterBoltIntegrationTest extends BaseIntegrationTest {
    *    "parserClassName":"org.apache.metron.writers.integration.WriterBoltIntegrationTest$EmptyObjectParser",
    *    "sensorTopic":"emptyobjectparser",
    *    "outputTopic": "enrichments",
-   *    "errorTopic": "parser_error"
+   *    "errorTopic": "parser_error",
+   *    "parserConfig": {
+   *        "batchSize" : 1
+   *    }
    * }
    */
   @Multiline

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BatchTimeoutHelper.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BatchTimeoutHelper.java b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BatchTimeoutHelper.java
index 22440ad..195e010 100644
--- a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BatchTimeoutHelper.java
+++ b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BatchTimeoutHelper.java
@@ -57,7 +57,7 @@ public class BatchTimeoutHelper {
   protected int minBatchTimeoutRequestedSecs; //min of all sensorType configured batchTimeout requests
   protected int recommendedTickIntervalSecs;  //the answer
 
-  BatchTimeoutHelper( Supplier<List<Integer>> listAllConfiguredTimeouts
+  public BatchTimeoutHelper( Supplier<List<Integer>> listAllConfiguredTimeouts
           , int batchTimeoutDivisor
           )
   {
@@ -130,7 +130,7 @@ public class BatchTimeoutHelper {
    * @return the max batchTimeout allowed, in seconds
    * Guaranteed positive number.
    */
-  protected int getDefaultBatchTimeout() {
+  public int getDefaultBatchTimeout() {
     if (!initialized) {this.init();}
     return maxBatchTimeoutAllowedSecs;
   }
@@ -160,7 +160,7 @@ public class BatchTimeoutHelper {
    * @return the recommended TickInterval to request, in seconds
    * Guaranteed positive number.
    */
-  protected int getRecommendedTickInterval() {
+  public int getRecommendedTickInterval() {
     if (!initialized) {this.init();}
     // Remember that parameter settings in the CLI override parameter settings set by the Storm component.
     // We shouldn't have to deal with this in the Metron environment, but just in case,

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
index 1d8f0c6..4bb3888 100644
--- a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
+++ b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
@@ -17,10 +17,15 @@
  */
 package org.apache.metron.writer.bolt;
 
+import static org.apache.storm.utils.TupleUtils.isTick;
+
 import com.google.common.collect.ImmutableList;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Function;
 import org.apache.metron.common.Constants;
-import org.apache.metron.common.bolt.ConfiguredIndexingBolt;
-import org.apache.metron.common.configuration.writer.IndexingWriterConfiguration;
+import org.apache.metron.common.bolt.ConfiguredBolt;
+import org.apache.metron.common.configuration.Configurations;
 import org.apache.metron.common.configuration.writer.WriterConfiguration;
 import org.apache.metron.common.message.MessageGetStrategy;
 import org.apache.metron.common.message.MessageGetters;
@@ -40,13 +45,7 @@ import org.json.simple.JSONObject;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
-import java.util.HashMap;
-import java.util.Map;
-import java.util.function.Function;
-
-import static org.apache.storm.utils.TupleUtils.isTick;
-
-public class BulkMessageWriterBolt extends ConfiguredIndexingBolt {
+public class BulkMessageWriterBolt<CONFIG_T extends Configurations> extends ConfiguredBolt<CONFIG_T> {
 
   private static final Logger LOG = LoggerFactory
           .getLogger(BulkMessageWriterBolt.class);
@@ -61,26 +60,26 @@ public class BulkMessageWriterBolt extends ConfiguredIndexingBolt {
   private int defaultBatchTimeout;
   private int batchTimeoutDivisor = 1;
 
-  public BulkMessageWriterBolt(String zookeeperUrl) {
-    super(zookeeperUrl);
+  public BulkMessageWriterBolt(String zookeeperUrl, String configurationStrategy) {
+    super(zookeeperUrl, configurationStrategy);
   }
 
-  public BulkMessageWriterBolt withBulkMessageWriter(BulkMessageWriter<JSONObject > bulkMessageWriter) {
+  public BulkMessageWriterBolt<CONFIG_T> withBulkMessageWriter(BulkMessageWriter<JSONObject> bulkMessageWriter) {
     this.bulkMessageWriter = bulkMessageWriter;
     return this;
   }
 
-  public BulkMessageWriterBolt withMessageWriter(MessageWriter<JSONObject> messageWriter) {
+  public BulkMessageWriterBolt<CONFIG_T> withMessageWriter(MessageWriter<JSONObject> messageWriter) {
     this.bulkMessageWriter = new WriterToBulkWriter<>(messageWriter);
     return this;
   }
 
-  public BulkMessageWriterBolt withMessageGetter(String messageGetStrategyType) {
+  public BulkMessageWriterBolt<CONFIG_T> withMessageGetter(String messageGetStrategyType) {
     this.messageGetStrategyType = messageGetStrategyType;
     return this;
   }
 
-  public BulkMessageWriterBolt withMessageGetterField(String messageGetField) {
+  public BulkMessageWriterBolt<CONFIG_T> withMessageGetterField(String messageGetField) {
     this.messageGetField = messageGetField;
     return this;
   }
@@ -103,7 +102,7 @@ public class BulkMessageWriterBolt extends ConfiguredIndexingBolt {
    * @param batchTimeoutDivisor
    * @return BulkMessageWriterBolt
    */
-  public BulkMessageWriterBolt withBatchTimeoutDivisor(int batchTimeoutDivisor) {
+  public BulkMessageWriterBolt<CONFIG_T> withBatchTimeoutDivisor(int batchTimeoutDivisor) {
     if (batchTimeoutDivisor <= 0) {
       throw new IllegalArgumentException(String.format("batchTimeoutDivisor must be positive. Value provided was %s", batchTimeoutDivisor));
     }
@@ -133,6 +132,7 @@ public class BulkMessageWriterBolt extends ConfiguredIndexingBolt {
   public void setWriterComponent(BulkWriterComponent<JSONObject> component) {
     writerComponent = component;
   }
+
   /**
    * This method is called by TopologyBuilder.createTopology() to obtain topology and
    * bolt specific configuration parameters.  We use it primarily to configure how often
@@ -151,8 +151,8 @@ public class BulkMessageWriterBolt extends ConfiguredIndexingBolt {
     else {
       configurationXform = x -> x;
     }
-    WriterConfiguration writerconf = configurationXform.apply(
-            new IndexingWriterConfiguration(bulkMessageWriter.getName(), getConfigurations()));
+    WriterConfiguration writerconf = configurationXform
+        .apply(getConfigurationStrategy().createWriterConfig(bulkMessageWriter, getConfigurations()));
 
     BatchTimeoutHelper timeoutHelper = new BatchTimeoutHelper(writerconf::getAllConfiguredTimeouts, batchTimeoutDivisor);
     this.requestedTickFreqSecs = timeoutHelper.getRecommendedTickInterval();
@@ -187,8 +187,8 @@ public class BulkMessageWriterBolt extends ConfiguredIndexingBolt {
       configurationTransformation = x -> x;
     }
     try {
-      WriterConfiguration writerconf = configurationTransformation.apply(
-              new IndexingWriterConfiguration(bulkMessageWriter.getName(), getConfigurations()));
+      WriterConfiguration writerconf = configurationTransformation
+          .apply(getConfigurationStrategy().createWriterConfig(bulkMessageWriter, getConfigurations()));
       if (defaultBatchTimeout == 0) {
         //This means getComponentConfiguration was never called to initialize defaultBatchTimeout,
         //probably because we are in a unit test scenario.  So calculate it here.
@@ -219,8 +219,8 @@ public class BulkMessageWriterBolt extends ConfiguredIndexingBolt {
           //WriterToBulkWriter doesn't allow batching, so no need to flush on Tick.
           LOG.debug("Flushing message queues older than their batchTimeouts");
           getWriterComponent().flushTimeouts(bulkMessageWriter, configurationTransformation.apply(
-                  new IndexingWriterConfiguration(bulkMessageWriter.getName(), getConfigurations()))
-                  , messageGetStrategy);
+              getConfigurationStrategy().createWriterConfig(bulkMessageWriter, getConfigurations())),
+              messageGetStrategy);
         }
       }
       catch(Exception e) {
@@ -247,8 +247,8 @@ public class BulkMessageWriterBolt extends ConfiguredIndexingBolt {
       }
 
       LOG.trace("Writing enrichment message: {}", message);
-      WriterConfiguration writerConfiguration = configurationTransformation.apply(
-              new IndexingWriterConfiguration(bulkMessageWriter.getName(), getConfigurations()));
+      WriterConfiguration writerConfiguration = configurationTransformation
+          .apply(getConfigurationStrategy().createWriterConfig(bulkMessageWriter, getConfigurations()));
 
       if (writerConfiguration.isDefault(sensorType)) {
         //want to warn, but not fail the tuple

http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/kafka/KafkaWriter.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/kafka/KafkaWriter.java b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/kafka/KafkaWriter.java
index c4e3998..efb2418 100644
--- a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/kafka/KafkaWriter.java
+++ b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/kafka/KafkaWriter.java
@@ -17,25 +17,32 @@
  */
 package org.apache.metron.writer.kafka;
 
-import org.apache.storm.tuple.Tuple;
 import com.google.common.base.Joiner;
+import java.io.Serializable;
+import java.util.AbstractMap;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Future;
 import org.apache.kafka.clients.producer.KafkaProducer;
+import org.apache.kafka.clients.producer.ProducerConfig;
 import org.apache.kafka.clients.producer.ProducerRecord;
+import org.apache.kafka.common.errors.InterruptException;
 import org.apache.metron.common.Constants;
 import org.apache.metron.common.configuration.writer.WriterConfiguration;
-import org.apache.metron.common.writer.MessageWriter;
-import org.apache.metron.stellar.common.utils.ConversionUtils;
 import org.apache.metron.common.utils.KafkaUtils;
 import org.apache.metron.common.utils.StringUtils;
+import org.apache.metron.common.writer.BulkMessageWriter;
+import org.apache.metron.common.writer.BulkWriterResponse;
+import org.apache.metron.stellar.common.utils.ConversionUtils;
 import org.apache.metron.writer.AbstractWriter;
+import org.apache.storm.task.TopologyContext;
+import org.apache.storm.tuple.Tuple;
 import org.json.simple.JSONObject;
 
-import java.io.Serializable;
-import java.util.HashMap;
-import java.util.Map;
-import java.util.Optional;
-
-public class KafkaWriter extends AbstractWriter implements MessageWriter<JSONObject>, Serializable {
+public class KafkaWriter extends AbstractWriter implements BulkMessageWriter<JSONObject>, Serializable {
   public enum Configurations {
      BROKER("kafka.brokerUrl")
     ,KEY_SERIALIZER("kafka.keySerializer")
@@ -60,6 +67,15 @@ public class KafkaWriter extends AbstractWriter implements MessageWriter<JSONObj
       return null;
     }
   }
+
+  /**
+   * Default batch size in bytes. Note, we don't want to expose this to clients. End users should
+   * set the writer batchSize setting via the BulkWriterComponent.
+   *
+   * @see ProducerConfig#BATCH_SIZE_DOC
+   * @see <a href="https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_kafka-component-guide/content/kafka-producer-settings.html">https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_kafka-component-guide/content/kafka-producer-settings.html</a>
+   */
+  private static final int DEFAULT_BATCH_SIZE = 1_024 * 64; // 64 kilobytes
   private String brokerUrl;
   private String keySerializer = "org.apache.kafka.common.serialization.StringSerializer";
   private String valueSerializer = "org.apache.kafka.common.serialization.StringSerializer";
@@ -156,33 +172,69 @@ public class KafkaWriter extends AbstractWriter implements MessageWriter<JSONObj
     }
   }
 
+  @Override
+  public void init(Map stormConf, TopologyContext topologyContext, WriterConfiguration config)
+      throws Exception {
+    if(this.zkQuorum != null && this.brokerUrl == null) {
+      try {
+        this.brokerUrl = Joiner.on(",").join(KafkaUtils.INSTANCE.getBrokersFromZookeeper(this.zkQuorum));
+      } catch (Exception e) {
+        throw new IllegalStateException("Cannot read kafka brokers from zookeeper and you didn't specify them, giving up!", e);
+      }
+    }
+    this.kafkaProducer = new KafkaProducer<>(createProducerConfigs());
+  }
+
   public Map<String, Object> createProducerConfigs() {
     Map<String, Object> producerConfig = new HashMap<>();
     producerConfig.put("bootstrap.servers", brokerUrl);
     producerConfig.put("key.serializer", keySerializer);
     producerConfig.put("value.serializer", valueSerializer);
     producerConfig.put("request.required.acks", requiredAcks);
+    producerConfig.put(ProducerConfig.BATCH_SIZE_CONFIG, DEFAULT_BATCH_SIZE);
     producerConfig.putAll(producerConfigs == null?new HashMap<>():producerConfigs);
     producerConfig = KafkaUtils.INSTANCE.normalizeProtocol(producerConfig);
     return producerConfig;
   }
 
   @Override
-  public void init() {
-    if(this.zkQuorum != null && this.brokerUrl == null) {
+  public BulkWriterResponse write(String sensorType, WriterConfiguration configurations,
+      Iterable<Tuple> tuples, List<JSONObject> messages) {
+    BulkWriterResponse writerResponse = new BulkWriterResponse();
+    List<Map.Entry<Tuple, Future>> results = new ArrayList<>();
+    int i = 0;
+    for (Tuple tuple : tuples) {
+      JSONObject message = messages.get(i++);
+      String jsonMessage;
       try {
-        this.brokerUrl = Joiner.on(",").join(KafkaUtils.INSTANCE.getBrokersFromZookeeper(this.zkQuorum));
-      } catch (Exception e) {
-        throw new IllegalStateException("Cannot read kafka brokers from zookeeper and you didn't specify them, giving up!", e);
+         jsonMessage = message.toJSONString();
+      } catch (Throwable t) {
+        writerResponse.addError(t, tuple);
+        continue;
       }
+      Future future = kafkaProducer
+          .send(new ProducerRecord<String, String>(kafkaTopic, jsonMessage));
+      // we want to manage the batching
+      results.add(new AbstractMap.SimpleEntry<>(tuple, future));
     }
-    this.kafkaProducer = new KafkaProducer<>(createProducerConfigs());
-  }
 
-  @SuppressWarnings("unchecked")
-  @Override
-  public void write(String sourceType, WriterConfiguration configurations, Tuple tuple, JSONObject message) throws Exception {
-    kafkaProducer.send(new ProducerRecord<String, String>(kafkaTopic, message.toJSONString()));
+    try {
+      // ensures all Future.isDone() == true
+      kafkaProducer.flush();
+    } catch (InterruptException e) {
+      writerResponse.addAllErrors(e, tuples);
+      return writerResponse;
+    }
+
+    for (Map.Entry<Tuple, Future> kv : results) {
+      try {
+        kv.getValue().get();
+        writerResponse.addSuccess(kv.getKey());
+      } catch (Exception e) {
+        writerResponse.addError(e, kv.getKey());
+      }
+    }
+    return writerResponse;
   }
 
   @Override


[42/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/css/maven-base.css
----------------------------------------------------------------------
diff --git a/site/current-book/css/maven-base.css b/site/current-book/css/maven-base.css
new file mode 100644
index 0000000..322efae
--- /dev/null
+++ b/site/current-book/css/maven-base.css
@@ -0,0 +1,168 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+body {
+  margin: 0px;
+  padding: 0px;
+}
+table {
+  padding:0px;
+  width: 100%;
+  margin-left: -2px;
+  margin-right: -2px;
+}
+acronym {
+  cursor: help;
+  border-bottom: 1px dotted #feb;
+}
+table.bodyTable th, table.bodyTable td {
+  padding: 2px 4px 2px 4px;
+  vertical-align: top;
+}
+div.clear{
+  clear:both;
+  visibility: hidden;
+}
+div.clear hr{
+  display: none;
+}
+#bannerLeft, #bannerRight {
+  font-size: xx-large;
+  font-weight: bold;
+}
+#bannerLeft img, #bannerRight img {
+  margin: 0px;
+}
+.xleft, #bannerLeft img {
+  float:left;
+}
+.xright, #bannerRight {
+  float:right;
+}
+#banner {
+  padding: 0px;
+}
+#breadcrumbs {
+  padding: 3px 10px 3px 10px;
+}
+#leftColumn {
+ width: 170px;
+ float:left;
+ overflow: auto;
+}
+#bodyColumn {
+  margin-right: 1.5em;
+  margin-left: 197px;
+}
+#legend {
+  padding: 8px 0 8px 0;
+}
+#navcolumn {
+  padding: 8px 4px 0 8px;
+}
+#navcolumn h5 {
+  margin: 0;
+  padding: 0;
+  font-size: small;
+}
+#navcolumn ul {
+  margin: 0;
+  padding: 0;
+  font-size: small;
+}
+#navcolumn li {
+  list-style-type: none;
+  background-image: none;
+  background-repeat: no-repeat;
+  background-position: 0 0.4em;
+  padding-left: 16px;
+  list-style-position: outside;
+  line-height: 1.2em;
+  font-size: smaller;
+}
+#navcolumn li.expanded {
+  background-image: url(../images/expanded.gif);
+}
+#navcolumn li.collapsed {
+  background-image: url(../images/collapsed.gif);
+}
+#navcolumn li.none {
+  text-indent: -1em;
+  margin-left: 1em;
+}
+#poweredBy {
+  text-align: center;
+}
+#navcolumn img {
+  margin-top: 10px;
+  margin-bottom: 3px;
+}
+#poweredBy img {
+  display:block;
+  margin: 20px 0 20px 17px;
+}
+#search img {
+    margin: 0px;
+    display: block;
+}
+#search #q, #search #btnG {
+    border: 1px solid #999;
+    margin-bottom:10px;
+}
+#search form {
+    margin: 0px;
+}
+#lastPublished {
+  font-size: x-small;
+}
+.navSection {
+  margin-bottom: 2px;
+  padding: 8px;
+}
+.navSectionHead {
+  font-weight: bold;
+  font-size: x-small;
+}
+.section {
+  padding: 4px;
+}
+#footer {
+  padding: 3px 10px 3px 10px;
+  font-size: x-small;
+}
+#breadcrumbs {
+  font-size: x-small;
+  margin: 0pt;
+}
+.source {
+  padding: 12px;
+  margin: 1em 7px 1em 7px;
+}
+.source pre {
+  margin: 0px;
+  padding: 0px;
+}
+#navcolumn img.imageLink, .imageLink {
+  padding-left: 0px;
+  padding-bottom: 0px;
+  padding-top: 0px;
+  padding-right: 2px;
+  border: 0px;
+  margin: 0px;
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/css/maven-theme.css
----------------------------------------------------------------------
diff --git a/site/current-book/css/maven-theme.css b/site/current-book/css/maven-theme.css
new file mode 100644
index 0000000..4e2bdfb
--- /dev/null
+++ b/site/current-book/css/maven-theme.css
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+body {
+  padding: 0px 0px 10px 0px;
+}
+body, td, select, input, li{
+  font-family: Verdana, Helvetica, Arial, sans-serif;
+  font-size: 13px;
+}
+code{
+  font-family: Courier, monospace;
+  font-size: 13px;
+}
+a {
+  text-decoration: none;
+}
+a:link {
+  color:#36a;
+}
+a:visited  {
+  color:#47a;
+}
+a:active, a:hover {
+  color:#69c;
+}
+#legend li.externalLink {
+  background: url(../images/external.png) left top no-repeat;
+  padding-left: 18px;
+}
+a.externalLink, a.externalLink:link, a.externalLink:visited, a.externalLink:active, a.externalLink:hover {
+  background: url(../images/external.png) right center no-repeat;
+  padding-right: 18px;
+}
+#legend li.newWindow {
+  background: url(../images/newwindow.png) left top no-repeat;
+  padding-left: 18px;
+}
+a.newWindow, a.newWindow:link, a.newWindow:visited, a.newWindow:active, a.newWindow:hover {
+  background: url(../images/newwindow.png) right center no-repeat;
+  padding-right: 18px;
+}
+h2 {
+  padding: 4px 4px 4px 6px;
+  border: 1px solid #999;
+  color: #900;
+  background-color: #ddd;
+  font-weight:900;
+  font-size: x-large;
+}
+h3 {
+  padding: 4px 4px 4px 6px;
+  border: 1px solid #aaa;
+  color: #900;
+  background-color: #eee;
+  font-weight: normal;
+  font-size: large;
+}
+h4 {
+  padding: 4px 4px 4px 6px;
+  border: 1px solid #bbb;
+  color: #900;
+  background-color: #fff;
+  font-weight: normal;
+  font-size: large;
+}
+h5 {
+  padding: 4px 4px 4px 6px;
+  color: #900;
+  font-size: medium;
+}
+p {
+  line-height: 1.3em;
+  font-size: small;
+}
+#breadcrumbs {
+  border-top: 1px solid #aaa;
+  border-bottom: 1px solid #aaa;
+  background-color: #ccc;
+}
+#leftColumn {
+  margin: 10px 0 0 5px;
+  border: 1px solid #999;
+  background-color: #eee;
+  padding-bottom: 3px; /* IE-9 scrollbar-fix */
+}
+#navcolumn h5 {
+  font-size: smaller;
+  border-bottom: 1px solid #aaaaaa;
+  padding-top: 2px;
+  color: #000;
+}
+
+table.bodyTable th {
+  color: white;
+  background-color: #bbb;
+  text-align: left;
+  font-weight: bold;
+}
+
+table.bodyTable th, table.bodyTable td {
+  font-size: 1em;
+}
+
+table.bodyTable tr.a {
+  background-color: #ddd;
+}
+
+table.bodyTable tr.b {
+  background-color: #eee;
+}
+
+.source {
+  border: 1px solid #999;
+}
+dl {
+  padding: 4px 4px 4px 6px;
+  border: 1px solid #aaa;
+  background-color: #ffc;
+}
+dt {
+  color: #900;
+}
+#organizationLogo img, #projectLogo img, #projectLogo span{
+  margin: 8px;
+}
+#banner {
+  border-bottom: 1px solid #fff;
+}
+.errormark, .warningmark, .donemark, .infomark {
+  background: url(../images/icon_error_sml.gif) no-repeat;
+}
+
+.warningmark {
+  background-image: url(../images/icon_warning_sml.gif);
+}
+
+.donemark {
+  background-image: url(../images/icon_success_sml.gif);
+}
+
+.infomark {
+  background-image: url(../images/icon_info_sml.gif);
+}
+

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/css/print.css
----------------------------------------------------------------------
diff --git a/site/current-book/css/print.css b/site/current-book/css/print.css
index 1cd02d9..18fcbad 100644
--- a/site/current-book/css/print.css
+++ b/site/current-book/css/print.css
@@ -17,7 +17,10 @@
  * under the License.
  */
 
-/* $Id: print.css 1201871 2011-11-14 20:18:24Z simonetripodi $ */
-
-#banner, #footer, #leftcol, #breadcrumbs, .docs #toc, .docs .courtesylinks, #leftColumn, #navColumn {display: none !important;}
-#bodyColumn, body.docs div.docs {margin: 0 !important;border: none !important}
+#banner, #footer, #leftcol, #breadcrumbs, .docs #toc, .docs .courtesylinks, #leftColumn, #navColumn {
+	display: none !important;
+}
+#bodyColumn, body.docs div.docs {
+	margin: 0 !important;
+	border: none !important
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/fonts/glyphicons-halflings-regular.eot
----------------------------------------------------------------------
diff --git a/site/current-book/fonts/glyphicons-halflings-regular.eot b/site/current-book/fonts/glyphicons-halflings-regular.eot
new file mode 100644
index 0000000..637452e
Binary files /dev/null and b/site/current-book/fonts/glyphicons-halflings-regular.eot differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/fonts/glyphicons-halflings-regular.svg
----------------------------------------------------------------------
diff --git a/site/current-book/fonts/glyphicons-halflings-regular.svg b/site/current-book/fonts/glyphicons-halflings-regular.svg
new file mode 100644
index 0000000..4469488
--- /dev/null
+++ b/site/current-book/fonts/glyphicons-halflings-regular.svg
@@ -0,0 +1,229 @@
+<?xml version="1.0" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd" >
+<svg xmlns="http://www.w3.org/2000/svg">
+<metadata></metadata>
+<defs>
+<font id="glyphicons_halflingsregular" horiz-adv-x="1200" >
+<font-face units-per-em="1200" ascent="960" descent="-240" />
+<missing-glyph horiz-adv-x="500" />
+<glyph />
+<glyph />
+<glyph unicode="&#xd;" />
+<glyph unicode=" " />
+<glyph unicode="*" d="M100 500v200h259l-183 183l141 141l183 -183v259h200v-259l183 183l141 -141l-183 -183h259v-200h-259l183 -183l-141 -141l-183 183v-259h-200v259l-183 -183l-141 141l183 183h-259z" />
+<glyph unicode="+" d="M0 400v300h400v400h300v-400h400v-300h-400v-400h-300v400h-400z" />
+<glyph unicode="&#xa0;" />
+<glyph unicode="&#x2000;" horiz-adv-x="652" />
+<glyph unicode="&#x2001;" horiz-adv-x="1304" />
+<glyph unicode="&#x2002;" horiz-adv-x="652" />
+<glyph unicode="&#x2003;" horiz-adv-x="1304" />
+<glyph unicode="&#x2004;" horiz-adv-x="434" />
+<glyph unicode="&#x2005;" horiz-adv-x="326" />
+<glyph unicode="&#x2006;" horiz-adv-x="217" />
+<glyph unicode="&#x2007;" horiz-adv-x="217" />
+<glyph unicode="&#x2008;" horiz-adv-x="163" />
+<glyph unicode="&#x2009;" horiz-adv-x="260" />
+<glyph unicode="&#x200a;" horiz-adv-x="72" />
+<glyph unicode="&#x202f;" horiz-adv-x="260" />
+<glyph unicode="&#x205f;" horiz-adv-x="326" />
+<glyph unicode="&#x20ac;" d="M100 500l100 100h113q0 47 5 100h-218l100 100h135q37 167 112 257q117 141 297 141q242 0 354 -189q60 -103 66 -209h-181q0 55 -25.5 99t-63.5 68t-75 36.5t-67 12.5q-24 0 -52.5 -10t-62.5 -32t-65.5 -67t-50.5 -107h379l-100 -100h-300q-6 -46 -6 -100h406l-100 -100 h-300q9 -74 33 -132t52.5 -91t62 -54.5t59 -29t46.5 -7.5q29 0 66 13t75 37t63.5 67.5t25.5 96.5h174q-31 -172 -128 -278q-107 -117 -274 -117q-205 0 -324 158q-36 46 -69 131.5t-45 205.5h-217z" />
+<glyph unicode="&#x2212;" d="M200 400h900v300h-900v-300z" />
+<glyph unicode="&#x2601;" d="M-14 494q0 -80 56.5 -137t135.5 -57h750q120 0 205 86t85 208q0 120 -85 206.5t-205 86.5q-46 0 -90 -14q-44 97 -134.5 156.5t-200.5 59.5q-152 0 -260 -107.5t-108 -260.5q0 -25 2 -37q-66 -14 -108.5 -67.5t-42.5 -122.5z" />
+<glyph unicode="&#x2709;" d="M0 100l400 400l200 -200l200 200l400 -400h-1200zM0 300v600l300 -300zM0 1100l600 -603l600 603h-1200zM900 600l300 300v-600z" />
+<glyph unicode="&#x270f;" d="M-13 -13l333 112l-223 223zM187 403l214 -214l614 614l-214 214zM887 1103l214 -214l99 92q13 13 13 32.5t-13 33.5l-153 153q-15 13 -33 13t-33 -13z" />
+<glyph unicode="&#xe000;" horiz-adv-x="500" d="M0 0z" />
+<glyph unicode="&#xe001;" d="M0 1200h1200l-500 -550v-550h300v-100h-800v100h300v550z" />
+<glyph unicode="&#xe002;" d="M14 84q18 -55 86 -75.5t147 5.5q65 21 109 69t44 90v606l600 155v-521q-64 16 -138 -7q-79 -26 -122.5 -83t-25.5 -111q17 -55 85.5 -75.5t147.5 4.5q70 23 111.5 63.5t41.5 95.5v881q0 10 -7 15.5t-17 2.5l-752 -193q-10 -3 -17 -12.5t-7 -19.5v-689q-64 17 -138 -7 q-79 -25 -122.5 -82t-25.5 -112z" />
+<glyph unicode="&#xe003;" d="M23 693q0 200 142 342t342 142t342 -142t142 -342q0 -142 -78 -261l300 -300q7 -8 7 -18t-7 -18l-109 -109q-8 -7 -18 -7t-18 7l-300 300q-119 -78 -261 -78q-200 0 -342 142t-142 342zM176 693q0 -136 97 -233t234 -97t233.5 96.5t96.5 233.5t-96.5 233.5t-233.5 96.5 t-234 -97t-97 -233z" />
+<glyph unicode="&#xe005;" d="M100 784q0 64 28 123t73 100.5t104.5 64t119 20.5t120 -38.5t104.5 -104.5q48 69 109.5 105t121.5 38t118.5 -20.5t102.5 -64t71 -100.5t27 -123q0 -57 -33.5 -117.5t-94 -124.5t-126.5 -127.5t-150 -152.5t-146 -174q-62 85 -145.5 174t-149.5 152.5t-126.5 127.5 t-94 124.5t-33.5 117.5z" />
+<glyph unicode="&#xe006;" d="M-72 800h479l146 400h2l146 -400h472l-382 -278l145 -449l-384 275l-382 -275l146 447zM168 71l2 1z" />
+<glyph unicode="&#xe007;" d="M-72 800h479l146 400h2l146 -400h472l-382 -278l145 -449l-384 275l-382 -275l146 447zM168 71l2 1zM237 700l196 -142l-73 -226l192 140l195 -141l-74 229l193 140h-235l-77 211l-78 -211h-239z" />
+<glyph unicode="&#xe008;" d="M0 0v143l400 257v100q-37 0 -68.5 74.5t-31.5 125.5v200q0 124 88 212t212 88t212 -88t88 -212v-200q0 -51 -31.5 -125.5t-68.5 -74.5v-100l400 -257v-143h-1200z" />
+<glyph unicode="&#xe009;" d="M0 0v1100h1200v-1100h-1200zM100 100h100v100h-100v-100zM100 300h100v100h-100v-100zM100 500h100v100h-100v-100zM100 700h100v100h-100v-100zM100 900h100v100h-100v-100zM300 100h600v400h-600v-400zM300 600h600v400h-600v-400zM1000 100h100v100h-100v-100z M1000 300h100v100h-100v-100zM1000 500h100v100h-100v-100zM1000 700h100v100h-100v-100zM1000 900h100v100h-100v-100z" />
+<glyph unicode="&#xe010;" d="M0 50v400q0 21 14.5 35.5t35.5 14.5h400q21 0 35.5 -14.5t14.5 -35.5v-400q0 -21 -14.5 -35.5t-35.5 -14.5h-400q-21 0 -35.5 14.5t-14.5 35.5zM0 650v400q0 21 14.5 35.5t35.5 14.5h400q21 0 35.5 -14.5t14.5 -35.5v-400q0 -21 -14.5 -35.5t-35.5 -14.5h-400 q-21 0 -35.5 14.5t-14.5 35.5zM600 50v400q0 21 14.5 35.5t35.5 14.5h400q21 0 35.5 -14.5t14.5 -35.5v-400q0 -21 -14.5 -35.5t-35.5 -14.5h-400q-21 0 -35.5 14.5t-14.5 35.5zM600 650v400q0 21 14.5 35.5t35.5 14.5h400q21 0 35.5 -14.5t14.5 -35.5v-400 q0 -21 -14.5 -35.5t-35.5 -14.5h-400q-21 0 -35.5 14.5t-14.5 35.5z" />
+<glyph unicode="&#xe011;" d="M0 50v200q0 21 14.5 35.5t35.5 14.5h200q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5zM0 450v200q0 21 14.5 35.5t35.5 14.5h200q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-200 q-21 0 -35.5 14.5t-14.5 35.5zM0 850v200q0 21 14.5 35.5t35.5 14.5h200q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5zM400 50v200q0 21 14.5 35.5t35.5 14.5h200q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5 t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5zM400 450v200q0 21 14.5 35.5t35.5 14.5h200q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5zM400 850v200q0 21 14.5 35.5t35.5 14.5h200q21 0 35.5 -14.5t14.5 -35.5 v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5zM800 50v200q0 21 14.5 35.5t35.5 14.5h200q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5zM800 450v200q0 
 21 14.5 35.5t35.5 14.5h200 q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5zM800 850v200q0 21 14.5 35.5t35.5 14.5h200q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5z" />
+<glyph unicode="&#xe012;" d="M0 50v200q0 21 14.5 35.5t35.5 14.5h200q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5zM0 450q0 -21 14.5 -35.5t35.5 -14.5h200q21 0 35.5 14.5t14.5 35.5v200q0 21 -14.5 35.5t-35.5 14.5h-200q-21 0 -35.5 -14.5 t-14.5 -35.5v-200zM0 850v200q0 21 14.5 35.5t35.5 14.5h200q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5zM400 50v200q0 21 14.5 35.5t35.5 14.5h700q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5 t-35.5 -14.5h-700q-21 0 -35.5 14.5t-14.5 35.5zM400 450v200q0 21 14.5 35.5t35.5 14.5h700q21 0 35.5 -14.5t14.5 -35.5v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-700q-21 0 -35.5 14.5t-14.5 35.5zM400 850v200q0 21 14.5 35.5t35.5 14.5h700q21 0 35.5 -14.5t14.5 -35.5 v-200q0 -21 -14.5 -35.5t-35.5 -14.5h-700q-21 0 -35.5 14.5t-14.5 35.5z" />
+<glyph unicode="&#xe013;" d="M29 454l419 -420l818 820l-212 212l-607 -607l-206 207z" />
+<glyph unicode="&#xe014;" d="M106 318l282 282l-282 282l212 212l282 -282l282 282l212 -212l-282 -282l282 -282l-212 -212l-282 282l-282 -282z" />
+<glyph unicode="&#xe015;" d="M23 693q0 200 142 342t342 142t342 -142t142 -342q0 -142 -78 -261l300 -300q7 -8 7 -18t-7 -18l-109 -109q-8 -7 -18 -7t-18 7l-300 300q-119 -78 -261 -78q-200 0 -342 142t-142 342zM176 693q0 -136 97 -233t234 -97t233.5 96.5t96.5 233.5t-96.5 233.5t-233.5 96.5 t-234 -97t-97 -233zM300 600v200h100v100h200v-100h100v-200h-100v-100h-200v100h-100z" />
+<glyph unicode="&#xe016;" d="M23 694q0 200 142 342t342 142t342 -142t142 -342q0 -141 -78 -262l300 -299q7 -7 7 -18t-7 -18l-109 -109q-8 -8 -18 -8t-18 8l-300 299q-120 -77 -261 -77q-200 0 -342 142t-142 342zM176 694q0 -136 97 -233t234 -97t233.5 97t96.5 233t-96.5 233t-233.5 97t-234 -97 t-97 -233zM300 601h400v200h-400v-200z" />
+<glyph unicode="&#xe017;" d="M23 600q0 183 105 331t272 210v-166q-103 -55 -165 -155t-62 -220q0 -177 125 -302t302 -125t302 125t125 302q0 120 -62 220t-165 155v166q167 -62 272 -210t105 -331q0 -118 -45.5 -224.5t-123 -184t-184 -123t-224.5 -45.5t-224.5 45.5t-184 123t-123 184t-45.5 224.5 zM500 750q0 -21 14.5 -35.5t35.5 -14.5h100q21 0 35.5 14.5t14.5 35.5v400q0 21 -14.5 35.5t-35.5 14.5h-100q-21 0 -35.5 -14.5t-14.5 -35.5v-400z" />
+<glyph unicode="&#xe018;" d="M100 1h200v300h-200v-300zM400 1v500h200v-500h-200zM700 1v800h200v-800h-200zM1000 1v1200h200v-1200h-200z" />
+<glyph unicode="&#xe019;" d="M26 601q0 -33 6 -74l151 -38l2 -6q14 -49 38 -93l3 -5l-80 -134q45 -59 105 -105l133 81l5 -3q45 -26 94 -39l5 -2l38 -151q40 -5 74 -5q27 0 74 5l38 151l6 2q46 13 93 39l5 3l134 -81q56 44 104 105l-80 134l3 5q24 44 39 93l1 6l152 38q5 40 5 74q0 28 -5 73l-152 38 l-1 6q-16 51 -39 93l-3 5l80 134q-44 58 -104 105l-134 -81l-5 3q-45 25 -93 39l-6 1l-38 152q-40 5 -74 5q-27 0 -74 -5l-38 -152l-5 -1q-50 -14 -94 -39l-5 -3l-133 81q-59 -47 -105 -105l80 -134l-3 -5q-25 -47 -38 -93l-2 -6l-151 -38q-6 -48 -6 -73zM385 601 q0 88 63 151t152 63t152 -63t63 -151q0 -89 -63 -152t-152 -63t-152 63t-63 152z" />
+<glyph unicode="&#xe020;" d="M100 1025v50q0 10 7.5 17.5t17.5 7.5h275v100q0 41 29.5 70.5t70.5 29.5h300q41 0 70.5 -29.5t29.5 -70.5v-100h275q10 0 17.5 -7.5t7.5 -17.5v-50q0 -11 -7 -18t-18 -7h-1050q-11 0 -18 7t-7 18zM200 100v800h900v-800q0 -41 -29.5 -71t-70.5 -30h-700q-41 0 -70.5 30 t-29.5 71zM300 100h100v700h-100v-700zM500 100h100v700h-100v-700zM500 1100h300v100h-300v-100zM700 100h100v700h-100v-700zM900 100h100v700h-100v-700z" />
+<glyph unicode="&#xe021;" d="M1 601l656 644l644 -644h-200v-600h-300v400h-300v-400h-300v600h-200z" />
+<glyph unicode="&#xe022;" d="M100 25v1150q0 11 7 18t18 7h475v-500h400v-675q0 -11 -7 -18t-18 -7h-850q-11 0 -18 7t-7 18zM700 800v300l300 -300h-300z" />
+<glyph unicode="&#xe023;" d="M4 600q0 162 80 299t217 217t299 80t299 -80t217 -217t80 -299t-80 -299t-217 -217t-299 -80t-299 80t-217 217t-80 299zM186 600q0 -171 121.5 -292.5t292.5 -121.5t292.5 121.5t121.5 292.5t-121.5 292.5t-292.5 121.5t-292.5 -121.5t-121.5 -292.5zM500 500v400h100 v-300h200v-100h-300z" />
+<glyph unicode="&#xe024;" d="M-100 0l431 1200h209l-21 -300h162l-20 300h208l431 -1200h-538l-41 400h-242l-40 -400h-539zM488 500h224l-27 300h-170z" />
+<glyph unicode="&#xe025;" d="M0 0v400h490l-290 300h200v500h300v-500h200l-290 -300h490v-400h-1100zM813 200h175v100h-175v-100z" />
+<glyph unicode="&#xe026;" d="M1 600q0 122 47.5 233t127.5 191t191 127.5t233 47.5t233 -47.5t191 -127.5t127.5 -191t47.5 -233t-47.5 -233t-127.5 -191t-191 -127.5t-233 -47.5t-233 47.5t-191 127.5t-127.5 191t-47.5 233zM188 600q0 -170 121 -291t291 -121t291 121t121 291t-121 291t-291 121 t-291 -121t-121 -291zM350 600h150v300h200v-300h150l-250 -300z" />
+<glyph unicode="&#xe027;" d="M4 600q0 162 80 299t217 217t299 80t299 -80t217 -217t80 -299t-80 -299t-217 -217t-299 -80t-299 80t-217 217t-80 299zM186 600q0 -171 121.5 -292.5t292.5 -121.5t292.5 121.5t121.5 292.5t-121.5 292.5t-292.5 121.5t-292.5 -121.5t-121.5 -292.5zM350 600l250 300 l250 -300h-150v-300h-200v300h-150z" />
+<glyph unicode="&#xe028;" d="M0 25v475l200 700h800q199 -700 200 -700v-475q0 -11 -7 -18t-18 -7h-1150q-11 0 -18 7t-7 18zM200 500h200l50 -200h300l50 200h200l-97 500h-606z" />
+<glyph unicode="&#xe029;" d="M4 600q0 162 80 299t217 217t299 80t299 -80t217 -217t80 -299t-80 -299t-217 -217t-299 -80t-299 80t-217 217t-80 299zM186 600q0 -172 121.5 -293t292.5 -121t292.5 121t121.5 293q0 171 -121.5 292.5t-292.5 121.5t-292.5 -121.5t-121.5 -292.5zM500 397v401 l297 -200z" />
+<glyph unicode="&#xe030;" d="M23 600q0 -118 45.5 -224.5t123 -184t184 -123t224.5 -45.5t224.5 45.5t184 123t123 184t45.5 224.5h-150q0 -177 -125 -302t-302 -125t-302 125t-125 302t125 302t302 125q136 0 246 -81l-146 -146h400v400l-145 -145q-157 122 -355 122q-118 0 -224.5 -45.5t-184 -123 t-123 -184t-45.5 -224.5z" />
+<glyph unicode="&#xe031;" d="M23 600q0 118 45.5 224.5t123 184t184 123t224.5 45.5q198 0 355 -122l145 145v-400h-400l147 147q-112 80 -247 80q-177 0 -302 -125t-125 -302h-150zM100 0v400h400l-147 -147q112 -80 247 -80q177 0 302 125t125 302h150q0 -118 -45.5 -224.5t-123 -184t-184 -123 t-224.5 -45.5q-198 0 -355 122z" />
+<glyph unicode="&#xe032;" d="M100 0h1100v1200h-1100v-1200zM200 100v900h900v-900h-900zM300 200v100h100v-100h-100zM300 400v100h100v-100h-100zM300 600v100h100v-100h-100zM300 800v100h100v-100h-100zM500 200h500v100h-500v-100zM500 400v100h500v-100h-500zM500 600v100h500v-100h-500z M500 800v100h500v-100h-500z" />
+<glyph unicode="&#xe033;" d="M0 100v600q0 41 29.5 70.5t70.5 29.5h100v200q0 82 59 141t141 59h300q82 0 141 -59t59 -141v-200h100q41 0 70.5 -29.5t29.5 -70.5v-600q0 -41 -29.5 -70.5t-70.5 -29.5h-900q-41 0 -70.5 29.5t-29.5 70.5zM400 800h300v150q0 21 -14.5 35.5t-35.5 14.5h-200 q-21 0 -35.5 -14.5t-14.5 -35.5v-150z" />
+<glyph unicode="&#xe034;" d="M100 0v1100h100v-1100h-100zM300 400q60 60 127.5 84t127.5 17.5t122 -23t119 -30t110 -11t103 42t91 120.5v500q-40 -81 -101.5 -115.5t-127.5 -29.5t-138 25t-139.5 40t-125.5 25t-103 -29.5t-65 -115.5v-500z" />
+<glyph unicode="&#xe035;" d="M0 275q0 -11 7 -18t18 -7h50q11 0 18 7t7 18v300q0 127 70.5 231.5t184.5 161.5t245 57t245 -57t184.5 -161.5t70.5 -231.5v-300q0 -11 7 -18t18 -7h50q11 0 18 7t7 18v300q0 116 -49.5 227t-131 192.5t-192.5 131t-227 49.5t-227 -49.5t-192.5 -131t-131 -192.5 t-49.5 -227v-300zM200 20v460q0 8 6 14t14 6h160q8 0 14 -6t6 -14v-460q0 -8 -6 -14t-14 -6h-160q-8 0 -14 6t-6 14zM800 20v460q0 8 6 14t14 6h160q8 0 14 -6t6 -14v-460q0 -8 -6 -14t-14 -6h-160q-8 0 -14 6t-6 14z" />
+<glyph unicode="&#xe036;" d="M0 400h300l300 -200v800l-300 -200h-300v-400zM688 459l141 141l-141 141l71 71l141 -141l141 141l71 -71l-141 -141l141 -141l-71 -71l-141 141l-141 -141z" />
+<glyph unicode="&#xe037;" d="M0 400h300l300 -200v800l-300 -200h-300v-400zM700 857l69 53q111 -135 111 -310q0 -169 -106 -302l-67 54q86 110 86 248q0 146 -93 257z" />
+<glyph unicode="&#xe038;" d="M0 401v400h300l300 200v-800l-300 200h-300zM702 858l69 53q111 -135 111 -310q0 -170 -106 -303l-67 55q86 110 86 248q0 145 -93 257zM889 951l7 -8q123 -151 123 -344q0 -189 -119 -339l-7 -8l81 -66l6 8q142 178 142 405q0 230 -144 408l-6 8z" />
+<glyph unicode="&#xe039;" d="M0 0h500v500h-200v100h-100v-100h-200v-500zM0 600h100v100h400v100h100v100h-100v300h-500v-600zM100 100v300h300v-300h-300zM100 800v300h300v-300h-300zM200 200v100h100v-100h-100zM200 900h100v100h-100v-100zM500 500v100h300v-300h200v-100h-100v-100h-200v100 h-100v100h100v200h-200zM600 0v100h100v-100h-100zM600 1000h100v-300h200v-300h300v200h-200v100h200v500h-600v-200zM800 800v300h300v-300h-300zM900 0v100h300v-100h-300zM900 900v100h100v-100h-100zM1100 200v100h100v-100h-100z" />
+<glyph unicode="&#xe040;" d="M0 200h100v1000h-100v-1000zM100 0v100h300v-100h-300zM200 200v1000h100v-1000h-100zM500 0v91h100v-91h-100zM500 200v1000h200v-1000h-200zM700 0v91h100v-91h-100zM800 200v1000h100v-1000h-100zM900 0v91h200v-91h-200zM1000 200v1000h200v-1000h-200z" />
+<glyph unicode="&#xe041;" d="M1 700v475q0 10 7.5 17.5t17.5 7.5h474l700 -700l-500 -500zM148 953q0 -42 29 -71q30 -30 71.5 -30t71.5 30q29 29 29 71t-29 71q-30 30 -71.5 30t-71.5 -30q-29 -29 -29 -71z" />
+<glyph unicode="&#xe042;" d="M2 700v475q0 11 7 18t18 7h474l700 -700l-500 -500zM148 953q0 -42 30 -71q29 -30 71 -30t71 30q30 29 30 71t-30 71q-29 30 -71 30t-71 -30q-30 -29 -30 -71zM701 1200h100l700 -700l-500 -500l-50 50l450 450z" />
+<glyph unicode="&#xe043;" d="M100 0v1025l175 175h925v-1000l-100 -100v1000h-750l-100 -100h750v-1000h-900z" />
+<glyph unicode="&#xe044;" d="M200 0l450 444l450 -443v1150q0 20 -14.5 35t-35.5 15h-800q-21 0 -35.5 -15t-14.5 -35v-1151z" />
+<glyph unicode="&#xe045;" d="M0 100v700h200l100 -200h600l100 200h200v-700h-200v200h-800v-200h-200zM253 829l40 -124h592l62 124l-94 346q-2 11 -10 18t-18 7h-450q-10 0 -18 -7t-10 -18zM281 24l38 152q2 10 11.5 17t19.5 7h500q10 0 19.5 -7t11.5 -17l38 -152q2 -10 -3.5 -17t-15.5 -7h-600 q-10 0 -15.5 7t-3.5 17z" />
+<glyph unicode="&#xe046;" d="M0 200q0 -41 29.5 -70.5t70.5 -29.5h1000q41 0 70.5 29.5t29.5 70.5v600q0 41 -29.5 70.5t-70.5 29.5h-150q-4 8 -11.5 21.5t-33 48t-53 61t-69 48t-83.5 21.5h-200q-41 0 -82 -20.5t-70 -50t-52 -59t-34 -50.5l-12 -20h-150q-41 0 -70.5 -29.5t-29.5 -70.5v-600z M356 500q0 100 72 172t172 72t172 -72t72 -172t-72 -172t-172 -72t-172 72t-72 172zM494 500q0 -44 31 -75t75 -31t75 31t31 75t-31 75t-75 31t-75 -31t-31 -75zM900 700v100h100v-100h-100z" />
+<glyph unicode="&#xe047;" d="M53 0h365v66q-41 0 -72 11t-49 38t1 71l92 234h391l82 -222q16 -45 -5.5 -88.5t-74.5 -43.5v-66h417v66q-34 1 -74 43q-18 19 -33 42t-21 37l-6 13l-385 998h-93l-399 -1006q-24 -48 -52 -75q-12 -12 -33 -25t-36 -20l-15 -7v-66zM416 521l178 457l46 -140l116 -317h-340 z" />
+<glyph unicode="&#xe048;" d="M100 0v89q41 7 70.5 32.5t29.5 65.5v827q0 28 -1 39.5t-5.5 26t-15.5 21t-29 14t-49 14.5v70h471q120 0 213 -88t93 -228q0 -55 -11.5 -101.5t-28 -74t-33.5 -47.5t-28 -28l-12 -7q8 -3 21.5 -9t48 -31.5t60.5 -58t47.5 -91.5t21.5 -129q0 -84 -59 -156.5t-142 -111 t-162 -38.5h-500zM400 200h161q89 0 153 48.5t64 132.5q0 90 -62.5 154.5t-156.5 64.5h-159v-400zM400 700h139q76 0 130 61.5t54 138.5q0 82 -84 130.5t-239 48.5v-379z" />
+<glyph unicode="&#xe049;" d="M200 0v57q77 7 134.5 40.5t65.5 80.5l173 849q10 56 -10 74t-91 37q-6 1 -10.5 2.5t-9.5 2.5v57h425l2 -57q-33 -8 -62 -25.5t-46 -37t-29.5 -38t-17.5 -30.5l-5 -12l-128 -825q-10 -52 14 -82t95 -36v-57h-500z" />
+<glyph unicode="&#xe050;" d="M-75 200h75v800h-75l125 167l125 -167h-75v-800h75l-125 -167zM300 900v300h150h700h150v-300h-50q0 29 -8 48.5t-18.5 30t-33.5 15t-39.5 5.5t-50.5 1h-200v-850l100 -50v-100h-400v100l100 50v850h-200q-34 0 -50.5 -1t-40 -5.5t-33.5 -15t-18.5 -30t-8.5 -48.5h-49z " />
+<glyph unicode="&#xe051;" d="M33 51l167 125v-75h800v75l167 -125l-167 -125v75h-800v-75zM100 901v300h150h700h150v-300h-50q0 29 -8 48.5t-18 30t-33.5 15t-40 5.5t-50.5 1h-200v-650l100 -50v-100h-400v100l100 50v650h-200q-34 0 -50.5 -1t-39.5 -5.5t-33.5 -15t-18.5 -30t-8 -48.5h-50z" />
+<glyph unicode="&#xe052;" d="M0 50q0 -20 14.5 -35t35.5 -15h1100q21 0 35.5 15t14.5 35v100q0 21 -14.5 35.5t-35.5 14.5h-1100q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM0 350q0 -20 14.5 -35t35.5 -15h800q21 0 35.5 15t14.5 35v100q0 21 -14.5 35.5t-35.5 14.5h-800q-21 0 -35.5 -14.5t-14.5 -35.5 v-100zM0 650q0 -20 14.5 -35t35.5 -15h1000q21 0 35.5 15t14.5 35v100q0 21 -14.5 35.5t-35.5 14.5h-1000q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM0 950q0 -20 14.5 -35t35.5 -15h600q21 0 35.5 15t14.5 35v100q0 21 -14.5 35.5t-35.5 14.5h-600q-21 0 -35.5 -14.5 t-14.5 -35.5v-100z" />
+<glyph unicode="&#xe053;" d="M0 50q0 -20 14.5 -35t35.5 -15h1100q21 0 35.5 15t14.5 35v100q0 21 -14.5 35.5t-35.5 14.5h-1100q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM0 650q0 -20 14.5 -35t35.5 -15h1100q21 0 35.5 15t14.5 35v100q0 21 -14.5 35.5t-35.5 14.5h-1100q-21 0 -35.5 -14.5t-14.5 -35.5 v-100zM200 350q0 -20 14.5 -35t35.5 -15h700q21 0 35.5 15t14.5 35v100q0 21 -14.5 35.5t-35.5 14.5h-700q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM200 950q0 -20 14.5 -35t35.5 -15h700q21 0 35.5 15t14.5 35v100q0 21 -14.5 35.5t-35.5 14.5h-700q-21 0 -35.5 -14.5 t-14.5 -35.5v-100z" />
+<glyph unicode="&#xe054;" d="M0 50v100q0 21 14.5 35.5t35.5 14.5h1100q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-1100q-21 0 -35.5 15t-14.5 35zM100 650v100q0 21 14.5 35.5t35.5 14.5h1000q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-1000q-21 0 -35.5 15 t-14.5 35zM300 350v100q0 21 14.5 35.5t35.5 14.5h800q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-800q-21 0 -35.5 15t-14.5 35zM500 950v100q0 21 14.5 35.5t35.5 14.5h600q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-600 q-21 0 -35.5 15t-14.5 35z" />
+<glyph unicode="&#xe055;" d="M0 50v100q0 21 14.5 35.5t35.5 14.5h1100q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-1100q-21 0 -35.5 15t-14.5 35zM0 350v100q0 21 14.5 35.5t35.5 14.5h1100q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-1100q-21 0 -35.5 15 t-14.5 35zM0 650v100q0 21 14.5 35.5t35.5 14.5h1100q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-1100q-21 0 -35.5 15t-14.5 35zM0 950v100q0 21 14.5 35.5t35.5 14.5h1100q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-1100 q-21 0 -35.5 15t-14.5 35z" />
+<glyph unicode="&#xe056;" d="M0 50v100q0 21 14.5 35.5t35.5 14.5h100q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-100q-21 0 -35.5 15t-14.5 35zM0 350v100q0 21 14.5 35.5t35.5 14.5h100q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-100q-21 0 -35.5 15 t-14.5 35zM0 650v100q0 21 14.5 35.5t35.5 14.5h100q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-100q-21 0 -35.5 15t-14.5 35zM0 950v100q0 21 14.5 35.5t35.5 14.5h100q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-100q-21 0 -35.5 15 t-14.5 35zM300 50v100q0 21 14.5 35.5t35.5 14.5h800q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-800q-21 0 -35.5 15t-14.5 35zM300 350v100q0 21 14.5 35.5t35.5 14.5h800q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-800 q-21 0 -35.5 15t-14.5 35zM300 650v100q0 21 14.5 35.5t35.5 14.5h800q21 0 35.5 -14.5t14.5 -35.5v-100q0 -20 -14.5 -35t-35.5 -15h-800q-21 0 -35.5 15t-14.5 35zM300 950v100q0 21 14.5 35.5t35.5 14.5h800q21 0 35.5 -14.5t14.5 -35.5v-10
 0q0 -20 -14.5 -35t-35.5 -15 h-800q-21 0 -35.5 15t-14.5 35z" />
+<glyph unicode="&#xe057;" d="M-101 500v100h201v75l166 -125l-166 -125v75h-201zM300 0h100v1100h-100v-1100zM500 50q0 -20 14.5 -35t35.5 -15h600q20 0 35 15t15 35v100q0 21 -15 35.5t-35 14.5h-600q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM500 350q0 -20 14.5 -35t35.5 -15h300q20 0 35 15t15 35 v100q0 21 -15 35.5t-35 14.5h-300q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM500 650q0 -20 14.5 -35t35.5 -15h500q20 0 35 15t15 35v100q0 21 -15 35.5t-35 14.5h-500q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM500 950q0 -20 14.5 -35t35.5 -15h100q20 0 35 15t15 35v100 q0 21 -15 35.5t-35 14.5h-100q-21 0 -35.5 -14.5t-14.5 -35.5v-100z" />
+<glyph unicode="&#xe058;" d="M1 50q0 -20 14.5 -35t35.5 -15h600q20 0 35 15t15 35v100q0 21 -15 35.5t-35 14.5h-600q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM1 350q0 -20 14.5 -35t35.5 -15h300q20 0 35 15t15 35v100q0 21 -15 35.5t-35 14.5h-300q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM1 650 q0 -20 14.5 -35t35.5 -15h500q20 0 35 15t15 35v100q0 21 -15 35.5t-35 14.5h-500q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM1 950q0 -20 14.5 -35t35.5 -15h100q20 0 35 15t15 35v100q0 21 -15 35.5t-35 14.5h-100q-21 0 -35.5 -14.5t-14.5 -35.5v-100zM801 0v1100h100v-1100 h-100zM934 550l167 -125v75h200v100h-200v75z" />
+<glyph unicode="&#xe059;" d="M0 275v650q0 31 22 53t53 22h750q31 0 53 -22t22 -53v-650q0 -31 -22 -53t-53 -22h-750q-31 0 -53 22t-22 53zM900 600l300 300v-600z" />
+<glyph unicode="&#xe060;" d="M0 44v1012q0 18 13 31t31 13h1112q19 0 31.5 -13t12.5 -31v-1012q0 -18 -12.5 -31t-31.5 -13h-1112q-18 0 -31 13t-13 31zM100 263l247 182l298 -131l-74 156l293 318l236 -288v500h-1000v-737zM208 750q0 56 39 95t95 39t95 -39t39 -95t-39 -95t-95 -39t-95 39t-39 95z " />
+<glyph unicode="&#xe062;" d="M148 745q0 124 60.5 231.5t165 172t226.5 64.5q123 0 227 -63t164.5 -169.5t60.5 -229.5t-73 -272q-73 -114 -166.5 -237t-150.5 -189l-57 -66q-10 9 -27 26t-66.5 70.5t-96 109t-104 135.5t-100.5 155q-63 139 -63 262zM342 772q0 -107 75.5 -182.5t181.5 -75.5 q107 0 182.5 75.5t75.5 182.5t-75.5 182t-182.5 75t-182 -75.5t-75 -181.5z" />
+<glyph unicode="&#xe063;" d="M1 600q0 122 47.5 233t127.5 191t191 127.5t233 47.5t233 -47.5t191 -127.5t127.5 -191t47.5 -233t-47.5 -233t-127.5 -191t-191 -127.5t-233 -47.5t-233 47.5t-191 127.5t-127.5 191t-47.5 233zM173 600q0 -177 125.5 -302t301.5 -125v854q-176 0 -301.5 -125 t-125.5 -302z" />
+<glyph unicode="&#xe064;" d="M117 406q0 94 34 186t88.5 172.5t112 159t115 177t87.5 194.5q21 -71 57.5 -142.5t76 -130.5t83 -118.5t82 -117t70 -116t50 -125.5t18.5 -136q0 -89 -39 -165.5t-102 -126.5t-140 -79.5t-156 -33.5q-114 6 -211.5 53t-161.5 138.5t-64 210.5zM243 414q14 -82 59.5 -136 t136.5 -80l16 98q-7 6 -18 17t-34 48t-33 77q-15 73 -14 143.5t10 122.5l9 51q-92 -110 -119.5 -185t-12.5 -156z" />
+<glyph unicode="&#xe065;" d="M0 400v300q0 165 117.5 282.5t282.5 117.5q366 -6 397 -14l-186 -186h-311q-41 0 -70.5 -29.5t-29.5 -70.5v-500q0 -41 29.5 -70.5t70.5 -29.5h500q41 0 70.5 29.5t29.5 70.5v125l200 200v-225q0 -165 -117.5 -282.5t-282.5 -117.5h-300q-165 0 -282.5 117.5 t-117.5 282.5zM436 341l161 50l412 412l-114 113l-405 -405zM995 1015l113 -113l113 113l-21 85l-92 28z" />
+<glyph unicode="&#xe066;" d="M0 400v300q0 165 117.5 282.5t282.5 117.5h261l2 -80q-133 -32 -218 -120h-145q-41 0 -70.5 -29.5t-29.5 -70.5v-500q0 -41 29.5 -70.5t70.5 -29.5h500q41 0 70.5 29.5t29.5 70.5l200 153v-53q0 -165 -117.5 -282.5t-282.5 -117.5h-300q-165 0 -282.5 117.5t-117.5 282.5 zM423 524q30 38 81.5 64t103 35.5t99 14t77.5 3.5l29 -1v-209l360 324l-359 318v-216q-7 0 -19 -1t-48 -8t-69.5 -18.5t-76.5 -37t-76.5 -59t-62 -88t-39.5 -121.5z" />
+<glyph unicode="&#xe067;" d="M0 400v300q0 165 117.5 282.5t282.5 117.5h300q60 0 127 -23l-178 -177h-349q-41 0 -70.5 -29.5t-29.5 -70.5v-500q0 -41 29.5 -70.5t70.5 -29.5h500q41 0 70.5 29.5t29.5 70.5v69l200 200v-169q0 -165 -117.5 -282.5t-282.5 -117.5h-300q-165 0 -282.5 117.5 t-117.5 282.5zM342 632l283 -284l566 567l-136 137l-430 -431l-147 147z" />
+<glyph unicode="&#xe068;" d="M0 603l300 296v-198h200v200h-200l300 300l295 -300h-195v-200h200v198l300 -296l-300 -300v198h-200v-200h195l-295 -300l-300 300h200v200h-200v-198z" />
+<glyph unicode="&#xe069;" d="M200 50v1000q0 21 14.5 35.5t35.5 14.5h100q21 0 35.5 -14.5t14.5 -35.5v-437l500 487v-1100l-500 488v-438q0 -21 -14.5 -35.5t-35.5 -14.5h-100q-21 0 -35.5 14.5t-14.5 35.5z" />
+<glyph unicode="&#xe070;" d="M0 50v1000q0 21 14.5 35.5t35.5 14.5h100q21 0 35.5 -14.5t14.5 -35.5v-437l500 487v-487l500 487v-1100l-500 488v-488l-500 488v-438q0 -21 -14.5 -35.5t-35.5 -14.5h-100q-21 0 -35.5 14.5t-14.5 35.5z" />
+<glyph unicode="&#xe071;" d="M136 550l564 550v-487l500 487v-1100l-500 488v-488z" />
+<glyph unicode="&#xe072;" d="M200 0l900 550l-900 550v-1100z" />
+<glyph unicode="&#xe073;" d="M200 150q0 -21 14.5 -35.5t35.5 -14.5h200q21 0 35.5 14.5t14.5 35.5v800q0 21 -14.5 35.5t-35.5 14.5h-200q-21 0 -35.5 -14.5t-14.5 -35.5v-800zM600 150q0 -21 14.5 -35.5t35.5 -14.5h200q21 0 35.5 14.5t14.5 35.5v800q0 21 -14.5 35.5t-35.5 14.5h-200 q-21 0 -35.5 -14.5t-14.5 -35.5v-800z" />
+<glyph unicode="&#xe074;" d="M200 150q0 -20 14.5 -35t35.5 -15h800q21 0 35.5 15t14.5 35v800q0 21 -14.5 35.5t-35.5 14.5h-800q-21 0 -35.5 -14.5t-14.5 -35.5v-800z" />
+<glyph unicode="&#xe075;" d="M0 0v1100l500 -487v487l564 -550l-564 -550v488z" />
+<glyph unicode="&#xe076;" d="M0 0v1100l500 -487v487l500 -487v437q0 21 14.5 35.5t35.5 14.5h100q21 0 35.5 -14.5t14.5 -35.5v-1000q0 -21 -14.5 -35.5t-35.5 -14.5h-100q-21 0 -35.5 14.5t-14.5 35.5v438l-500 -488v488z" />
+<glyph unicode="&#xe077;" d="M300 0v1100l500 -487v437q0 21 14.5 35.5t35.5 14.5h100q21 0 35.5 -14.5t14.5 -35.5v-1000q0 -21 -14.5 -35.5t-35.5 -14.5h-100q-21 0 -35.5 14.5t-14.5 35.5v438z" />
+<glyph unicode="&#xe078;" d="M100 250v100q0 21 14.5 35.5t35.5 14.5h1000q21 0 35.5 -14.5t14.5 -35.5v-100q0 -21 -14.5 -35.5t-35.5 -14.5h-1000q-21 0 -35.5 14.5t-14.5 35.5zM100 500h1100l-550 564z" />
+<glyph unicode="&#xe079;" d="M185 599l592 -592l240 240l-353 353l353 353l-240 240z" />
+<glyph unicode="&#xe080;" d="M272 194l353 353l-353 353l241 240l572 -571l21 -22l-1 -1v-1l-592 -591z" />
+<glyph unicode="&#xe081;" d="M3 600q0 162 80 299.5t217.5 217.5t299.5 80t299.5 -80t217.5 -217.5t80 -299.5t-80 -300t-217.5 -218t-299.5 -80t-299.5 80t-217.5 218t-80 300zM300 500h200v-200h200v200h200v200h-200v200h-200v-200h-200v-200z" />
+<glyph unicode="&#xe082;" d="M3 600q0 162 80 299.5t217.5 217.5t299.5 80t299.5 -80t217.5 -217.5t80 -299.5t-80 -300t-217.5 -218t-299.5 -80t-299.5 80t-217.5 218t-80 300zM300 500h600v200h-600v-200z" />
+<glyph unicode="&#xe083;" d="M3 600q0 162 80 299.5t217.5 217.5t299.5 80t299.5 -80t217.5 -217.5t80 -299.5t-80 -300t-217.5 -218t-299.5 -80t-299.5 80t-217.5 218t-80 300zM246 459l213 -213l141 142l141 -142l213 213l-142 141l142 141l-213 212l-141 -141l-141 142l-212 -213l141 -141z" />
+<glyph unicode="&#xe084;" d="M3 600q0 162 80 299.5t217.5 217.5t299.5 80t299.5 -80t217.5 -217.5t80 -299.5t-80 -299.5t-217.5 -217.5t-299.5 -80t-299.5 80t-217.5 217.5t-80 299.5zM270 551l276 -277l411 411l-175 174l-236 -236l-102 102z" />
+<glyph unicode="&#xe085;" d="M3 600q0 162 80 299.5t217.5 217.5t299.5 80t299.5 -80t217.5 -217.5t80 -299.5t-80 -300t-217.5 -218t-299.5 -80t-299.5 80t-217.5 218t-80 300zM363 700h144q4 0 11.5 -1t11 -1t6.5 3t3 9t1 11t3.5 8.5t3.5 6t5.5 4t6.5 2.5t9 1.5t9 0.5h11.5h12.5q19 0 30 -10t11 -26 q0 -22 -4 -28t-27 -22q-5 -1 -12.5 -3t-27 -13.5t-34 -27t-26.5 -46t-11 -68.5h200q5 3 14 8t31.5 25.5t39.5 45.5t31 69t14 94q0 51 -17.5 89t-42 58t-58.5 32t-58.5 15t-51.5 3q-105 0 -172 -56t-67 -183zM500 300h200v100h-200v-100z" />
+<glyph unicode="&#xe086;" d="M3 600q0 162 80 299.5t217.5 217.5t299.5 80t299.5 -80t217.5 -217.5t80 -299.5t-80 -300t-217.5 -218t-299.5 -80t-299.5 80t-217.5 218t-80 300zM400 300h400v100h-100v300h-300v-100h100v-200h-100v-100zM500 800h200v100h-200v-100z" />
+<glyph unicode="&#xe087;" d="M0 500v200h194q15 60 36 104.5t55.5 86t88 69t126.5 40.5v200h200v-200q54 -20 113 -60t112.5 -105.5t71.5 -134.5h203v-200h-203q-25 -102 -116.5 -186t-180.5 -117v-197h-200v197q-140 27 -208 102.5t-98 200.5h-194zM290 500q24 -73 79.5 -127.5t130.5 -78.5v206h200 v-206q149 48 201 206h-201v200h200q-25 74 -76 127.5t-124 76.5v-204h-200v203q-75 -24 -130 -77.5t-79 -125.5h209v-200h-210z" />
+<glyph unicode="&#xe088;" d="M4 600q0 162 80 299t217 217t299 80t299 -80t217 -217t80 -299t-80 -299t-217 -217t-299 -80t-299 80t-217 217t-80 299zM186 600q0 -171 121.5 -292.5t292.5 -121.5t292.5 121.5t121.5 292.5t-121.5 292.5t-292.5 121.5t-292.5 -121.5t-121.5 -292.5zM356 465l135 135 l-135 135l109 109l135 -135l135 135l109 -109l-135 -135l135 -135l-109 -109l-135 135l-135 -135z" />
+<glyph unicode="&#xe089;" d="M4 600q0 162 80 299t217 217t299 80t299 -80t217 -217t80 -299t-80 -299t-217 -217t-299 -80t-299 80t-217 217t-80 299zM186 600q0 -171 121.5 -292.5t292.5 -121.5t292.5 121.5t121.5 292.5t-121.5 292.5t-292.5 121.5t-292.5 -121.5t-121.5 -292.5zM322 537l141 141 l87 -87l204 205l142 -142l-346 -345z" />
+<glyph unicode="&#xe090;" d="M4 600q0 162 80 299t217 217t299 80t299 -80t217 -217t80 -299t-80 -299t-217 -217t-299 -80t-299 80t-217 217t-80 299zM186 600q0 -115 62 -215l568 567q-100 62 -216 62q-171 0 -292.5 -121.5t-121.5 -292.5zM391 245q97 -59 209 -59q171 0 292.5 121.5t121.5 292.5 q0 112 -59 209z" />
+<glyph unicode="&#xe091;" d="M0 547l600 453v-300h600v-300h-600v-301z" />
+<glyph unicode="&#xe092;" d="M0 400v300h600v300l600 -453l-600 -448v301h-600z" />
+<glyph unicode="&#xe093;" d="M204 600l450 600l444 -600h-298v-600h-300v600h-296z" />
+<glyph unicode="&#xe094;" d="M104 600h296v600h300v-600h298l-449 -600z" />
+<glyph unicode="&#xe095;" d="M0 200q6 132 41 238.5t103.5 193t184 138t271.5 59.5v271l600 -453l-600 -448v301q-95 -2 -183 -20t-170 -52t-147 -92.5t-100 -135.5z" />
+<glyph unicode="&#xe096;" d="M0 0v400l129 -129l294 294l142 -142l-294 -294l129 -129h-400zM635 777l142 -142l294 294l129 -129v400h-400l129 -129z" />
+<glyph unicode="&#xe097;" d="M34 176l295 295l-129 129h400v-400l-129 130l-295 -295zM600 600v400l129 -129l295 295l142 -141l-295 -295l129 -130h-400z" />
+<glyph unicode="&#xe101;" d="M23 600q0 118 45.5 224.5t123 184t184 123t224.5 45.5t224.5 -45.5t184 -123t123 -184t45.5 -224.5t-45.5 -224.5t-123 -184t-184 -123t-224.5 -45.5t-224.5 45.5t-184 123t-123 184t-45.5 224.5zM456 851l58 -302q4 -20 21.5 -34.5t37.5 -14.5h54q20 0 37.5 14.5 t21.5 34.5l58 302q4 20 -8 34.5t-33 14.5h-207q-20 0 -32 -14.5t-8 -34.5zM500 300h200v100h-200v-100z" />
+<glyph unicode="&#xe102;" d="M0 800h100v-200h400v300h200v-300h400v200h100v100h-111v6t-1 15t-3 18l-34 172q-11 39 -41.5 63t-69.5 24q-32 0 -61 -17l-239 -144q-22 -13 -40 -35q-19 24 -40 36l-238 144q-33 18 -62 18q-39 0 -69.5 -23t-40.5 -61l-35 -177q-2 -8 -3 -18t-1 -15v-6h-111v-100z M100 0h400v400h-400v-400zM200 900q-3 0 14 48t35 96l18 47l214 -191h-281zM700 0v400h400v-400h-400zM731 900l202 197q5 -12 12 -32.5t23 -64t25 -72t7 -28.5h-269z" />
+<glyph unicode="&#xe103;" d="M0 -22v143l216 193q-9 53 -13 83t-5.5 94t9 113t38.5 114t74 124q47 60 99.5 102.5t103 68t127.5 48t145.5 37.5t184.5 43.5t220 58.5q0 -189 -22 -343t-59 -258t-89 -181.5t-108.5 -120t-122 -68t-125.5 -30t-121.5 -1.5t-107.5 12.5t-87.5 17t-56.5 7.5l-99 -55z M238.5 300.5q19.5 -6.5 86.5 76.5q55 66 367 234q70 38 118.5 69.5t102 79t99 111.5t86.5 148q22 50 24 60t-6 19q-7 5 -17 5t-26.5 -14.5t-33.5 -39.5q-35 -51 -113.5 -108.5t-139.5 -89.5l-61 -32q-369 -197 -458 -401q-48 -111 -28.5 -117.5z" />
+<glyph unicode="&#xe104;" d="M111 408q0 -33 5 -63q9 -56 44 -119.5t105 -108.5q31 -21 64 -16t62 23.5t57 49.5t48 61.5t35 60.5q32 66 39 184.5t-13 157.5q79 -80 122 -164t26 -184q-5 -33 -20.5 -69.5t-37.5 -80.5q-10 -19 -14.5 -29t-12 -26t-9 -23.5t-3 -19t2.5 -15.5t11 -9.5t19.5 -5t30.5 2.5 t42 8q57 20 91 34t87.5 44.5t87 64t65.5 88.5t47 122q38 172 -44.5 341.5t-246.5 278.5q22 -44 43 -129q39 -159 -32 -154q-15 2 -33 9q-79 33 -120.5 100t-44 175.5t48.5 257.5q-13 -8 -34 -23.5t-72.5 -66.5t-88.5 -105.5t-60 -138t-8 -166.5q2 -12 8 -41.5t8 -43t6 -39.5 t3.5 -39.5t-1 -33.5t-6 -31.5t-13.5 -24t-21 -20.5t-31 -12q-38 -10 -67 13t-40.5 61.5t-15 81.5t10.5 75q-52 -46 -83.5 -101t-39 -107t-7.5 -85z" />
+<glyph unicode="&#xe105;" d="M-61 600l26 40q6 10 20 30t49 63.5t74.5 85.5t97 90t116.5 83.5t132.5 59t145.5 23.5t145.5 -23.5t132.5 -59t116.5 -83.5t97 -90t74.5 -85.5t49 -63.5t20 -30l26 -40l-26 -40q-6 -10 -20 -30t-49 -63.5t-74.5 -85.5t-97 -90t-116.5 -83.5t-132.5 -59t-145.5 -23.5 t-145.5 23.5t-132.5 59t-116.5 83.5t-97 90t-74.5 85.5t-49 63.5t-20 30zM120 600q7 -10 40.5 -58t56 -78.5t68 -77.5t87.5 -75t103 -49.5t125 -21.5t123.5 20t100.5 45.5t85.5 71.5t66.5 75.5t58 81.5t47 66q-1 1 -28.5 37.5t-42 55t-43.5 53t-57.5 63.5t-58.5 54 q49 -74 49 -163q0 -124 -88 -212t-212 -88t-212 88t-88 212q0 85 46 158q-102 -87 -226 -258zM377 656q49 -124 154 -191l105 105q-37 24 -75 72t-57 84l-20 36z" />
+<glyph unicode="&#xe106;" d="M-61 600l26 40q6 10 20 30t49 63.5t74.5 85.5t97 90t116.5 83.5t132.5 59t145.5 23.5q61 0 121 -17l37 142h148l-314 -1200h-148l37 143q-82 21 -165 71.5t-140 102t-109.5 112t-72 88.5t-29.5 43zM120 600q210 -282 393 -336l37 141q-107 18 -178.5 101.5t-71.5 193.5 q0 85 46 158q-102 -87 -226 -258zM377 656q49 -124 154 -191l47 47l23 87q-30 28 -59 69t-44 68l-14 26zM780 161l38 145q22 15 44.5 34t46 44t40.5 44t41 50.5t33.5 43.5t33 44t24.5 34q-97 127 -140 175l39 146q67 -54 131.5 -125.5t87.5 -103.5t36 -52l26 -40l-26 -40 q-7 -12 -25.5 -38t-63.5 -79.5t-95.5 -102.5t-124 -100t-146.5 -79z" />
+<glyph unicode="&#xe107;" d="M-97.5 34q13.5 -34 50.5 -34h1294q37 0 50.5 35.5t-7.5 67.5l-642 1056q-20 33 -48 36t-48 -29l-642 -1066q-21 -32 -7.5 -66zM155 200l445 723l445 -723h-345v100h-200v-100h-345zM500 600l100 -300l100 300v100h-200v-100z" />
+<glyph unicode="&#xe108;" d="M100 262v41q0 20 11 44.5t26 38.5l363 325v339q0 62 44 106t106 44t106 -44t44 -106v-339l363 -325q15 -14 26 -38.5t11 -44.5v-41q0 -20 -12 -26.5t-29 5.5l-359 249v-263q100 -91 100 -113v-64q0 -21 -13 -29t-32 1l-94 78h-222l-94 -78q-19 -9 -32 -1t-13 29v64 q0 22 100 113v263l-359 -249q-17 -12 -29 -5.5t-12 26.5z" />
+<glyph unicode="&#xe109;" d="M0 50q0 -20 14.5 -35t35.5 -15h1000q21 0 35.5 15t14.5 35v750h-1100v-750zM0 900h1100v150q0 21 -14.5 35.5t-35.5 14.5h-150v100h-100v-100h-500v100h-100v-100h-150q-21 0 -35.5 -14.5t-14.5 -35.5v-150zM100 100v100h100v-100h-100zM100 300v100h100v-100h-100z M100 500v100h100v-100h-100zM300 100v100h100v-100h-100zM300 300v100h100v-100h-100zM300 500v100h100v-100h-100zM500 100v100h100v-100h-100zM500 300v100h100v-100h-100zM500 500v100h100v-100h-100zM700 100v100h100v-100h-100zM700 300v100h100v-100h-100zM700 500 v100h100v-100h-100zM900 100v100h100v-100h-100zM900 300v100h100v-100h-100zM900 500v100h100v-100h-100z" />
+<glyph unicode="&#xe110;" d="M0 200v200h259l600 600h241v198l300 -295l-300 -300v197h-159l-600 -600h-341zM0 800h259l122 -122l141 142l-181 180h-341v-200zM678 381l141 142l122 -123h159v198l300 -295l-300 -300v197h-241z" />
+<glyph unicode="&#xe111;" d="M0 400v600q0 41 29.5 70.5t70.5 29.5h1000q41 0 70.5 -29.5t29.5 -70.5v-600q0 -41 -29.5 -70.5t-70.5 -29.5h-596l-304 -300v300h-100q-41 0 -70.5 29.5t-29.5 70.5z" />
+<glyph unicode="&#xe112;" d="M100 600v200h300v-250q0 -113 6 -145q17 -92 102 -117q39 -11 92 -11q37 0 66.5 5.5t50 15.5t36 24t24 31.5t14 37.5t7 42t2.5 45t0 47v25v250h300v-200q0 -42 -3 -83t-15 -104t-31.5 -116t-58 -109.5t-89 -96.5t-129 -65.5t-174.5 -25.5t-174.5 25.5t-129 65.5t-89 96.5 t-58 109.5t-31.5 116t-15 104t-3 83zM100 900v300h300v-300h-300zM800 900v300h300v-300h-300z" />
+<glyph unicode="&#xe113;" d="M-30 411l227 -227l352 353l353 -353l226 227l-578 579z" />
+<glyph unicode="&#xe114;" d="M70 797l580 -579l578 579l-226 227l-353 -353l-352 353z" />
+<glyph unicode="&#xe115;" d="M-198 700l299 283l300 -283h-203v-400h385l215 -200h-800v600h-196zM402 1000l215 -200h381v-400h-198l299 -283l299 283h-200v600h-796z" />
+<glyph unicode="&#xe116;" d="M18 939q-5 24 10 42q14 19 39 19h896l38 162q5 17 18.5 27.5t30.5 10.5h94q20 0 35 -14.5t15 -35.5t-15 -35.5t-35 -14.5h-54l-201 -961q-2 -4 -6 -10.5t-19 -17.5t-33 -11h-31v-50q0 -20 -14.5 -35t-35.5 -15t-35.5 15t-14.5 35v50h-300v-50q0 -20 -14.5 -35t-35.5 -15 t-35.5 15t-14.5 35v50h-50q-21 0 -35.5 15t-14.5 35q0 21 14.5 35.5t35.5 14.5h535l48 200h-633q-32 0 -54.5 21t-27.5 43z" />
+<glyph unicode="&#xe117;" d="M0 0v800h1200v-800h-1200zM0 900v100h200q0 41 29.5 70.5t70.5 29.5h300q41 0 70.5 -29.5t29.5 -70.5h500v-100h-1200z" />
+<glyph unicode="&#xe118;" d="M1 0l300 700h1200l-300 -700h-1200zM1 400v600h200q0 41 29.5 70.5t70.5 29.5h300q41 0 70.5 -29.5t29.5 -70.5h500v-200h-1000z" />
+<glyph unicode="&#xe119;" d="M302 300h198v600h-198l298 300l298 -300h-198v-600h198l-298 -300z" />
+<glyph unicode="&#xe120;" d="M0 600l300 298v-198h600v198l300 -298l-300 -297v197h-600v-197z" />
+<glyph unicode="&#xe121;" d="M0 100v100q0 41 29.5 70.5t70.5 29.5h1000q41 0 70.5 -29.5t29.5 -70.5v-100q0 -41 -29.5 -70.5t-70.5 -29.5h-1000q-41 0 -70.5 29.5t-29.5 70.5zM31 400l172 739q5 22 23 41.5t38 19.5h672q19 0 37.5 -22.5t23.5 -45.5l172 -732h-1138zM800 100h100v100h-100v-100z M1000 100h100v100h-100v-100z" />
+<glyph unicode="&#xe122;" d="M-101 600v50q0 24 25 49t50 38l25 13v-250l-11 5.5t-24 14t-30 21.5t-24 27.5t-11 31.5zM99 500v250v5q0 13 0.5 18.5t2.5 13t8 10.5t15 3h200l675 250v-850l-675 200h-38l47 -276q2 -12 -3 -17.5t-11 -6t-21 -0.5h-8h-83q-20 0 -34.5 14t-18.5 35q-56 337 -56 351z M1100 200v850q0 21 14.5 35.5t35.5 14.5q20 0 35 -14.5t15 -35.5v-850q0 -20 -15 -35t-35 -15q-21 0 -35.5 15t-14.5 35z" />
+<glyph unicode="&#xe123;" d="M74 350q0 21 13.5 35.5t33.5 14.5h17l118 173l63 327q15 77 76 140t144 83l-18 32q-6 19 3 32t29 13h94q20 0 29 -10.5t3 -29.5l-18 -37q83 -19 144 -82.5t76 -140.5l63 -327l118 -173h17q20 0 33.5 -14.5t13.5 -35.5q0 -20 -13 -40t-31 -27q-22 -9 -63 -23t-167.5 -37 t-251.5 -23t-245.5 20.5t-178.5 41.5l-58 20q-18 7 -31 27.5t-13 40.5zM497 110q12 -49 40 -79.5t63 -30.5t63 30.5t39 79.5q-48 -6 -102 -6t-103 6z" />
+<glyph unicode="&#xe124;" d="M21 445l233 -45l-78 -224l224 78l45 -233l155 179l155 -179l45 233l224 -78l-78 224l234 45l-180 155l180 156l-234 44l78 225l-224 -78l-45 233l-155 -180l-155 180l-45 -233l-224 78l78 -225l-233 -44l179 -156z" />
+<glyph unicode="&#xe125;" d="M0 200h200v600h-200v-600zM300 275q0 -75 100 -75h61q123 -100 139 -100h250q46 0 83 57l238 344q29 31 29 74v100q0 44 -30.5 84.5t-69.5 40.5h-328q28 118 28 125v150q0 44 -30.5 84.5t-69.5 40.5h-50q-27 0 -51 -20t-38 -48l-96 -198l-145 -196q-20 -26 -20 -63v-400z M400 300v375l150 212l100 213h50v-175l-50 -225h450v-125l-250 -375h-214l-136 100h-100z" />
+<glyph unicode="&#xe126;" d="M0 400v600h200v-600h-200zM300 525v400q0 75 100 75h61q123 100 139 100h250q46 0 83 -57l238 -344q29 -31 29 -74v-100q0 -44 -30.5 -84.5t-69.5 -40.5h-328q28 -118 28 -125v-150q0 -44 -30.5 -84.5t-69.5 -40.5h-50q-27 0 -51 20t-38 48l-96 198l-145 196 q-20 26 -20 63zM400 525l150 -212l100 -213h50v175l-50 225h450v125l-250 375h-214l-136 -100h-100v-375z" />
+<glyph unicode="&#xe127;" d="M8 200v600h200v-600h-200zM308 275v525q0 17 14 35.5t28 28.5l14 9l362 230q14 6 25 6q17 0 29 -12l109 -112q14 -14 14 -34q0 -18 -11 -32l-85 -121h302q85 0 138.5 -38t53.5 -110t-54.5 -111t-138.5 -39h-107l-130 -339q-7 -22 -20.5 -41.5t-28.5 -19.5h-341 q-7 0 -90 81t-83 94zM408 289l100 -89h293l131 339q6 21 19.5 41t28.5 20h203q16 0 25 15t9 36q0 20 -9 34.5t-25 14.5h-457h-6.5h-7.5t-6.5 0.5t-6 1t-5 1.5t-5.5 2.5t-4 4t-4 5.5q-5 12 -5 20q0 14 10 27l147 183l-86 83l-339 -236v-503z" />
+<glyph unicode="&#xe128;" d="M-101 651q0 72 54 110t139 37h302l-85 121q-11 16 -11 32q0 21 14 34l109 113q13 12 29 12q11 0 25 -6l365 -230q7 -4 16.5 -10.5t26 -26t16.5 -36.5v-526q0 -13 -85.5 -93.5t-93.5 -80.5h-342q-15 0 -28.5 20t-19.5 41l-131 339h-106q-84 0 -139 39t-55 111zM-1 601h222 q15 0 28.5 -20.5t19.5 -40.5l131 -339h293l106 89v502l-342 237l-87 -83l145 -184q10 -11 10 -26q0 -11 -5 -20q-1 -3 -3.5 -5.5l-4 -4t-5 -2.5t-5.5 -1.5t-6.5 -1t-6.5 -0.5h-7.5h-6.5h-476v-100zM999 201v600h200v-600h-200z" />
+<glyph unicode="&#xe129;" d="M97 719l230 -363q4 -6 10.5 -15.5t26 -25t36.5 -15.5h525q13 0 94 83t81 90v342q0 15 -20 28.5t-41 19.5l-339 131v106q0 84 -39 139t-111 55t-110 -53.5t-38 -138.5v-302l-121 84q-15 12 -33.5 11.5t-32.5 -13.5l-112 -110q-22 -22 -6 -53zM172 739l83 86l183 -146 q22 -18 47 -5q3 1 5.5 3.5l4 4t2.5 5t1.5 5.5t1 6.5t0.5 6v7.5v7v456q0 22 25 31t50 -0.5t25 -30.5v-202q0 -16 20 -29.5t41 -19.5l339 -130v-294l-89 -100h-503zM400 0v200h600v-200h-600z" />
+<glyph unicode="&#xe130;" d="M1 585q-15 -31 7 -53l112 -110q13 -13 32 -13.5t34 10.5l121 85l-1 -302q0 -84 38.5 -138t110.5 -54t111 55t39 139v106l339 131q20 6 40.5 19.5t20.5 28.5v342q0 7 -81 90t-94 83h-525q-17 0 -35.5 -14t-28.5 -28l-10 -15zM76 565l237 339h503l89 -100v-294l-340 -130 q-20 -6 -40 -20t-20 -29v-202q0 -22 -25 -31t-50 0t-25 31v456v14.5t-1.5 11.5t-5 12t-9.5 7q-24 13 -46 -5l-184 -146zM305 1104v200h600v-200h-600z" />
+<glyph unicode="&#xe131;" d="M5 597q0 122 47.5 232.5t127.5 190.5t190.5 127.5t232.5 47.5q162 0 299.5 -80t217.5 -218t80 -300t-80 -299.5t-217.5 -217.5t-299.5 -80t-300 80t-218 217.5t-80 299.5zM300 500h300l-2 -194l402 294l-402 298v-197h-298v-201z" />
+<glyph unicode="&#xe132;" d="M0 597q0 122 47.5 232.5t127.5 190.5t190.5 127.5t231.5 47.5q122 0 232.5 -47.5t190.5 -127.5t127.5 -190.5t47.5 -232.5q0 -162 -80 -299.5t-218 -217.5t-300 -80t-299.5 80t-217.5 217.5t-80 299.5zM200 600l400 -294v194h302v201h-300v197z" />
+<glyph unicode="&#xe133;" d="M5 597q0 122 47.5 232.5t127.5 190.5t190.5 127.5t232.5 47.5q121 0 231.5 -47.5t190.5 -127.5t127.5 -190.5t47.5 -232.5q0 -162 -80 -299.5t-217.5 -217.5t-299.5 -80t-300 80t-218 217.5t-80 299.5zM300 600h200v-300h200v300h200l-300 400z" />
+<glyph unicode="&#xe134;" d="M5 597q0 122 47.5 232.5t127.5 190.5t190.5 127.5t232.5 47.5q121 0 231.5 -47.5t190.5 -127.5t127.5 -190.5t47.5 -232.5q0 -162 -80 -299.5t-217.5 -217.5t-299.5 -80t-300 80t-218 217.5t-80 299.5zM300 600l300 -400l300 400h-200v300h-200v-300h-200z" />
+<glyph unicode="&#xe135;" d="M5 597q0 122 47.5 232.5t127.5 190.5t190.5 127.5t232.5 47.5q121 0 231.5 -47.5t190.5 -127.5t127.5 -190.5t47.5 -232.5q0 -162 -80 -299.5t-217.5 -217.5t-299.5 -80t-300 80t-218 217.5t-80 299.5zM254 780q-8 -34 5.5 -93t7.5 -87q0 -9 17 -44t16 -60q12 0 23 -5.5 t23 -15t20 -13.5q20 -10 108 -42q22 -8 53 -31.5t59.5 -38.5t57.5 -11q8 -18 -15 -55.5t-20 -57.5q12 -21 22.5 -34.5t28 -27t36.5 -17.5q0 -6 -3 -15.5t-3.5 -14.5t4.5 -17q101 -2 221 111q31 30 47 48t34 49t21 62q-14 9 -37.5 9.5t-35.5 7.5q-14 7 -49 15t-52 19 q-9 0 -39.5 -0.5t-46.5 -1.5t-39 -6.5t-39 -16.5q-50 -35 -66 -12q-4 2 -3.5 25.5t0.5 25.5q-6 13 -26.5 17t-24.5 7q2 22 -2 41t-16.5 28t-38.5 -20q-23 -25 -42 4q-19 28 -8 58q8 16 22 22q6 -1 26 -1.5t33.5 -4.5t19.5 -13q12 -19 32 -37.5t34 -27.5l14 -8q0 3 9.5 39.5 t5.5 57.5q-4 23 14.5 44.5t22.5 31.5q5 14 10 35t8.5 31t15.5 22.5t34 21.5q-6 18 10 37q8 0 23.5 -1.5t24.5 -1.5t20.5 4.5t20.5 15.5q-10 23 -30.5 42.5t-38 30t-49 26.5t-43.5 23q11 41 1 44q31 -13 58.5 -14.5t39.5 3.5l11 4q6 3
 6 -17 53.5t-64 28.5t-56 23 q-19 -3 -37 0q-15 -12 -36.5 -21t-34.5 -12t-44 -8t-39 -6q-15 -3 -46 0t-45 -3q-20 -6 -51.5 -25.5t-34.5 -34.5q-3 -11 6.5 -22.5t8.5 -18.5q-3 -34 -27.5 -91t-29.5 -79zM518 915q3 12 16 30.5t16 25.5q10 -10 18.5 -10t14 6t14.5 14.5t16 12.5q0 -18 8 -42.5t16.5 -44 t9.5 -23.5q-6 1 -39 5t-53.5 10t-36.5 16z" />
+<glyph unicode="&#xe136;" d="M0 164.5q0 21.5 15 37.5l600 599q-33 101 6 201.5t135 154.5q164 92 306 -9l-259 -138l145 -232l251 126q13 -175 -151 -267q-123 -70 -253 -23l-596 -596q-15 -16 -36.5 -16t-36.5 16l-111 110q-15 15 -15 36.5z" />
+<glyph unicode="&#xe137;" horiz-adv-x="1220" d="M0 196v100q0 41 29.5 70.5t70.5 29.5h1000q41 0 70.5 -29.5t29.5 -70.5v-100q0 -41 -29.5 -70.5t-70.5 -29.5h-1000q-41 0 -70.5 29.5t-29.5 70.5zM0 596v100q0 41 29.5 70.5t70.5 29.5h1000q41 0 70.5 -29.5t29.5 -70.5v-100q0 -41 -29.5 -70.5t-70.5 -29.5h-1000 q-41 0 -70.5 29.5t-29.5 70.5zM0 996v100q0 41 29.5 70.5t70.5 29.5h1000q41 0 70.5 -29.5t29.5 -70.5v-100q0 -41 -29.5 -70.5t-70.5 -29.5h-1000q-41 0 -70.5 29.5t-29.5 70.5zM600 596h500v100h-500v-100zM800 196h300v100h-300v-100zM900 996h200v100h-200v-100z" />
+<glyph unicode="&#xe138;" d="M100 1100v100h1000v-100h-1000zM150 1000h900l-350 -500v-300l-200 -200v500z" />
+<glyph unicode="&#xe139;" d="M0 200v200h1200v-200q0 -41 -29.5 -70.5t-70.5 -29.5h-1000q-41 0 -70.5 29.5t-29.5 70.5zM0 500v400q0 41 29.5 70.5t70.5 29.5h300v100q0 41 29.5 70.5t70.5 29.5h200q41 0 70.5 -29.5t29.5 -70.5v-100h300q41 0 70.5 -29.5t29.5 -70.5v-400h-500v100h-200v-100h-500z M500 1000h200v100h-200v-100z" />
+<glyph unicode="&#xe140;" d="M0 0v400l129 -129l200 200l142 -142l-200 -200l129 -129h-400zM0 800l129 129l200 -200l142 142l-200 200l129 129h-400v-400zM729 329l142 142l200 -200l129 129v-400h-400l129 129zM729 871l200 200l-129 129h400v-400l-129 129l-200 -200z" />
+<glyph unicode="&#xe141;" d="M0 596q0 162 80 299t217 217t299 80t299 -80t217 -217t80 -299t-80 -299t-217 -217t-299 -80t-299 80t-217 217t-80 299zM182 596q0 -172 121.5 -293t292.5 -121t292.5 121t121.5 293q0 171 -121.5 292.5t-292.5 121.5t-292.5 -121.5t-121.5 -292.5zM291 655 q0 23 15.5 38.5t38.5 15.5t39 -16t16 -38q0 -23 -16 -39t-39 -16q-22 0 -38 16t-16 39zM400 850q0 22 16 38.5t39 16.5q22 0 38 -16t16 -39t-16 -39t-38 -16q-23 0 -39 16.5t-16 38.5zM513 609q0 32 21 56.5t52 29.5l122 126l1 1q-9 14 -9 28q0 22 16 38.5t39 16.5 q22 0 38 -16t16 -39t-16 -39t-38 -16q-16 0 -29 10l-55 -145q17 -22 17 -51q0 -36 -25.5 -61.5t-61.5 -25.5q-37 0 -62.5 25.5t-25.5 61.5zM800 655q0 22 16 38t39 16t38.5 -15.5t15.5 -38.5t-16 -39t-38 -16q-23 0 -39 16t-16 39z" />
+<glyph unicode="&#xe142;" d="M-40 375q-13 -95 35 -173q35 -57 94 -89t129 -32q63 0 119 28q33 16 65 40.5t52.5 45.5t59.5 64q40 44 57 61l394 394q35 35 47 84t-3 96q-27 87 -117 104q-20 2 -29 2q-46 0 -79.5 -17t-67.5 -51l-388 -396l-7 -7l69 -67l377 373q20 22 39 38q23 23 50 23q38 0 53 -36 q16 -39 -20 -75l-547 -547q-52 -52 -125 -52q-55 0 -100 33t-54 96q-5 35 2.5 66t31.5 63t42 50t56 54q24 21 44 41l348 348q52 52 82.5 79.5t84 54t107.5 26.5q25 0 48 -4q95 -17 154 -94.5t51 -175.5q-7 -101 -98 -192l-252 -249l-253 -256l7 -7l69 -60l517 511 q67 67 95 157t11 183q-16 87 -67 154t-130 103q-69 33 -152 33q-107 0 -197 -55q-40 -24 -111 -95l-512 -512q-68 -68 -81 -163z" />
+<glyph unicode="&#xe143;" d="M79 784q0 131 99 229.5t230 98.5q144 0 242 -129q103 129 245 129q130 0 227 -98.5t97 -229.5q0 -46 -17.5 -91t-61 -99t-77 -89.5t-104.5 -105.5q-197 -191 -293 -322l-17 -23l-16 23q-43 58 -100 122.5t-92 99.5t-101 100l-84.5 84.5t-68 74t-60 78t-33.5 70.5t-15 78z M250 784q0 -27 30.5 -70t61.5 -75.5t95 -94.5l22 -22q93 -90 190 -201q82 92 195 203l12 12q64 62 97.5 97t64.5 79t31 72q0 71 -48 119.5t-106 48.5q-73 0 -131 -83l-118 -171l-114 174q-51 80 -124 80q-59 0 -108.5 -49.5t-49.5 -118.5z" />
+<glyph unicode="&#xe144;" d="M57 353q0 -94 66 -160l141 -141q66 -66 159 -66q95 0 159 66l283 283q66 66 66 159t-66 159l-141 141q-12 12 -19 17l-105 -105l212 -212l-389 -389l-247 248l95 95l-18 18q-46 45 -75 101l-55 -55q-66 -66 -66 -159zM269 706q0 -93 66 -159l141 -141l19 -17l105 105 l-212 212l389 389l247 -247l-95 -96l18 -18q46 -46 77 -99l29 29q35 35 62.5 88t27.5 96q0 93 -66 159l-141 141q-66 66 -159 66q-95 0 -159 -66l-283 -283q-66 -64 -66 -159z" />
+<glyph unicode="&#xe145;" d="M200 100v953q0 21 30 46t81 48t129 38t163 15t162 -15t127 -38t79 -48t29 -46v-953q0 -41 -29.5 -70.5t-70.5 -29.5h-600q-41 0 -70.5 29.5t-29.5 70.5zM300 300h600v700h-600v-700zM496 150q0 -43 30.5 -73.5t73.5 -30.5t73.5 30.5t30.5 73.5t-30.5 73.5t-73.5 30.5 t-73.5 -30.5t-30.5 -73.5z" />
+<glyph unicode="&#xe146;" d="M0 0l303 380l207 208l-210 212h300l267 279l-35 36q-15 14 -15 35t15 35q14 15 35 15t35 -15l283 -282q15 -15 15 -36t-15 -35q-14 -15 -35 -15t-35 15l-36 35l-279 -267v-300l-212 210l-208 -207z" />
+<glyph unicode="&#xe148;" d="M295 433h139q5 -77 48.5 -126.5t117.5 -64.5v335l-27 7q-46 14 -79 26.5t-72 36t-62.5 52t-40 72.5t-16.5 99q0 92 44 159.5t109 101t144 40.5v78h100v-79q38 -4 72.5 -13.5t75.5 -31.5t71 -53.5t51.5 -84t24.5 -118.5h-159q-8 72 -35 109.5t-101 50.5v-307l64 -14 q34 -7 64 -16.5t70 -31.5t67.5 -52t47.5 -80.5t20 -112.5q0 -139 -89 -224t-244 -96v-77h-100v78q-152 17 -237 104q-40 40 -52.5 93.5t-15.5 139.5zM466 889q0 -29 8 -51t16.5 -34t29.5 -22.5t31 -13.5t38 -10q7 -2 11 -3v274q-61 -8 -97.5 -37.5t-36.5 -102.5zM700 237 q170 18 170 151q0 64 -44 99.5t-126 60.5v-311z" />
+<glyph unicode="&#xe149;" d="M100 600v100h166q-24 49 -44 104q-10 26 -14.5 55.5t-3 72.5t25 90t68.5 87q97 88 263 88q129 0 230 -89t101 -208h-153q0 52 -34 89.5t-74 51.5t-76 14q-37 0 -79 -14.5t-62 -35.5q-41 -44 -41 -101q0 -11 2.5 -24.5t5.5 -24t9.5 -26.5t10.5 -25t14 -27.5t14 -25.5 t15.5 -27t13.5 -24h242v-100h-197q8 -50 -2.5 -115t-31.5 -94q-41 -59 -99 -113q35 11 84 18t70 7q32 1 102 -16t104 -17q76 0 136 30l50 -147q-41 -25 -80.5 -36.5t-59 -13t-61.5 -1.5q-23 0 -128 33t-155 29q-39 -4 -82 -17t-66 -25l-24 -11l-55 145l16.5 11t15.5 10 t13.5 9.5t14.5 12t14.5 14t17.5 18.5q48 55 54 126.5t-30 142.5h-221z" />
+<glyph unicode="&#xe150;" d="M2 300l298 -300l298 300h-198v900h-200v-900h-198zM602 900l298 300l298 -300h-198v-900h-200v900h-198z" />
+<glyph unicode="&#xe151;" d="M2 300h198v900h200v-900h198l-298 -300zM700 0v200h100v-100h200v-100h-300zM700 400v100h300v-200h-99v-100h-100v100h99v100h-200zM700 700v500h300v-500h-100v100h-100v-100h-100zM801 900h100v200h-100v-200z" />
+<glyph unicode="&#xe152;" d="M2 300h198v900h200v-900h198l-298 -300zM700 0v500h300v-500h-100v100h-100v-100h-100zM700 700v200h100v-100h200v-100h-300zM700 1100v100h300v-200h-99v-100h-100v100h99v100h-200zM801 200h100v200h-100v-200z" />
+<glyph unicode="&#xe153;" d="M2 300l298 -300l298 300h-198v900h-200v-900h-198zM800 100v400h300v-500h-100v100h-200zM800 1100v100h200v-500h-100v400h-100zM901 200h100v200h-100v-200z" />
+<glyph unicode="&#xe154;" d="M2 300l298 -300l298 300h-198v900h-200v-900h-198zM800 400v100h200v-500h-100v400h-100zM800 800v400h300v-500h-100v100h-200zM901 900h100v200h-100v-200z" />
+<glyph unicode="&#xe155;" d="M2 300l298 -300l298 300h-198v900h-200v-900h-198zM700 100v200h500v-200h-500zM700 400v200h400v-200h-400zM700 700v200h300v-200h-300zM700 1000v200h200v-200h-200z" />
+<glyph unicode="&#xe156;" d="M2 300l298 -300l298 300h-198v900h-200v-900h-198zM700 100v200h200v-200h-200zM700 400v200h300v-200h-300zM700 700v200h400v-200h-400zM700 1000v200h500v-200h-500z" />
+<glyph unicode="&#xe157;" d="M0 400v300q0 165 117.5 282.5t282.5 117.5h300q162 0 281 -118.5t119 -281.5v-300q0 -165 -118.5 -282.5t-281.5 -117.5h-300q-165 0 -282.5 117.5t-117.5 282.5zM200 300q0 -41 29.5 -70.5t70.5 -29.5h500q41 0 70.5 29.5t29.5 70.5v500q0 41 -29.5 70.5t-70.5 29.5 h-500q-41 0 -70.5 -29.5t-29.5 -70.5v-500z" />
+<glyph unicode="&#xe158;" d="M0 400v300q0 163 119 281.5t281 118.5h300q165 0 282.5 -117.5t117.5 -282.5v-300q0 -165 -117.5 -282.5t-282.5 -117.5h-300q-163 0 -281.5 117.5t-118.5 282.5zM200 300q0 -41 29.5 -70.5t70.5 -29.5h500q41 0 70.5 29.5t29.5 70.5v500q0 41 -29.5 70.5t-70.5 29.5 h-500q-41 0 -70.5 -29.5t-29.5 -70.5v-500zM400 300l333 250l-333 250v-500z" />
+<glyph unicode="&#xe159;" d="M0 400v300q0 163 117.5 281.5t282.5 118.5h300q163 0 281.5 -119t118.5 -281v-300q0 -165 -117.5 -282.5t-282.5 -117.5h-300q-165 0 -282.5 117.5t-117.5 282.5zM200 300q0 -41 29.5 -70.5t70.5 -29.5h500q41 0 70.5 29.5t29.5 70.5v500q0 41 -29.5 70.5t-70.5 29.5 h-500q-41 0 -70.5 -29.5t-29.5 -70.5v-500zM300 700l250 -333l250 333h-500z" />
+<glyph unicode="&#xe160;" d="M0 400v300q0 165 117.5 282.5t282.5 117.5h300q165 0 282.5 -117.5t117.5 -282.5v-300q0 -162 -118.5 -281t-281.5 -119h-300q-165 0 -282.5 118.5t-117.5 281.5zM200 300q0 -41 29.5 -70.5t70.5 -29.5h500q41 0 70.5 29.5t29.5 70.5v500q0 41 -29.5 70.5t-70.5 29.5 h-500q-41 0 -70.5 -29.5t-29.5 -70.5v-500zM300 400h500l-250 333z" />
+<glyph unicode="&#xe161;" d="M0 400v300h300v200l400 -350l-400 -350v200h-300zM500 0v200h500q41 0 70.5 29.5t29.5 70.5v500q0 41 -29.5 70.5t-70.5 29.5h-500v200h400q165 0 282.5 -117.5t117.5 -282.5v-300q0 -165 -117.5 -282.5t-282.5 -117.5h-400z" />
+<glyph unicode="&#xe162;" d="M216 519q10 -19 32 -19h302q-155 -438 -160 -458q-5 -21 4 -32l9 -8l9 -1q13 0 26 16l538 630q15 19 6 36q-8 18 -32 16h-300q1 4 78 219.5t79 227.5q2 17 -6 27l-8 8h-9q-16 0 -25 -15q-4 -5 -98.5 -111.5t-228 -257t-209.5 -238.5q-17 -19 -7 -40z" />
+<glyph unicode="&#xe163;" d="M0 400q0 -165 117.5 -282.5t282.5 -117.5h300q47 0 100 15v185h-500q-41 0 -70.5 29.5t-29.5 70.5v500q0 41 29.5 70.5t70.5 29.5h500v185q-14 4 -114 7.5t-193 5.5l-93 2q-165 0 -282.5 -117.5t-117.5 -282.5v-300zM600 400v300h300v200l400 -350l-400 -350v200h-300z " />
+<glyph unicode="&#xe164;" d="M0 400q0 -165 117.5 -282.5t282.5 -117.5h300q163 0 281.5 117.5t118.5 282.5v98l-78 73l-122 -123v-148q0 -41 -29.5 -70.5t-70.5 -29.5h-500q-41 0 -70.5 29.5t-29.5 70.5v500q0 41 29.5 70.5t70.5 29.5h156l118 122l-74 78h-100q-165 0 -282.5 -117.5t-117.5 -282.5 v-300zM496 709l353 342l-149 149h500v-500l-149 149l-342 -353z" />
+<glyph unicode="&#xe165;" d="M4 600q0 162 80 299t217 217t299 80t299 -80t217 -217t80 -299t-80 -299t-217 -217t-299 -80t-299 80t-217 217t-80 299zM186 600q0 -171 121.5 -292.5t292.5 -121.5t292.5 121.5t121.5 292.5t-121.5 292.5t-292.5 121.5t-292.5 -121.5t-121.5 -292.5zM406 600 q0 80 57 137t137 57t137 -57t57 -137t-57 -137t-137 -57t-137 57t-57 137z" />
+<glyph unicode="&#xe166;" d="M0 0v275q0 11 7 18t18 7h1048q11 0 19 -7.5t8 -17.5v-275h-1100zM100 800l445 -500l450 500h-295v400h-300v-400h-300zM900 150h100v50h-100v-50z" />
+<glyph unicode="&#xe167;" d="M0 0v275q0 11 7 18t18 7h1048q11 0 19 -7.5t8 -17.5v-275h-1100zM100 700h300v-300h300v300h295l-445 500zM900 150h100v50h-100v-50z" />
+<glyph unicode="&#xe168;" d="M0 0v275q0 11 7 18t18 7h1048q11 0 19 -7.5t8 -17.5v-275h-1100zM100 705l305 -305l596 596l-154 155l-442 -442l-150 151zM900 150h100v50h-100v-50z" />
+<glyph unicode="&#xe169;" d="M0 0v275q0 11 7 18t18 7h1048q11 0 19 -7.5t8 -17.5v-275h-1100zM100 988l97 -98l212 213l-97 97zM200 401h700v699l-250 -239l-149 149l-212 -212l149 -149zM900 150h100v50h-100v-50z" />
+<glyph unicode="&#xe170;" d="M0 0v275q0 11 7 18t18 7h1048q11 0 19 -7.5t8 -17.5v-275h-1100zM200 612l212 -212l98 97l-213 212zM300 1200l239 -250l-149 -149l212 -212l149 148l248 -237v700h-699zM900 150h100v50h-100v-50z" />
+<glyph unicode="&#xe171;" d="M23 415l1177 784v-1079l-475 272l-310 -393v416h-392zM494 210l672 938l-672 -712v-226z" />
+<glyph unicode="&#xe172;" d="M0 150v1000q0 20 14.5 35t35.5 15h250v-300h500v300h100l200 -200v-850q0 -21 -15 -35.5t-35 -14.5h-150v400h-700v-400h-150q-21 0 -35.5 14.5t-14.5 35.5zM600 1000h100v200h-100v-200z" />
+<glyph unicode="&#xe173;" d="M0 150v1000q0 20 14.5 35t35.5 15h250v-300h500v300h100l200 -200v-218l-276 -275l-120 120l-126 -127h-378v-400h-150q-21 0 -35.5 14.5t-14.5 35.5zM581 306l123 123l120 -120l353 352l123 -123l-475 -476zM600 1000h100v200h-100v-200z" />
+<glyph unicode="&#xe174;" d="M0 150v1000q0 20 14.5 35t35.5 15h250v-300h500v300h100l200 -200v-269l-103 -103l-170 170l-298 -298h-329v-400h-150q-21 0 -35.5 14.5t-14.5 35.5zM600 1000h100v200h-100v-200zM700 133l170 170l-170 170l127 127l170 -170l170 170l127 -128l-170 -169l170 -170 l-127 -127l-170 170l-170 -170z" />
+<glyph unicode="&#xe175;" d="M0 150v1000q0 20 14.5 35t35.5 15h250v-300h500v300h100l200 -200v-300h-400v-200h-500v-400h-150q-21 0 -35.5 14.5t-14.5 35.5zM600 300l300 -300l300 300h-200v300h-200v-300h-200zM600 1000v200h100v-200h-100z" />
+<glyph unicode="&#xe176;" d="M0 150v1000q0 20 14.5 35t35.5 15h250v-300h500v300h100l200 -200v-402l-200 200l-298 -298h-402v-400h-150q-21 0 -35.5 14.5t-14.5 35.5zM600 300h200v-300h200v300h200l-300 300zM600 1000v200h100v-200h-100z" />
+<glyph unicode="&#xe177;" d="M0 250q0 -21 14.5 -35.5t35.5 -14.5h1100q21 0 35.5 14.5t14.5 35.5v550h-1200v-550zM0 900h1200v150q0 21 -14.5 35.5t-35.5 14.5h-1100q-21 0 -35.5 -14.5t-14.5 -35.5v-150zM100 300v200h400v-200h-400z" />
+<glyph unicode="&#xe178;" d="M0 400l300 298v-198h400v-200h-400v-198zM100 800v200h100v-200h-100zM300 800v200h100v-200h-100zM500 800v200h400v198l300 -298l-300 -298v198h-400zM800 300v200h100v-200h-100zM1000 300h100v200h-100v-200z" />
+<glyph unicode="&#xe179;" d="M100 700v400l50 100l50 -100v-300h100v300l50 100l50 -100v-300h100v300l50 100l50 -100v-400l-100 -203v-447q0 -21 -14.5 -35.5t-35.5 -14.5h-200q-21 0 -35.5 14.5t-14.5 35.5v447zM800 597q0 -29 10.5 -55.5t25 -43t29 -28.5t25.5 -18l10 -5v-397q0 -21 14.5 -35.5 t35.5 -14.5h200q21 0 35.5 14.5t14.5 35.5v1106q0 31 -18 40.5t-44 -7.5l-276 -117q-25 -16 -43.5 -50.5t-18.5 -65.5v-359z" />
+<glyph unicode="&#xe180;" d="M100 0h400v56q-75 0 -87.5 6t-12.5 44v394h500v-394q0 -38 -12.5 -44t-87.5 -6v-56h400v56q-4 0 -11 0.5t-24 3t-30 7t-24 15t-11 24.5v888q0 22 25 34.5t50 13.5l25 2v56h-400v-56q75 0 87.5 -6t12.5 -44v-394h-500v394q0 38 12.5 44t87.5 6v56h-400v-56q4 0 11 -0.5 t24 -3t30 -7t24 -15t11 -24.5v-888q0 -22 -25 -34.5t-50 -13.5l-25 -2v-56z" />
+<glyph unicode="&#xe181;" d="M0 300q0 -41 29.5 -70.5t70.5 -29.5h300q41 0 70.5 29.5t29.5 70.5v500q0 41 -29.5 70.5t-70.5 29.5h-300q-41 0 -70.5 -29.5t-29.5 -70.5v-500zM100 100h400l200 200h105l295 98v-298h-425l-100 -100h-375zM100 300v200h300v-200h-300zM100 600v200h300v-200h-300z M100 1000h400l200 -200v-98l295 98h105v200h-425l-100 100h-375zM700 402v163l400 133v-163z" />
+<glyph unicode="&#xe182;" d="M16.5 974.5q0.5 -21.5 16 -90t46.5 -140t104 -177.5t175 -208q103 -103 207.5 -176t180 -103.5t137 -47t92.5 -16.5l31 1l163 162q16 17 13 40.5t-22 37.5l-192 136q-19 14 -45 12t-42 -19l-119 -118q-143 103 -267 227q-126 126 -227 268l118 118q17 17 20 41.5 t-11 44.5l-139 194q-14 19 -36.5 22t-40.5 -14l-162 -162q-1 -11 -0.5 -32.5z" />
+<glyph unicode="&#xe183;" d="M0 50v212q0 20 10.5 45.5t24.5 39.5l365 303v50q0 4 1 10.5t12 22.5t30 28.5t60 23t97 10.5t97 -10t60 -23.5t30 -27.5t12 -24l1 -10v-50l365 -303q14 -14 24.5 -39.5t10.5 -45.5v-212q0 -21 -15 -35.5t-35 -14.5h-1100q-21 0 -35.5 14.5t-14.5 35.5zM0 712 q0 -21 14.5 -33.5t34.5 -8.5l202 33q20 4 34.5 21t14.5 38v146q141 24 300 24t300 -24v-146q0 -21 14.5 -38t34.5 -21l202 -33q20 -4 34.5 8.5t14.5 33.5v200q-6 8 -19 20.5t-63 45t-112 57t-171 45t-235 20.5q-92 0 -175 -10.5t-141.5 -27t-108.5 -36.5t-81.5 -40 t-53.5 -36.5t-31 -27.5l-9 -10v-200z" />
+<glyph unicode="&#xe184;" d="M100 0v100h1100v-100h-1100zM175 200h950l-125 150v250l100 100v400h-100v-200h-100v200h-200v-200h-100v200h-200v-200h-100v200h-100v-400l100 -100v-250z" />
+<glyph unicode="&#xe185;" d="M100 0h300v400q0 41 -29.5 70.5t-70.5 29.5h-100q-41 0 -70.5 -29.5t-29.5 -70.5v-400zM500 0v1000q0 41 29.5 70.5t70.5 29.5h100q41 0 70.5 -29.5t29.5 -70.5v-1000h-300zM900 0v700q0 41 29.5 70.5t70.5 29.5h100q41 0 70.5 -29.5t29.5 -70.5v-700h-300z" />
+<glyph unicode="&#xe186;" d="M-100 300v500q0 124 88 212t212 88h700q124 0 212 -88t88 -212v-500q0 -124 -88 -212t-212 -88h-700q-124 0 -212 88t-88 212zM100 200h900v700h-900v-700zM200 300h300v300h-200v100h200v100h-300v-300h200v-100h-200v-100zM600 300h200v100h100v300h-100v100h-200v-500 zM700 400v300h100v-300h-100z" />
+<glyph unicode="&#xe187;" d="M-100 300v500q0 124 88 212t212 88h700q124 0 212 -88t88 -212v-500q0 -124 -88 -212t-212 -88h-700q-124 0 -212 88t-88 212zM100 200h900v700h-900v-700zM200 300h100v200h100v-200h100v500h-100v-200h-100v200h-100v-500zM600 300h200v100h100v300h-100v100h-200v-500 zM700 400v300h100v-300h-100z" />
+<glyph unicode="&#xe188;" d="M-100 300v500q0 124 88 212t212 88h700q124 0 212 -88t88 -212v-500q0 -124 -88 -212t-212 -88h-700q-124 0 -212 88t-88 212zM100 200h900v700h-900v-700zM200 300h300v100h-200v300h200v100h-300v-500zM600 300h300v100h-200v300h200v100h-300v-500z" />
+<glyph unicode="&#xe189;" d="M-100 300v500q0 124 88 212t212 88h700q124 0 212 -88t88 -212v-500q0 -124 -88 -212t-212 -88h-700q-124 0 -212 88t-88 212zM100 200h900v700h-900v-700zM200 550l300 -150v300zM600 400l300 150l-300 150v-300z" />
+<glyph unicode="&#xe190;" d="M-100 300v500q0 124 88 212t212 88h700q124 0 212 -88t88 -212v-500q0 -124 -88 -212t-212 -88h-700q-124 0 -212 88t-88 212zM100 200h900v700h-900v-700zM200 300v500h700v-500h-700zM300 400h130q41 0 68 42t27 107t-28.5 108t-66.5 43h-130v-300zM575 549 q0 -65 27 -107t68 -42h130v300h-130q-38 0 -66.5 -43t-28.5 -108z" />
+<glyph unicode="&#xe191;" d="M-100 300v500q0 124 88 212t212 88h700q124 0 212 -88t88 -212v-500q0 -124 -88 -212t-212 -88h-700q-124 0 -212 88t-88 212zM100 200h900v700h-900v-700zM200 300h300v300h-200v100h200v100h-300v-300h200v-100h-200v-100zM601 300h100v100h-100v-100zM700 700h100 v-400h100v500h-200v-100z" />
+<glyph unicode="&#xe192;" d="M-100 300v500q0 124 88 212t212 88h700q124 0 212 -88t88 -212v-500q0 -124 -88 -212t-212 -88h-700q-124 0 -212 88t-88 212zM100 200h900v700h-900v-700zM200 300h300v400h-200v100h-100v-500zM301 400v200h100v-200h-100zM601 300h100v100h-100v-100zM700 700h100 v-400h100v500h-200v-100z" />
+<glyph unicode="&#xe193;" d="M-100 300v500q0 124 88 212t212 88h700q124 0 212 -88t88 -212v-500q0 -124 -88 -212t-212 -88h-700q-124 0 -212 88t-88 212zM100 200h900v700h-900v-700zM200 700v100h300v-300h-99v-100h-100v100h99v200h-200zM201 300v100h100v-100h-100zM601 300v100h100v-100h-100z M700 700v100h200v-500h-100v400h-100z" />
+<glyph unicode="&#xe194;" d="M4 600q0 162 80 299t217 217t299 80t299 -80t217 -217t80 -299t-80 -299t-217 -217t-299 -80t-299 80t-217 217t-80 299zM186 600q0 -171 121.5 -292.5t292.5 -121.5t292.5 121.5t121.5 292.5t-121.5 292.5t-292.5 121.5t-292.5 -121.5t-121.5 -292.5zM400 500v200 l100 100h300v-100h-300v-200h300v-100h-300z" />
+<glyph unicode="&#xe195;" d="M0 600q0 162 80 299t217 217t299 80t299 -80t217 -217t80 -299t-80 -299t-217 -217t-299 -80t-299 80t-217 217t-80 299zM182 600q0 -171 121.5 -292.5t292.5 -121.5t292.5 121.5t121.5 292.5t-121.5 292.5t-292.5 121.5t-292.5 -121.5t-121.5 -292.5zM400 400v400h300 l100 -100v-100h-100v100h-200v-100h200v-100h-200v-100h-100zM700 400v100h100v-100h-100z" />
+<glyph unicode="&#xe197;" d="M-14 494q0 -80 56.5 -137t135.5 -57h222v300h400v-300h128q120 0 205 86t85 208q0 120 -85 206.5t-205 86.5q-46 0 -90 -14q-44 97 -134.5 156.5t-200.5 59.5q-152 0 -260 -107.5t-108 -260.5q0 -25 2 -37q-66 -14 -108.5 -67.5t-42.5 -122.5zM300 200h200v300h200v-300 h200l-300 -300z" />
+<glyph unicode="&#xe198;" d="M-14 494q0 -80 56.5 -137t135.5 -57h8l414 414l403 -403q94 26 154.5 104t60.5 178q0 121 -85 207.5t-205 86.5q-46 0 -90 -14q-44 97 -134.5 156.5t-200.5 59.5q-152 0 -260 -107.5t-108 -260.5q0 -25 2 -37q-66 -14 -108.5 -67.5t-42.5 -122.5zM300 200l300 300 l300 -300h-200v-300h-200v300h-200z" />
+<glyph unicode="&#xe199;" d="M100 200h400v-155l-75 -45h350l-75 45v155h400l-270 300h170l-270 300h170l-300 333l-300 -333h170l-270 -300h170z" />
+<glyph unicode="&#xe200;" d="M121 700q0 -53 28.5 -97t75.5 -65q-4 -16 -4 -38q0 -74 52.5 -126.5t126.5 -52.5q56 0 100 30v-306l-75 -45h350l-75 45v306q46 -30 100 -30q74 0 126.5 52.5t52.5 126.5q0 24 -9 55q50 32 79.5 83t29.5 112q0 90 -61.5 155.5t-150.5 71.5q-26 89 -99.5 145.5 t-167.5 56.5q-116 0 -197.5 -81.5t-81.5 -197.5q0 -4 1 -12t1 -11q-14 2 -23 2q-74 0 -126.5 -52.5t-52.5 -126.5z" />
+</font>
+</defs></svg> 
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/fonts/glyphicons-halflings-regular.ttf
----------------------------------------------------------------------
diff --git a/site/current-book/fonts/glyphicons-halflings-regular.ttf b/site/current-book/fonts/glyphicons-halflings-regular.ttf
new file mode 100644
index 0000000..2824015
Binary files /dev/null and b/site/current-book/fonts/glyphicons-halflings-regular.ttf differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/fonts/glyphicons-halflings-regular.woff
----------------------------------------------------------------------
diff --git a/site/current-book/fonts/glyphicons-halflings-regular.woff b/site/current-book/fonts/glyphicons-halflings-regular.woff
new file mode 100644
index 0000000..f824ac1
Binary files /dev/null and b/site/current-book/fonts/glyphicons-halflings-regular.woff differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/images/apache-maven-project-2.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/apache-maven-project-2.png b/site/current-book/images/apache-maven-project-2.png
index 6c096ec..a44db6e 100644
Binary files a/site/current-book/images/apache-maven-project-2.png and b/site/current-book/images/apache-maven-project-2.png differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/images/collapsed.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/collapsed.gif b/site/current-book/images/collapsed.gif
new file mode 100644
index 0000000..6e71084
Binary files /dev/null and b/site/current-book/images/collapsed.gif differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/images/expanded.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/expanded.gif b/site/current-book/images/expanded.gif
new file mode 100644
index 0000000..0fef3d8
Binary files /dev/null and b/site/current-book/images/expanded.gif differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/images/external.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/external.png b/site/current-book/images/external.png
new file mode 100644
index 0000000..3f999fc
Binary files /dev/null and b/site/current-book/images/external.png differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/images/icon_error_sml.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/icon_error_sml.gif b/site/current-book/images/icon_error_sml.gif
index 12e9a01..61132ef 100644
Binary files a/site/current-book/images/icon_error_sml.gif and b/site/current-book/images/icon_error_sml.gif differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/images/icon_info_sml.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/icon_info_sml.gif b/site/current-book/images/icon_info_sml.gif
index b776326..c6cb9ad 100644
Binary files a/site/current-book/images/icon_info_sml.gif and b/site/current-book/images/icon_info_sml.gif differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/images/icon_success_sml.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/icon_success_sml.gif b/site/current-book/images/icon_success_sml.gif
index 0a19527..52e85a4 100644
Binary files a/site/current-book/images/icon_success_sml.gif and b/site/current-book/images/icon_success_sml.gif differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/images/icon_warning_sml.gif
----------------------------------------------------------------------
diff --git a/site/current-book/images/icon_warning_sml.gif b/site/current-book/images/icon_warning_sml.gif
index ac6ad6a..873bbb5 100644
Binary files a/site/current-book/images/icon_warning_sml.gif and b/site/current-book/images/icon_warning_sml.gif differ

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/images/newwindow.png
----------------------------------------------------------------------
diff --git a/site/current-book/images/newwindow.png b/site/current-book/images/newwindow.png
new file mode 100644
index 0000000..6287f72
Binary files /dev/null and b/site/current-book/images/newwindow.png differ


[38/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-analytics/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/index.html b/site/current-book/metron-analytics/index.html
index fc4137a..02a9064 100644
--- a/site/current-book/metron-analytics/index.html
+++ b/site/current-book/metron-analytics/index.html
@@ -1,250 +1,121 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-analytics/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Metron Analytics</title>
-    <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../css/site.css" />
     <link rel="stylesheet" href="../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Metron Analytics</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron Analytics</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                          
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Analytics</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../metron-analytics/metron-maas-service/index.html" title="Maas-service">
-          <i class="none"></i>
-        Maas-service</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-analytics/metron-profiler/index.html" title="Profiler">
-          <i class="none"></i>
-        Profiler</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-analytics/metron-profiler-client/index.html" title="Profiler-client">
-          <i class="none"></i>
-        Profiler-client</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-analytics/metron-statistics/index.html" title="Statistics">
-          <i class="icon-chevron-right"></i>
-        Statistics</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Analytics</a>
+    <ul class="nav nav-list">
+    <li><a href="../metron-analytics/metron-maas-service/index.html" title="Maas-service"><span class="none"></span>Maas-service</a></li>
+    <li><a href="../metron-analytics/metron-profiler/index.html" title="Profiler"><span class="none"></span>Profiler</a></li>
+    <li><a href="../metron-analytics/metron-profiler-client/index.html" title="Profiler-client"><span class="none"></span>Profiler-client</a></li>
+    <li><a href="../metron-analytics/metron-statistics/index.html" title="Statistics"><span class="icon-chevron-right"></span>Statistics</a></li>
+    </ul>
+</li>
+    <li><a href="../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Metron Analytics</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron Analytics</h1>
 <p><a name="Metron_Analytics"></a></p>
 <p>Metron analytics consists of:</p>
-
 <ul>
-  
+
 <li>Model-as-a-Service (MAAS) access to Machine Learning services</li>
-  
 <li>Profiler and Profiler Client</li>
-  
 <li>Statistics</li>
 </ul>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-analytics/metron-maas-service/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/metron-maas-service/index.html b/site/current-book/metron-analytics/metron-maas-service/index.html
index 0298a29..b47dbea 100644
--- a/site/current-book/metron-analytics/metron-maas-service/index.html
+++ b/site/current-book/metron-analytics/metron-maas-service/index.html
@@ -1,280 +1,146 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-analytics/metron-maas-service/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Model Management Infrastructure</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Model Management Infrastructure</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Model Management Infrastructure</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                                
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-down"></i>
-        Analytics</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Maas-service</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-profiler/index.html" title="Profiler">
-          <i class="none"></i>
-        Profiler</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client">
-          <i class="none"></i>
-        Profiler-client</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-analytics/metron-statistics/index.html" title="Statistics">
-          <i class="icon-chevron-right"></i>
-        Statistics</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-down"></span>Analytics</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>Maas-service</a></li>
+    <li><a href="../../metron-analytics/metron-profiler/index.html" title="Profiler"><span class="none"></span>Profiler</a></li>
+    <li><a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client"><span class="none"></span>Profiler-client</a></li>
+    <li><a href="../../metron-analytics/metron-statistics/index.html" title="Statistics"><span class="icon-chevron-right"></span>Statistics</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Model Management Infrastructure</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Model Management Infrastructure</h1>
 <p><a name="Model_Management_Infrastructure"></a></p>
 <div class="section">
 <h2><a name="Introduction"></a>Introduction</h2>
-<p>One of the main features envisioned and requested is the ability to augment the threat intelligence and enrichment processes with insights derived from machine learning or statistical models. The challenges with this sort of infrastructure are</p>
-
+<p>One of the main features envisioned and requested is the ability to augment the threat intelligence and enrichment processes with insights derived from machine learning or statistical models.  The challenges with this sort of infrastructure are</p>
 <ul>
-  
+
 <li>Applying the model may be sufficiently computationally/resource intensive that we need to support scaling via load balancing, which will require service discovery and management.</li>
-  
 <li>Models require out of band and frequent training to react to growing threats and new patterns that emerge.</li>
-  
-<li>Models should be language/environment agnostic as much as possible. These should include small-data and big-data libraries and languages.</li>
+<li>Models should be language/environment agnostic as much as possible.  These should include small-data and big-data libraries and languages.</li>
 </ul>
-<p>To support a high throughput environment that is manageable, it is evident that </p>
-
+<p>To support a high throughput environment that is manageable, it is evident that</p>
 <ul>
-  
+
 <li>Multiple versions of models will need to be exposed</li>
-  
 <li>Deployment should happen using Yarn to manage resources</li>
-  
 <li>Clients should have new model endpoints pushed to them</li>
 </ul></div>
 <div class="section">
 <h2><a name="Architecture"></a>Architecture</h2>
 <p><img src="../../images/maas_arch.png" alt="Architecture" /></p>
 <p>To support these requirements, the following components have been created:</p>
-
 <ul>
-  
+
 <li>A Yarn application which will listen for model deployment requests and upon execution, register their endpoints in zookeeper:
-  
 <ul>
-    
+
 <li>Operation type: ADD, REMOVE, LIST</li>
-    
 <li>Model Name</li>
-    
 <li>Model Version</li>
-    
 <li>Memory requirements (in megabytes)</li>
-    
 <li>Number of instances</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>A command line deployment client which will localize the model payload onto HDFS and submit a model request</li>
-  
 <li>A Java client which will interact with zookeeper and receive updates about model state changes (new deployments, removals, etc.)</li>
-  
 <li>A series of Stellar functions for interacting with models deployed via the Model as a Service infrastructure.</li>
 </ul></div>
 <div class="section">
 <h2><a name="maas_service.sh"></a><tt>maas_service.sh</tt></h2>
-<p>The <tt>maas_service.sh</tt> script starts the Yarn application which will listen for requests. Right now the queue for the requests is a distributed queue stored in <a class="externalLink" href="http://curator.apache.org/curator-recipes/distributed-queue.html">zookeeper</a> for convenience.</p>
+<p>The <tt>maas_service.sh</tt> script starts the Yarn application which will listen for requests.  Right now the queue for the requests is a distributed queue stored in <a class="externalLink" href="http://curator.apache.org/curator-recipes/distributed-queue.html">zookeeper</a> for convenience.</p>
 
-<div class="source">
-<div class="source">
-<pre>./maas_service.sh
+<div>
+<div>
+<pre class="source">./maas_service.sh
 usage: MaaSClient
  -c,--create                          Flag to indicate whether to create
                                       the domain specified with -domain.
@@ -311,23 +177,21 @@ usage: MaaSClient
                                       given domain
  -zq,--zk_quorum &lt;arg&gt;                Zookeeper Quorum
  -zr,--zk_root &lt;arg&gt;                  Zookeeper Root
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="maas_deploy.sh"></a><tt>maas_deploy.sh</tt></h2>
-<p>The <tt>maas_deploy.sh</tt> script allows users to deploy models and their collateral from their local disk to the cluster. It is assumed that the </p>
-
+<p>The <tt>maas_deploy.sh</tt> script allows users to deploy models and their collateral from their local disk to the cluster. It is assumed that the</p>
 <ul>
-  
+
 <li>Collateral has exactly one <tt>.sh</tt> script capable of starting the endpoint</li>
-  
 <li>The model service executable will expose itself as a URL endpoint (e.g. as a REST interface, but not necessarily)</li>
-  
 <li>The model service executable will write out to local disk a JSON blob indicating the endpoint (see <a class="externalLink" href="https://gist.github.com/cestella/cba10aff0f970078a4c2c8cade3a4d1a#file-dga-py-L21">here</a> for an example mock service using Python and Flask).</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>./maas_deploy.sh
+<div>
+<div>
+<pre class="source">./maas_deploy.sh
 usage: ModelSubmission
  -h,--help                       This screen
  -hmp,--hdfs_model_path &lt;arg&gt;    Model Path (HDFS)
@@ -340,92 +204,82 @@ usage: ModelSubmission
  -v,--version &lt;arg&gt;              Model version
  -zq,--zk_quorum &lt;arg&gt;           Zookeeper Quorum
  -zr,--zk_root &lt;arg&gt;             Zookeeper Root
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Kerberos_Support"></a>Kerberos Support</h2>
-<p>Model as a service will run on a kerberized cluster (see <a href="../../metron-deployment/vagrant/Kerberos-setup.html">here</a> for instructions for vagrant) with a caveat. The user who submits the service will be the user who executes the models on the cluster. That is to say that user impersonation of models deployed is not done at the moment.</p></div>
+<p>Model as a service will run on a kerberized cluster (see <a href="../../metron-deployment/vagrant/Kerberos-setup.html">here</a> for instructions for vagrant) with a caveat.  The user who submits the service will be the user who executes the models on the cluster.  That is to say that user impersonation of models deployed is not done at the moment.</p></div>
 <div class="section">
 <h2><a name="Stellar_Integration"></a>Stellar Integration</h2>
 <p>Two Stellar functions have been added to provide the ability to call out to models deployed via Model as a Service. One aimed at recovering a load balanced endpoint of a deployed model given the name and, optionally, the version. The second is aimed at calling that endpoint assuming that it is exposed as a REST endpoint.</p>
-
 <ul>
-  
-<li><tt>MAAS_MODEL_APPLY(endpoint, function?, model_args)</tt> : Returns the output of a model deployed via model which is deployed at endpoint. <tt>endpoint</tt> is a map containing <tt>name</tt>, <tt>version</tt>, <tt>url</tt> for the REST endpoint, <tt>function</tt> is the endpoint path and is optional, and <tt>model_args</tt> is a dictionary of arguments for the model (these become request params).</li>
-  
-<li><tt>MAAS_GET_ENDPOINT(model_name, model_version?)</tt> : Inspects zookeeper and returns a map containing the <tt>name</tt>, <tt>version</tt> and <tt>url</tt> for the model referred to by <tt>model_name</tt> and <tt>model_version</tt>. If <tt>model_version</tt> is not specified, the most current model associated with <tt>model_name</tt> is returned. In the instance where more than one model is deployed, a random one is selected with uniform probability.</li>
+
+<li><tt>MAAS_MODEL_APPLY(endpoint, function?, model_args)</tt> : Returns the output of a model deployed via model which is deployed at endpoint.  <tt>endpoint</tt> is a map containing <tt>name</tt>, <tt>version</tt>, <tt>url</tt> for the REST endpoint, <tt>function</tt> is the endpoint path and is optional, and <tt>model_args</tt> is a dictionary of arguments for the model (these become request params).</li>
+<li><tt>MAAS_GET_ENDPOINT(model_name, model_version?)</tt> : Inspects zookeeper and returns a map containing the <tt>name</tt>, <tt>version</tt> and <tt>url</tt> for the model referred to by <tt>model_name</tt> and <tt>model_version</tt>.  If <tt>model_version</tt> is not specified, the most current model associated with <tt>model_name</tt> is returned.  In the instance where more than one model is deployed, a random one is selected with uniform probability.</li>
 </ul>
 <p><a name="Example"></a></p>
 <h1>Example</h1>
-<p>Let&#x2019;s augment the <tt>squid</tt> proxy sensor to use a model that will determine if the destination host is a domain generating algorithm. For the purposes of demonstration, this algorithm is super simple and is implemented using Python with a REST interface exposed via the Flask python library.</p></div>
+<p>Let&#x2019;s augment the <tt>squid</tt> proxy sensor to use a model that will determine if the destination host is a domain generating algorithm.  For the purposes of demonstration, this algorithm is super simple and is implemented using Python with a REST interface exposed via the Flask python library.</p></div>
 <div class="section">
 <h2><a name="Install_Prerequisites_and_Mock_DGA_Service"></a>Install Prerequisites and Mock DGA Service</h2>
 <p>Now let&#x2019;s install some prerequisites:</p>
-
 <ul>
-  
+
 <li>Flask via <tt>yum install python-flask</tt></li>
-  
 <li>Jinja2 via <tt>yum install python-jinja2</tt></li>
-  
 <li>Squid client via <tt>yum install squid</tt></li>
-  
 <li>ES Head plugin via <tt>/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head</tt></li>
 </ul>
 <p>Start Squid via <tt>service squid start</tt></p>
 <p>Now that we have flask and jinja, we can create a mock DGA service to deploy with MaaS:</p>
-
 <ul>
-  
+
 <li>Download the files in <a class="externalLink" href="https://gist.github.com/cestella/cba10aff0f970078a4c2c8cade3a4d1a">this</a> gist into the <tt>$HOME/mock_dga</tt> directory</li>
-  
 <li>Make <tt>rest.sh</tt> executable via <tt>chmod +x $HOME/mock_dga/rest.sh</tt></li>
 </ul>
-<p>This service will treat <tt>yahoo.com</tt> and <tt>amazon.com</tt> as legit and everything else as malicious. The contract is that the REST service exposes an endpoint <tt>/apply</tt> and returns back JSON maps with a single key <tt>is_malicious</tt> which can be <tt>malicious</tt> or <tt>legit</tt>.</p></div>
+<p>This service will treat <tt>yahoo.com</tt> and <tt>amazon.com</tt> as legit and everything else as malicious.  The contract is that the REST service exposes an endpoint <tt>/apply</tt> and returns back JSON maps with a single key <tt>is_malicious</tt> which can be <tt>malicious</tt> or <tt>legit</tt>.</p></div>
 <div class="section">
 <h2><a name="Deploy_Mock_DGA_Service_via_MaaS"></a>Deploy Mock DGA Service via MaaS</h2>
-<p>The following presumes that you are a logged in as a user who has a home directory in HDFS under <tt>/user/$USER</tt>. If you do not, please create one and ensure the permissions are set appropriate:</p>
+<p>The following presumes that you are a logged in as a user who has a home directory in HDFS under <tt>/user/$USER</tt>.  If you do not, please create one and ensure the permissions are set appropriate:</p>
 
-<div class="source">
-<div class="source">
-<pre>su - hdfs -c &quot;hadoop fs -mkdir /user/$USER&quot;
+<div>
+<div>
+<pre class="source">su - hdfs -c &quot;hadoop fs -mkdir /user/$USER&quot;
 su - hdfs -c &quot;hadoop fs -chown $USER:$USER /user/$USER&quot;
 </pre></div></div>
+
 <p>Or, in the common case for the <tt>metron</tt> user:</p>
 
-<div class="source">
-<div class="source">
-<pre>su - hdfs -c &quot;hadoop fs -mkdir /user/metron&quot;
+<div>
+<div>
+<pre class="source">su - hdfs -c &quot;hadoop fs -mkdir /user/metron&quot;
 su - hdfs -c &quot;hadoop fs -chown metron:metron /user/metron&quot;
 </pre></div></div>
-<p>Now let&#x2019;s start MaaS and deploy the Mock DGA Service:</p>
 
+<p>Now let&#x2019;s start MaaS and deploy the Mock DGA Service:</p>
 <ul>
-  
+
 <li>Start MaaS via <tt>$METRON_HOME/bin/maas_service.sh -zq node1:2181</tt></li>
-  
 <li>Start one instance of the mock DGA model with 512M of memory via <tt>$METRON_HOME/bin/maas_deploy.sh -zq node1:2181 -lmp $HOME/mock_dga -hmp /user/$USER/models -mo ADD -m 512 -n dga -v 1.0 -ni 1</tt></li>
-  
 <li>As a sanity check:
-  
 <ul>
-    
-<li>Ensure that the model is running via <tt>$METRON_HOME/bin/maas_deploy.sh -zq node1:2181 -mo LIST</tt>. You should see <tt>Model dga @ 1.0</tt> be displayed and under that a url such as (but not exactly) <tt>http://node1:36161</tt></li>
-    
+
+<li>Ensure that the model is running via <tt>$METRON_HOME/bin/maas_deploy.sh -zq node1:2181 -mo LIST</tt>.  You should see <tt>Model dga @ 1.0</tt> be displayed and under that a url such as (but not exactly) <tt>http://node1:36161</tt></li>
 <li>Try to hit the model via curl: <tt>curl 'http://localhost:36161/apply?host=caseystella.com'</tt> and ensure that it returns a JSON map indicating the domain is malicious.</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h2><a name="Adjust_Configurations_for_Squid_to_Call_Model"></a>Adjust Configurations for Squid to Call Model</h2>
 <p>Now that we have a deployed model, let&#x2019;s adjust the configurations for the Squid topology to annotate the messages with the output of the model.</p>
-
 <ul>
-  
+
 <li>Edit the squid parser configuration at <tt>$METRON_HOME/config/zookeeper/parsers/squid.json</tt> in your favorite text editor and add a new FieldTransformation to indicate a threat alert based on the model (note the addition of <tt>is_malicious</tt> and <tt>is_alert</tt>):</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;parserClassName&quot;: &quot;org.apache.metron.parsers.GrokParser&quot;,
   &quot;sensorTopic&quot;: &quot;squid&quot;,
   &quot;parserConfig&quot;: {
@@ -449,13 +303,13 @@ su - hdfs -c &quot;hadoop fs -chown metron:metron /user/metron&quot;
 </pre></div></div>
 
 <ul>
-  
+
 <li>Edit the squid enrichment configuration at <tt>$METRON_HOME/config/zookeeper/enrichments/squid.json</tt> (this file will not exist, so create a new one) to make the threat triage adjust the level of risk based on the model output:</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;enrichment&quot; : {
     &quot;fieldMap&quot;: {}
   },
@@ -475,56 +329,43 @@ su - hdfs -c &quot;hadoop fs -chown metron:metron /user/metron&quot;
 </pre></div></div>
 
 <ul>
-  
+
 <li>Upload new configs via <tt>$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -i $METRON_HOME/config/zookeeper -z node1:2181</tt></li>
-  
 <li>Make the Squid topic in kafka via <tt>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper node1:2181 --create --topic squid --partitions 1 --replication-factor 1</tt></li>
 </ul></div>
 <div class="section">
 <h2><a name="Start_Topologies_and_Send_Data"></a>Start Topologies and Send Data</h2>
 <p>Now we need to start the topologies and send some data:</p>
-
 <ul>
-  
+
 <li>Start the squid topology via <tt>$METRON_HOME/bin/start_parser_topology.sh -k node1:6667 -z node1:2181 -s squid</tt></li>
-  
 <li>Generate some data via the squid client:
-  
 <ul>
-    
+
 <li>Generate a legit example: <tt>squidclient http://yahoo.com</tt></li>
-    
 <li>Generate a malicious example: <tt>squidclient http://cnn.com</tt></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Send the data to kafka via <tt>cat /var/log/squid/access.log | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic squid</tt></li>
-  
 <li>Browse the data in elasticsearch via the ES Head plugin @ <a class="externalLink" href="http://node1:9200/_plugin/head/">http://node1:9200/_plugin/head/</a> and verify that in the squid index you have two documents
-  
 <ul>
-    
+
 <li>One from <tt>yahoo.com</tt> which does not have <tt>is_alert</tt> set and does have <tt>is_malicious</tt> set to <tt>legit</tt></li>
-    
 <li>One from <tt>cnn.com</tt> which does have <tt>is_alert</tt> set to <tt>true</tt>, <tt>is_malicious</tt> set to <tt>malicious</tt> and <tt>threat:triage:level</tt> set to 100</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[11/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-sensors/fastcapa/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-sensors/fastcapa/index.html b/site/current-book/metron-sensors/fastcapa/index.html
index 59beb04..1b23080 100644
--- a/site/current-book/metron-sensors/fastcapa/index.html
+++ b/site/current-book/metron-sensors/fastcapa/index.html
@@ -1,256 +1,137 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-sensors/fastcapa/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Fastcapa</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Fastcapa</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Fastcapa</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                                    
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-down"></i>
-        Sensors</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Fastcapa</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-sensors/pycapa/index.html" title="Pycapa">
-          <i class="none"></i>
-        Pycapa</a>
-            </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-down"></span>Sensors</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>Fastcapa</a></li>
+    <li><a href="../../metron-sensors/pycapa/index.html" title="Pycapa"><span class="none"></span>Pycapa</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Fastcapa</h1>
-<p>Fastcapa is a probe that performs fast network packet capture by leveraging Linux kernel-bypass and user space networking technology. The probe will bind to a network interface, capture network packets, and send the raw packet data to Kafka. This provides a scalable mechanism for ingesting high-volumes of network packet data into a Hadoop-y cluster.</p>
-<p>Fastcapa leverages the Data Plane Development Kit (<a class="externalLink" href="http://dpdk.org/">DPDK</a>). DPDK is a set of libraries and drivers to perform fast packet processing in Linux user space. </p>
-
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Fastcapa</h1>
+<p>Fastcapa is a probe that performs fast network packet capture by leveraging Linux kernel-bypass and user space networking technology.  The probe will bind to a network interface, capture network packets, and send the raw packet data to Kafka.  This provides a scalable mechanism for ingesting high-volumes of network packet data into a Hadoop-y cluster.</p>
+<p>Fastcapa leverages the Data Plane Development Kit (<a class="externalLink" href="http://dpdk.org/">DPDK</a>).  DPDK is a set of libraries and drivers to perform fast packet processing in Linux user space.</p>
 <ul>
-  
+
 <li><a href="#Quick_Start">Quick Start</a></li>
-  
 <li><a href="#Requirements">Requirements</a></li>
-  
 <li><a href="#Installation">Installation</a></li>
-  
 <li><a href="#Usage">Usage</a>
-  
 <ul>
-    
+
 <li><a href="#Parameters">Parameters</a></li>
-    
 <li><a href="#Output">Output</a></li>
-    
 <li><a href="#Kerberos">Kerberos</a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><a href="#How_It_Works">How It Works</a></li>
-  
 <li><a href="#Performance">Performance</a></li>
-  
 <li><a href="#FAQs">FAQs</a></li>
 </ul>
 <div class="section">
 <h2><a name="Quick_Start"></a>Quick Start</h2>
-<p>The quickest way to see Fastcapa in action is to use a Virtualbox environment on your local machine. The necessary files and instructions to do this are located at <a href="../../metron-deployment/vagrant/fastcapa-test-platform/index.html"><tt>metron-deployment/vagrant/fastcapa-vagrant</tt></a>. All you need to do is execute the following.</p>
+<p>The quickest way to see Fastcapa in action is to use a Virtualbox environment on your local machine.  The necessary files and instructions to do this are located at <a href="../../metron-deployment/vagrant/fastcapa-test-platform/index.html"><tt>metron-deployment/vagrant/fastcapa-vagrant</tt></a>. All you need to do is execute the following.</p>
 
-<div class="source">
-<div class="source">
-<pre>cd metron-deployment/vagrant/fastcapa-test-platform
+<div>
+<div>
+<pre class="source">cd metron-deployment/vagrant/fastcapa-test-platform
 vagrant up
 </pre></div></div>
-<p>This environment sets up two nodes. One node produces network packets over a network interface. The second node uses Fastcapa to capture those packets and write them to a Kafka broker running on the first node. Basic validation is performed to ensure that Fastcapa is able to land packet data in Kafka.</p></div>
+
+<p>This environment sets up two nodes.  One node produces network packets over a network interface.  The second node uses Fastcapa to capture those packets and write them to a Kafka broker running on the first node.  Basic validation is performed to ensure that Fastcapa is able to land packet data in Kafka.</p></div>
 <div class="section">
 <h2><a name="Requirements"></a>Requirements</h2>
 <p>The following system requirements must be met to run the Fastcapa probe.</p>
-
 <ul>
-  
+
 <li>Linux kernel &gt;= 2.6.34</li>
-  
 <li>A <a class="externalLink" href="http://dpdk.org/doc/nics">DPDK supported ethernet device; NIC</a>.</li>
-  
 <li>Port(s) on the ethernet device that can be dedicated for exclusive use by Fastcapa</li>
 </ul></div>
 <div class="section">
@@ -258,265 +139,279 @@ vagrant up
 <p>The process of installing Fastcapa has a fair number of steps and involves building DPDK, loading specific kernel modules, enabling huge page memory, and binding compatible network interface cards.</p>
 <div class="section">
 <h3><a name="Automated_Installation"></a>Automated Installation</h3>
-<p>The best documentation is code that actually does this for you. An Ansible role that performs the entire installation procedure can be found at <a href="../../metron-deployment/roles/fastcapa/index.html"><tt>metron-deployment/roles/fastcapa</tt></a>. Use this to install Fastcapa or as a guide for manual installation. The automated installation assumes CentOS 7.1 and is directly tested against <a class="externalLink" href="https://atlas.hashicorp.com/bento/boxes/centos-7.1">bento/centos-7.1</a>.</p></div>
+<p>The best documentation is code that actually does this for you.  An Ansible role that performs the entire installation procedure can be found at <a href="../../metron-deployment/roles/fastcapa/index.html"><tt>metron-deployment/roles/fastcapa</tt></a>.  Use this to install Fastcapa or as a guide for manual installation.  The automated installation assumes CentOS 7.1 and is directly tested against <a class="externalLink" href="https://atlas.hashicorp.com/bento/boxes/centos-7.1">bento/centos-7.1</a>.</p></div>
 <div class="section">
 <h3><a name="Manual_Installation"></a>Manual Installation</h3>
-<p>The following manual installation steps assume that they are executed on CentOS 7.1. Some minor differences may result if you use a different Linux distribution.</p>
-
+<p>The following manual installation steps assume that they are executed on CentOS 7.1.  Some minor differences may result if you use a different Linux distribution.</p>
 <ul>
-  
+
 <li><a href="#Enable_Transparent_Huge_Pages">Enable Transparent Huge Pages</a></li>
-  
 <li><a href="#Install_DPDK">Install DPDK</a></li>
-  
 <li><a href="#Install_Librdkafka">Install Librdkafka</a></li>
-  
 <li><a href="#Install_Fastcapa">Install Fastcapa</a></li>
 </ul>
 <div class="section">
 <h4><a name="Enable_Transparent_Huge_Pages"></a>Enable Transparent Huge Pages</h4>
-<p>The probe performs its own memory management by leveraging transparent huge pages. In Linux, Transparent Huge Pages (THP) can be enabled either dynamically or on boot. It is recommended that these be allocated on boot to increase the chance that a larger, physically contiguous chunk of memory can be allocated.</p>
-<p>The size of THPs that are supported will vary based on your CPU. These typically include 2 MB and 1 GB THPs. For better performance, allocate 1 GB THPs if supported by your CPU.</p>
-
+<p>The probe performs its own memory management by leveraging transparent huge pages.  In Linux, Transparent Huge Pages (THP) can be enabled either dynamically or on boot.  It is recommended that these be allocated on boot to increase the chance that a larger, physically contiguous chunk of memory can be allocated.</p>
+<p>The size of THPs that are supported will vary based on your CPU.  These typically include 2 MB and 1 GB THPs.  For better performance, allocate 1 GB THPs if supported by your CPU.</p>
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Ensure that your CPU supports 1 GB THPs. A CPU flag <tt>pdpe1gb</tt> indicates whether or not the CPU supports 1 GB THPs.</p>
-  
-<div class="source">
-<div class="source">
-<pre>grep --color=always pdpe1gb /proc/cpuinfo | uniq
-</pre></div></div></li>
-  
+
+<p>Ensure that your CPU supports 1 GB THPs.  A CPU flag <tt>pdpe1gb</tt> indicates whether or not the CPU supports 1 GB THPs.</p>
+
+<div>
+<div>
+<pre class="source">grep --color=always pdpe1gb /proc/cpuinfo | uniq
+</pre></div></div>
+</li>
 <li>
-<p>Add the following boot parameters to the Linux kernel. Edit <tt>/etc/default/grub</tt> and add the additional kernel parameters to the line starting with <tt>GRUB_CMDLINE_LINUX</tt>.</p>
-  
-<div class="source">
-<div class="source">
-<pre>GRUB_CMDLINE_LINUX=... default_hugepagesz=1G hugepagesz=1G hugepages=16
-</pre></div></div></li>
-  
+
+<p>Add the following boot parameters to the Linux kernel.  Edit <tt>/etc/default/grub</tt> and add the additional kernel parameters to the line starting with <tt>GRUB_CMDLINE_LINUX</tt>.</p>
+
+<div>
+<div>
+<pre class="source">GRUB_CMDLINE_LINUX=... default_hugepagesz=1G hugepagesz=1G hugepages=16
+</pre></div></div>
+</li>
 <li>
-<p>Rebuild the grub configuration then reboot. The location of the Grub configuration file will differ across Linux distributions.</p>
-  
-<div class="source">
-<div class="source">
-<pre>cp /etc/grub2-efi.cfg /etc/grub2-efi.cfg.orig
+
+<p>Rebuild the grub configuration then reboot.  The location of the Grub configuration file will differ across Linux distributions.</p>
+
+<div>
+<div>
+<pre class="source">cp /etc/grub2-efi.cfg /etc/grub2-efi.cfg.orig
 /sbin/grub2-mkconfig -o /etc/grub2-efi.cfg
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Once the host has been rebooted, ensure that the THPs were successfully allocated.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ grep HugePage /proc/meminfo
+
+<div>
+<div>
+<pre class="source">$ grep HugePage /proc/meminfo
 AnonHugePages:    933888 kB
 HugePages_Total:      16
 HugePages_Free:        0
 HugePages_Rsvd:        0
 HugePages_Surp:        0
 </pre></div></div>
-<p>The total number of huge pages that you have been allocated should be distributed fairly evenly across each NUMA node. In this example, a total of 16 were requested and 8 have been assigned on each of the 2 NUMA nodes.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ cat /sys/devices/system/node/node*/hugepages/hugepages-1048576kB/nr_hugepages
+
+<p>The total number of huge pages that you have been allocated should be distributed fairly evenly across each NUMA node.  In this example, a total of 16 were requested and 8 have been assigned on each of the 2 NUMA nodes.</p>
+
+<div>
+<div>
+<pre class="source">$ cat /sys/devices/system/node/node*/hugepages/hugepages-1048576kB/nr_hugepages
 8
 8
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Once the THPs have been reserved, they need to be mounted to make them available to the probe.</p>
-  
-<div class="source">
-<div class="source">
-<pre>cp /etc/fstab /etc/fstab.orig
+
+<div>
+<div>
+<pre class="source">cp /etc/fstab /etc/fstab.orig
 mkdir -p /mnt/huge_1GB
 echo &quot;nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0&quot; &gt;&gt; /etc/fstab
 mount -fav
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h4><a name="Install_DPDK"></a>Install DPDK</h4>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Install the required dependencies.</p>
-  
-<div class="source">
-<div class="source">
-<pre>yum -y install &quot;@Development tools&quot;
+
+<div>
+<div>
+<pre class="source">yum -y install &quot;@Development tools&quot;
 yum -y install pciutils net-tools glib2 glib2-devel git
 yum -y install kernel kernel-devel kernel-headers
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Decide where DPDK will be installed.</p>
-  
-<div class="source">
-<div class="source">
-<pre>export DPDK_HOME=/usr/local/dpdk/
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">export DPDK_HOME=/usr/local/dpdk/
+</pre></div></div>
+</li>
 <li>
+
 <p>Download, build and install DPDK.</p>
-  
-<div class="source">
-<div class="source">
-<pre>wget http://fast.dpdk.org/rel/dpdk-16.11.1.tar.xz -O - | tar -xJ
+
+<div>
+<div>
+<pre class="source">wget http://fast.dpdk.org/rel/dpdk-16.11.1.tar.xz -O - | tar -xJ
 cd dpdk-stable-16.11.1/
 make config install T=x86_64-native-linuxapp-gcc DESTDIR=$DPDK_HOME
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Find the PCI address of the ethernet device that you plan on using to capture network packets. In the following example I plan on binding <tt>enp9s0f0</tt> which has a PCI address of <tt>09:00.0</tt>.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ lspci | grep &quot;VIC Ethernet&quot;
+
+<p>Find the PCI address of the ethernet device that you plan on using to capture network packets.  In the following example I plan on binding <tt>enp9s0f0</tt> which has a PCI address of <tt>09:00.0</tt>.</p>
+
+<div>
+<div>
+<pre class="source">$ lspci | grep &quot;VIC Ethernet&quot;
 09:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2)
 0a:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2)
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Bind the device. Replace the device name and PCI address with what is appropriate for your environment.</p>
-  
-<div class="source">
-<div class="source">
-<pre>ifdown enp9s0f0
+
+<p>Bind the device.  Replace the device name and PCI address with what is appropriate for your environment.</p>
+
+<div>
+<div>
+<pre class="source">ifdown enp9s0f0
 modprobe uio_pci_generic
 $DPDK_HOME/sbin/dpdk-devbind --bind=uio_pci_generic &quot;09:00.0&quot;
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Ensure that the device was bound. It should be shown as a &#x2018;network device using DPDK-compatible driver.&#x2019;</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ dpdk-devbind --status
+
+<div>
+<div>
+<pre class="source">$ dpdk-devbind --status
 Network devices using DPDK-compatible driver
 ============================================
 0000:09:00.0 'VIC Ethernet NIC' drv=uio_pci_generic unused=enic
 Network devices using kernel driver
 ===================================
 0000:01:00.0 'I350 Gigabit Network Connection' if=eno1 drv=igb unused=uio_pci_generic
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h4><a name="Install_Librdkafka"></a>Install Librdkafka</h4>
 <p>The probe has been tested with <a class="externalLink" href="https://github.com/edenhill/librdkafka/releases/tag/v0.9.4">Librdkafka 0.9.4</a>.</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Choose an installation path. In this example, the libs will actually be installed at <tt>/usr/local/lib</tt>; note that <tt>lib</tt> is appended to the prefix.</p>
-  
-<div class="source">
-<div class="source">
-<pre>export RDK_PREFIX=/usr/local
-</pre></div></div></li>
-  
+
+<p>Choose an installation path.  In this example, the libs will actually be installed at <tt>/usr/local/lib</tt>; note that <tt>lib</tt> is appended to the prefix.</p>
+
+<div>
+<div>
+<pre class="source">export RDK_PREFIX=/usr/local
+</pre></div></div>
+</li>
 <li>
+
 <p>Download, build and install.</p>
-  
-<div class="source">
-<div class="source">
-<pre>wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz  -O - | tar -xz
+
+<div>
+<div>
+<pre class="source">wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz  -O - | tar -xz
 cd librdkafka-0.9.4/
 ./configure --prefix=$RDK_PREFIX
 make
 make install
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Ensure that the installation location is on the search path for the runtime shared library loader.</p>
-  
-<div class="source">
-<div class="source">
-<pre>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$RDK_PREFIX/lib
-</pre></div></div></li>
+
+<div>
+<div>
+<pre class="source">export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$RDK_PREFIX/lib
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h4><a name="Install_Fastcapa"></a>Install Fastcapa</h4>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Set the required environment variables.</p>
-  
-<div class="source">
-<div class="source">
-<pre>export RTE_SDK=$DPDK_HOME/share/dpdk/
+
+<div>
+<div>
+<pre class="source">export RTE_SDK=$DPDK_HOME/share/dpdk/
 export RTE_TARGET=x86_64-native-linuxapp-gcc
 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$RDK_HOME
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Build Fastcapa. The resulting binary will be placed at <tt>build/app/fastcapa</tt>.</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron/metron-sensors/fastcapa
+
+<p>Build Fastcapa.  The resulting binary will be placed at <tt>build/app/fastcapa</tt>.</p>
+
+<div>
+<div>
+<pre class="source">cd metron/metron-sensors/fastcapa
 make
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div></div></div>
 <div class="section">
 <h2><a name="Usage"></a>Usage</h2>
 <p>Follow these steps to run Fastcapa.</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Create a configuration file that at a minimum specifies your Kafka broker. An example configuration file, <tt>conf/fastcapa.conf</tt>, is available that documents other useful parameters.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[kafka-global]
+
+<p>Create a configuration file that at a minimum specifies your Kafka broker.  An example configuration file, <tt>conf/fastcapa.conf</tt>, is available that documents other useful parameters.</p>
+
+<div>
+<div>
+<pre class="source">[kafka-global]
 metadata.broker.list = kafka-broker1:9092
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Bind the capture device. This is only needed if the device is not already bound. In this example, the device <tt>enp9s0f0</tt> with a PCI address of <tt>09:00:0</tt> is bound. Use values specific to your environment.</p>
-  
-<div class="source">
-<div class="source">
-<pre>ifdown enp9s0f0
+
+<p>Bind the capture device.  This is only needed if the device is not already bound.  In this example, the device <tt>enp9s0f0</tt> with a PCI address of <tt>09:00:0</tt> is bound.  Use values specific to your environment.</p>
+
+<div>
+<div>
+<pre class="source">ifdown enp9s0f0
 modprobe uio_pci_generic
 $DPDK_HOME/sbin/dpdk-devbind --bind=uio_pci_generic &quot;09:00.0&quot;
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Run Fastcapa.</p>
-  
-<div class="source">
-<div class="source">
-<pre>fastcapa -c 0x03 --huge-dir /mnt/huge_1GB -- -p 0x01 -t pcap -c /etc/fastcapa.conf
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">fastcapa -c 0x03 --huge-dir /mnt/huge_1GB -- -p 0x01 -t pcap -c /etc/fastcapa.conf
+</pre></div></div>
+</li>
 <li>
-<p>Terminate Fastcapa with <tt>SIGINT</tt> or by entering <tt>CTRL-C</tt>. The probe will cleanly shut down all of the workers and allow the backlog of packets to drain. To terminate the process without clearing the queue, send a <tt>SIGKILL</tt> or be entering <tt>killall -9 fastcapa</tt>.</p></li>
+
+<p>Terminate Fastcapa with <tt>SIGINT</tt> or by entering <tt>CTRL-C</tt>.  The probe will cleanly shut down all of the workers and allow the backlog of packets to drain.  To terminate the process without clearing the queue, send a <tt>SIGKILL</tt> or be entering <tt>killall -9 fastcapa</tt>.</p>
+</li>
 </ol>
 <div class="section">
 <h3><a name="Parameters"></a>Parameters</h3>
-<p>Fastcapa accepts three sets of parameters. </p>
-
+<p>Fastcapa accepts three sets of parameters.</p>
 <ol style="list-style-type: decimal">
-  
+
 <li>Command-line parameters passed directly to DPDK&#x2019;s Environmental Abstraction Layer (EAL)</li>
-  
-<li>Command-line parameters that define how Fastcapa will interact with DPDK. These parametera are separated on the command line by a double-dash (<tt>--</tt>).</li>
-  
+<li>Command-line parameters that define how Fastcapa will interact with DPDK.  These parametera are separated on the command line by a  double-dash (<tt>--</tt>).</li>
 <li>A configuration file that define how Fastcapa interacts with Librdkafka.</li>
 </ol>
 <div class="section">
 <h4><a name="Environmental_Abstraction_Layer_Parameters"></a>Environmental Abstraction Layer Parameters</h4>
-<p>The most commonly used EAL parameter involves specifying which logical CPU cores should be used for processing. This can be specified in any of the following ways.</p>
+<p>The most commonly used EAL parameter involves specifying which logical CPU cores should be used for processing.  This can be specified in any of the following ways.</p>
 
-<div class="source">
-<div class="source">
-<pre>  -c COREMASK         Hexadecimal bitmask of cores to run on
+<div>
+<div>
+<pre class="source">  -c COREMASK         Hexadecimal bitmask of cores to run on
   -l CORELIST         List of cores to run on
                       The argument format is &lt;c1&gt;[-c2][,c3[-c4],...]
                       where c1, c2, etc are core indexes between 0 and 128
@@ -529,136 +424,78 @@ $DPDK_HOME/sbin/dpdk-devbind --bind=uio_pci_generic &quot;09:00.0&quot;
                       '( )' can be omitted for single element group,
                       '@' can be omitted if cpus and lcores have the same value                     
 </pre></div></div>
+
 <p>To get more information about other EAL parameters, run the following.</p>
 
-<div class="source">
-<div class="source">
-<pre>fastcapa -h
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">fastcapa -h
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Fastcapa-Core_Parameters"></a>Fastcapa-Core Parameters</h4>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Name </th>
-      
-<th>Command </th>
-      
-<th>Description </th>
-      
-<th>Default </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Name </th>
+<th> Command </th>
+<th> Description </th>
+<th> Default </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>Port Mask </td>
-      
-<td>-p PORT_MASK </td>
-      
-<td>A bit mask identifying which ports to bind. </td>
-      
-<td>0x01 </td>
-    </tr>
-    
+<td> Port Mask </td>
+<td> -p PORT_MASK </td>
+<td> A bit mask identifying which ports to bind. </td>
+<td> 0x01 </td></tr>
 <tr class="a">
-      
-<td>Receive Burst Size </td>
-      
-<td>-b RX_BURST_SIZE </td>
-      
-<td>The max number of packets processed by a receive worker. </td>
-      
-<td>32 </td>
-    </tr>
-    
+<td> Receive Burst Size </td>
+<td> -b RX_BURST_SIZE </td>
+<td> The max number of packets processed by a receive worker. </td>
+<td> 32 </td></tr>
 <tr class="b">
-      
-<td>Transmit Burst Size </td>
-      
-<td>-w TX_BURST_SIZE </td>
-      
-<td>The max number of packets processed by a transmit worker. </td>
-      
-<td>256 </td>
-    </tr>
-    
+<td> Transmit Burst Size </td>
+<td> -w TX_BURST_SIZE </td>
+<td> The max number of packets processed by a transmit worker.  </td>
+<td> 256 </td></tr>
 <tr class="a">
-      
-<td>Receive Descriptors </td>
-      
-<td>-d NB_RX_DESC </td>
-      
-<td>The number of descriptors for each receive queue (the size of the receive queue.) Limited by the ethernet device in use. </td>
-      
-<td>1024 </td>
-    </tr>
-    
+<td> Receive Descriptors </td>
+<td> -d NB_RX_DESC </td>
+<td> The number of descriptors for each receive queue (the size of the receive queue.)  Limited by the ethernet device in use. </td>
+<td> 1024 </td></tr>
 <tr class="b">
-      
-<td>Transmission Ring Size </td>
-      
-<td>-x TX_RING_SIZE </td>
-      
-<td>The size of each transmission ring. This must be a power of 2. </td>
-      
-<td>2048 </td>
-    </tr>
-    
+<td> Transmission Ring Size </td>
+<td> -x TX_RING_SIZE </td>
+<td> The size of each transmission ring.  This must be a power of 2. </td>
+<td> 2048 </td></tr>
 <tr class="a">
-      
-<td>Number Receive Queues </td>
-      
-<td>-q NB_RX_QUEUE </td>
-      
-<td>Number of receive queues to use for each port. Limited by the ethernet device in use. </td>
-      
-<td>2 </td>
-    </tr>
-    
+<td> Number Receive Queues </td>
+<td> -q NB_RX_QUEUE </td>
+<td> Number of receive queues to use for each port.  Limited by the ethernet device in use. </td>
+<td> 2 </td></tr>
 <tr class="b">
-      
-<td>Kafka Topic </td>
-      
-<td>-t KAFKA_TOPIC </td>
-      
-<td>The name of the Kafka topic. </td>
-      
-<td>pcap </td>
-    </tr>
-    
+<td> Kafka Topic </td>
+<td> -t KAFKA_TOPIC </td>
+<td> The name of the Kafka topic. </td>
+<td> pcap </td></tr>
 <tr class="a">
-      
-<td>Configuration File </td>
-      
-<td>-c KAFKA_CONF </td>
-      
-<td>Path to a file containing configuration values. </td>
-      
-<td> </td>
-    </tr>
-    
+<td> Configuration File </td>
+<td> -c KAFKA_CONF </td>
+<td> Path to a file containing configuration values. </td>
+<td>  </td></tr>
 <tr class="b">
-      
-<td>Stats </td>
-      
-<td>-s KAFKA_STATS </td>
-      
-<td>Appends performance metrics in the form of JSON strings to the specified file. </td>
-      
-<td> </td>
-    </tr>
-  </tbody>
+<td> Stats </td>
+<td> -s KAFKA_STATS </td>
+<td> Appends performance metrics in the form of JSON strings to the specified file. </td>
+<td>  </td></tr>
+</tbody>
 </table>
-<p>To get more information about the Fastcapa specific parameters, run the following. Note that this puts the <tt>-h</tt> after the double-dash <tt>--</tt>.</p>
+<p>To get more information about the Fastcapa specific parameters, run the following.  Note that this puts the <tt>-h</tt> after the double-dash <tt>--</tt>.</p>
 
-<div class="source">
-<div class="source">
-<pre>fastcapa -- -h
+<div>
+<div>
+<pre class="source">fastcapa -- -h
 
 fastcapa [EAL options] -- [APP options]
   -p PORT_MASK        bitmask of ports to bind                     [0x01]
@@ -671,168 +508,100 @@ fastcapa [EAL options] -- [APP options]
   -c KAFKA_CONF       file containing configs for kafka client
   -s KAFKA_STATS      append kafka client stats to a file
   -h                  print this help message
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Fastcapa-Kafka_Configuration_File"></a>Fastcapa-Kafka Configuration File</h4>
-<p>The path to the configuration file is specified with the <tt>-c</tt> command line argument. The file can contain any global or topic-specific, producer-focused <a class="externalLink" href="https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md">configuration values accepted by Librdkafka</a>. </p>
-<p>The configuration file is a <tt>.ini</tt>-like Glib configuration file. The global configuration values should be placed under a <tt>[kafka-global]</tt> header and topic-specific values should be placed under <tt>[kafka-topic]</tt>.</p>
+<p>The path to the configuration file is specified with the <tt>-c</tt> command line argument.  The file can contain any global or topic-specific, producer-focused <a class="externalLink" href="https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md">configuration values accepted by Librdkafka</a>.</p>
+<p>The configuration file is a <tt>.ini</tt>-like Glib configuration file.  The global configuration values should be placed under a <tt>[kafka-global]</tt> header and topic-specific values should be placed under <tt>[kafka-topic]</tt>.</p>
 <p>A minimally viable configuration file would only need to include the Kafka broker to connect to.</p>
 
-<div class="source">
-<div class="source">
-<pre>[kafka-global]
+<div>
+<div>
+<pre class="source">[kafka-global]
 metadata.broker.list = kafka-broker1:9092, kafka-broker2:9092
 </pre></div></div>
+
 <p>The configuration parameters that are important for either basic functioning or performance tuning of Fastcapa include the following.</p>
 <p>Global configuration values that should be located under the <tt>[kafka-global]</tt> header.</p>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th><i>Name</i> </th>
-      
-<th><i>Description</i> </th>
-      
-<th><i>Default</i> </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> <i>Name</i> </th>
+<th> <i>Description</i> </th>
+<th> <i>Default</i> </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>metadata.broker.list </td>
-      
-<td>Initial list of brokers as a CSV list of broker host or host:port </td>
-      
-<td> </td>
-    </tr>
-    
+<td> metadata.broker.list </td>
+<td> Initial list of brokers as a CSV list of broker host or host:port </td>
+<td>  </td></tr>
 <tr class="a">
-      
-<td>client.id </td>
-      
-<td>Client identifier. </td>
-      
-<td> </td>
-    </tr>
-    
+<td> client.id </td>
+<td> Client identifier. </td>
+<td>  </td></tr>
 <tr class="b">
-      
-<td>queue.buffering.max.messages </td>
-      
-<td>Maximum number of messages allowed on the producer queue </td>
-      
-<td>100000 </td>
-    </tr>
-    
+<td> queue.buffering.max.messages </td>
+<td> Maximum number of messages allowed on the producer queue </td>
+<td> 100000 </td></tr>
 <tr class="a">
-      
-<td>queue.buffering.max.ms </td>
-      
-<td>Maximum time, in milliseconds, for buffering data on the producer queue </td>
-      
-<td>1000 </td>
-    </tr>
-    
+<td> queue.buffering.max.ms </td>
+<td> Maximum time, in milliseconds, for buffering data on the producer queue </td>
+<td> 1000 </td></tr>
 <tr class="b">
-      
-<td>message.copy.max.bytes </td>
-      
-<td>Maximum size for message to be copied to buffer. Messages larger than this will be passed by reference (zero-copy) at the expense of larger iovecs. </td>
-      
-<td>65535 </td>
-    </tr>
-    
+<td> message.copy.max.bytes </td>
+<td> Maximum size for message to be copied to buffer. Messages larger than this will be passed by reference (zero-copy) at the expense of larger iovecs. </td>
+<td> 65535 </td></tr>
 <tr class="a">
-      
-<td>batch.num.messages </td>
-      
-<td>Maximum number of messages batched in one MessageSet </td>
-      
-<td>10000 </td>
-    </tr>
-    
+<td> batch.num.messages </td>
+<td> Maximum number of messages batched in one MessageSet </td>
+<td> 10000 </td></tr>
 <tr class="b">
-      
-<td>statistics.interval.ms </td>
-      
-<td>How often statistics are emitted; 0 = never </td>
-      
-<td>0 </td>
-    </tr>
-    
+<td> statistics.interval.ms </td>
+<td> How often statistics are emitted; 0 = never </td>
+<td> 0 </td></tr>
 <tr class="a">
-      
-<td>compression.codec </td>
-      
-<td>Compression codec to use for compressing message sets; {none, gzip, snappy, lz4 } </td>
-      
-<td>none </td>
-    </tr>
-  </tbody>
+<td> compression.codec </td>
+<td> Compression codec to use for compressing message sets; {none, gzip, snappy, lz4 } </td>
+<td> none </td></tr>
+</tbody>
 </table>
 <p>Topic configuration values that should be located under the <tt>[kafka-topic]</tt> header.</p>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th><i>Name</i> </th>
-      
-<th><i>Description</i> </th>
-      
-<th><i>Default</i> </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> <i>Name</i> </th>
+<th> <i>Description</i> </th>
+<th> <i>Default</i> </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>compression.codec </td>
-      
-<td>Compression codec to use for compressing message sets; {none, gzip, snappy, lz4 } </td>
-      
-<td>none </td>
-    </tr>
-    
+<td> compression.codec </td>
+<td> Compression codec to use for compressing message sets; {none, gzip, snappy, lz4 } </td>
+<td> none </td></tr>
 <tr class="a">
-      
-<td>request.required.acks </td>
-      
-<td>How many acknowledgements the leader broker must receive from ISR brokers before responding to the request; { 0 = no ack, 1 = leader ack, -1 = all ISRs } </td>
-      
-<td>1 </td>
-    </tr>
-    
+<td> request.required.acks </td>
+<td> How many acknowledgements the leader broker must receive from ISR brokers before responding to the request; { 0 = no ack, 1 = leader ack, -1 = all ISRs } </td>
+<td> 1 </td></tr>
 <tr class="b">
-      
-<td>message.timeout.ms </td>
-      
-<td>Local message timeout. This value is only enforced locally and limits the time a produced message waits for successful delivery. A time of 0 is infinite. </td>
-      
-<td>300000 </td>
-    </tr>
-    
+<td> message.timeout.ms </td>
+<td> Local message timeout. This value is only enforced locally and limits the time a produced message waits for successful delivery. A time of 0 is infinite. </td>
+<td> 300000 </td></tr>
 <tr class="a">
-      
-<td>queue.buffering.max.kbytes </td>
-      
-<td>Maximum total message size sum allowed on the producer queue </td>
-      
-<td> </td>
-    </tr>
-  </tbody>
+<td> queue.buffering.max.kbytes </td>
+<td> Maximum total message size sum allowed on the producer queue </td>
+<td>  </td></tr>
+</tbody>
 </table></div></div>
 <div class="section">
 <h3><a name="Output"></a>Output</h3>
-<p>When running the probe some basic counters are output to stdout. Of course during normal operation these values will be much larger.</p>
+<p>When running the probe some basic counters are output to stdout.  Of course during normal operation these values will be much larger.</p>
 
-<div class="source">
-<div class="source">
-<pre>     ------ in ------  --- queued --- ----- out ----- ---- drops ----
+<div>
+<div>
+<pre class="source">     ------ in ------  --- queued --- ----- out ----- ---- drops ----
 [nic]               8               -               -               -
 [rx]                8               0                8                0
 [tx]                8               0                8                0
@@ -840,168 +609,160 @@ metadata.broker.list = kafka-broker1:9092, kafka-broker2:9092
 </pre></div></div>
 
 <ul>
-  
-<li><tt>[nic]</tt> + <tt>in</tt>: The ethernet device is reporting that it has seen 8 packets.</li>
-  
+
+<li><tt>[nic]</tt> +  <tt>in</tt>:  The ethernet device is reporting that it has seen 8 packets.</li>
 <li><tt>[rx]</tt> + <tt>in</tt>: The receive workers have consumed 8 packets from the device.</li>
-  
 <li><tt>[rx]</tt> + <tt>out</tt>: The receive workers have enqueued 8 packets onto the transmission rings.</li>
-  
-<li><tt>[rx]</tt> + <tt>drops</tt>: If the transmission rings become full it will prevent the receive workers from enqueuing additional packets. The excess packets are dropped. This value will never decrease.</li>
-  
+<li><tt>[rx]</tt> + <tt>drops</tt>: If the transmission rings become full it will prevent the receive workers from enqueuing additional packets.  The excess packets are dropped.  This value will never decrease.</li>
 <li><tt>[tx]</tt> + <tt>in</tt>: The transmission workers have consumed 8 packets.</li>
-  
 <li><tt>[tx]</tt> + <tt>out</tt>: The transmission workers have packaged 8 packets into Kafka messages.</li>
-  
-<li><tt>[tx]</tt> + <tt>drops</tt>: If the Kafka client library accepted fewer packets than expected. This value can increase or decrease over time as additional packets are acknowledged by the Kafka client library at a later point in time.</li>
-  
+<li><tt>[tx]</tt> + <tt>drops</tt>: If the Kafka client library accepted fewer packets than expected.  This value can increase or decrease over time as additional packets are acknowledged by the Kafka client library at a later point in time.</li>
 <li><tt>[kaf]</tt> + <tt>in</tt>: The Kafka client library has received 8 packets.</li>
-  
 <li><tt>[kaf]</tt> + <tt>out</tt>: A total of 7 packets has successfully reached Kafka.</li>
-  
 <li><tt>[kaf]</tt> + <tt>queued</tt>: There is 1 packet within the <tt>rdkafka</tt> queue waiting to be sent.</li>
 </ul></div>
 <div class="section">
 <h3><a name="Kerberos"></a>Kerberos</h3>
-<p>The probe can be used in a Kerberized environment. Follow these additional steps to use Fastcapa with Kerberos. The following assumptions have been made. These may need altered to fit your environment.</p>
-
+<p>The probe can be used in a Kerberized environment.  Follow these additional steps to use Fastcapa with Kerberos.  The following assumptions have been made.  These may need altered to fit your environment.</p>
 <ul>
-  
+
 <li>The Kafka broker is at <tt>kafka1:6667</tt></li>
-  
 <li>Zookeeper is at <tt>zookeeper1:2181</tt></li>
-  
 <li>The Kafka security protocol is <tt>SASL_PLAINTEXT</tt></li>
-  
 <li>The keytab used is located at <tt>/etc/security/keytabs/metron.headless.keytab</tt></li>
-  
 <li>The service principal is <tt>metron@EXAMPLE.COM</tt></li>
 </ul>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Build Librdkafka with SASL support (<tt>--enable-sasl</tt>).</p>
-  
-<div class="source">
-<div class="source">
-<pre>wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz  -O - | tar -xz
+
+<div>
+<div>
+<pre class="source">wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz  -O - | tar -xz
 cd librdkafka-0.9.4/
 ./configure --prefix=$RDK_PREFIX --enable-sasl
 make
 make install
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Validate Librdkafka does indeed support SASL. Run the following command and ensure that <tt>sasl</tt> is returned as a built-in feature.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ examples/rdkafka_example -X builtin.features
+
+<p>Validate Librdkafka does indeed support SASL.  Run the following command and ensure that <tt>sasl</tt> is returned as a built-in feature.</p>
+
+<div>
+<div>
+<pre class="source">$ examples/rdkafka_example -X builtin.features
 builtin.features = gzip,snappy,ssl,sasl,regex
 </pre></div></div>
-<p>If it is not, ensure that you have <tt>libsasl</tt> or <tt>libsasl2</tt> installed. On CentOS, this can be installed with the following command.</p>
-  
-<div class="source">
-<div class="source">
-<pre>yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi
-</pre></div></div></li>
-  
+
+<p>If it is not, ensure that you have <tt>libsasl</tt> or <tt>libsasl2</tt> installed.  On CentOS, this can be installed with the following command.</p>
+
+<div>
+<div>
+<pre class="source">yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi
+</pre></div></div>
+</li>
 <li>
-<p>Grant access to your Kafka topic. In this example, it is simply named <tt>pcap</tt>.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$KAFKA_HOME/bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer \
+
+<p>Grant access to your Kafka topic.  In this example, it is simply named <tt>pcap</tt>.</p>
+
+<div>
+<div>
+<pre class="source">$KAFKA_HOME/bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer \
   --authorizer-properties zookeeper.connect=zookeeper1:2181 \
   --add --allow-principal User:metron --topic pcap
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Obtain a Kerberos ticket.</p>
-  
-<div class="source">
-<div class="source">
-<pre>kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM
+</pre></div></div>
+</li>
 <li>
+
 <p>Add the following additional configuration values to your Fastcapa configuration file.</p>
-  
-<div class="source">
-<div class="source">
-<pre>security.protocol = SASL_PLAINTEXT
+
+<div>
+<div>
+<pre class="source">security.protocol = SASL_PLAINTEXT
 sasl.kerberos.keytab = /etc/security/keytabs/metron.headless.keytab
 sasl.kerberos.principal = metron@EXAMPLE.COM
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Now run Fastcapa as you normally would. It should have no problem landing packets in your kerberized Kafka broker.</p></li>
+
+<p>Now run Fastcapa as you normally would.  It should have no problem landing packets in your kerberized Kafka broker.</p>
+</li>
 </ol></div></div>
 <div class="section">
 <h2><a name="How_It_Works"></a>How It Works</h2>
-<p>The probe leverages a poll-mode, burst-oriented mechanism to capture packets from a network interface and transmit them efficiently to a Kafka topic. Each packet is wrapped within a single Kafka message and the current timestamp, as epoch microseconds in network byte order, is attached as the message&#x2019;s key.</p>
-<p>The probe leverages Receive Side Scaling (RSS), a feature provided by some ethernet devices that allows processing of received data to occur across multiple processes and logical cores. It does this by running a hash function on each packet, whose value assigns the packet to one, of possibly many, receive queues. The total number and size of these receive queues are limited by the ethernet device in use. More capable ethernet devices will offer a greater number and greater sized receive queues. </p>
-
+<p>The probe leverages a poll-mode, burst-oriented mechanism to capture packets from a network interface and transmit them efficiently to a Kafka topic.  Each packet is wrapped within a single Kafka message and the current timestamp, as epoch microseconds in network byte order, is attached as the message&#x2019;s key.</p>
+<p>The probe leverages Receive Side Scaling (RSS), a feature provided by some ethernet devices that allows processing of received data to occur across multiple processes and logical cores.  It does this by running a hash function on each packet, whose value assigns the packet to one, of possibly many, receive queues.  The total number and size of these receive queues are limited by the ethernet device in use.  More capable ethernet devices will offer a greater number and greater sized receive queues.</p>
 <ul>
-  
+
 <li>Increasing the number of receive queues allows for greater parallelization of receive side processing.</li>
-  
 <li>Increasing the size of each receive queue can allow the probe to handle larger, temporary spikes of network packets that can often occur.</li>
 </ul>
-<p>A set of receive workers, each assigned to a unique logical core, are responsible for fetching packets from the receive queues. There can only be one receive worker for each receive queue. The receive workers continually poll the receive queues and attempt to fetch multiple packets on each request. The maximum number of packets fetched in one request is known as the &#x2018;burst size&#x2019;. If the receive worker actually receives &#x2018;burst size&#x2019; packets, then it is likely that the queue is under pressure and more packets are available. In this case the worker immediately fetches another &#x2018;burst size&#x2019; set of packets. It repeats this process up to a fixed number of times while the queue is under pressure.</p>
-<p>The receive workers then enqueue the received packets into a fixed size ring buffer known as a transmit ring. There is always one transmit ring for each receive queue. A set of transmit workers then dequeue packets from the transmit rings. There can be one or more transmit workers assigned to any single transmit ring. Each transmit worker has its own unique connection to Kafka.</p>
-
+<p>A set of receive workers, each assigned to a unique logical core, are responsible for fetching packets from the receive queues.  There can only be one receive worker for each receive queue.  The receive workers continually poll the receive queues and attempt to fetch multiple packets on each request.  The maximum number of packets fetched in one request is known as the &#x2018;burst size&#x2019;.  If the receive worker actually receives &#x2018;burst size&#x2019; packets, then it is likely that the queue is under pressure and more packets are available.  In this case the worker immediately fetches another &#x2018;burst size&#x2019; set of packets.  It repeats this process up to a fixed number of times while the queue is under pressure.</p>
+<p>The receive workers then enqueue the received packets into a fixed size ring buffer known as a transmit ring.  There is always one transmit ring for each receive queue.  A set of transmit workers then dequeue packets from the transmit rings.  There can be one or more transmit workers assigned to any single transmit ring.  Each transmit worker has its own unique connection to Kafka.</p>
 <ul>
-  
+
 <li>Increasing the number of transmit workers allows for greater parallelization when writing data to Kafka.</li>
-  
 <li>Increasing the size of the transmit rings allows the probe to better handle temporary interruptions and latency when writing to Kafka.</li>
 </ul>
-<p>After receiving the network packets from the transmit worker, the Kafka client library internally maintains its own send queue of messages. Multiple threads are then responsible for managing this queue and creating batches of messages that are sent in bulk to a Kafka broker. No control is exercised over this additional send queue and its worker threads, which can be an impediment to performance. This is an opportunity for improvement that can be addressed as follow-on work.</p></div>
+<p>After receiving the network packets from the transmit worker, the Kafka client library internally maintains its own send queue of messages.  Multiple threads are then responsible for managing this queue and creating batches of messages that are sent in bulk to a Kafka broker.  No control is exercised over this additional send queue and its worker threads, which can be an impediment to performance.  This is an opportunity for improvement that can be addressed as follow-on work.</p></div>
 <div class="section">
 <h2><a name="Performance"></a>Performance</h2>
 <p>Beyond tuning the parameters previously described, the following should be carefully considered to achieve maximum performance.</p>
 <div class="section">
 <h3><a name="Kafka_Partitions"></a>Kafka Partitions</h3>
-<p>Parallelizing access to a topic in Kafka is achieved by defining multiple partitions. The greater the number of partitions, the more parallelizable access to that topic becomes. To achieve high throughput it is important to ensure that the Kafka topic in use has a large number of partitions, evenly distributed across each of the nodes in your Kafka cluster.</p>
+<p>Parallelizing access to a topic in Kafka is achieved by defining multiple partitions.  The greater the number of partitions, the more parallelizable access to that topic becomes.  To achieve high throughput it is important to ensure that the Kafka topic in use has a large number of partitions, evenly distributed across each of the nodes in your Kafka cluster.</p>
 <p>The specific number of partitions needed will differ for each environment, but at least 128 partitions has been shown to significantly increase performance in some environments.</p></div>
 <div class="section">
 <h3><a name="Physical_Layout"></a>Physical Layout</h3>
-<p>It is important to note the physical layout of the hardware when assigning worker cores to the probe. The worker cores should be on the same NUMA node or socket as the ethernet device itself. Assigning logical cores across NUMA boundaries can significantly impede performance.</p>
-<p>The following commands can help identify logical cores that are located on the same NUMA node or socket as the ethernet device itself. These commands should be run when the device is still managed by the kernel itself; before binding the interface.</p>
+<p>It is important to note the physical layout of the hardware when assigning worker cores to the probe.  The worker cores should be on the same NUMA node or socket as the ethernet device itself.  Assigning logical cores across NUMA boundaries can significantly impede performance.</p>
+<p>The following commands can help identify logical cores that are located on the same NUMA node or socket as the ethernet device itself.  These commands should be run when the device is still managed by the kernel itself; before binding the interface.</p>
 
-<div class="source">
-<div class="source">
-<pre>$ cat /sys/class/net/enp9s0f0/device/local_cpulist
+<div>
+<div>
+<pre class="source">$ cat /sys/class/net/enp9s0f0/device/local_cpulist
 0-7,16-23
 </pre></div></div>
+
 <p>The following command can be used to better understand the physical layout of the CPU in relation to NUMA nodes.</p>
 
-<div class="source">
-<div class="source">
-<pre>$ lscpu
+<div>
+<div>
+<pre class="source">$ lscpu
 ...
 NUMA node0 CPU(s):     0-7,16-23
 NUMA node1 CPU(s):     8-15,24-31
 </pre></div></div>
-<p>In this example <tt>enp9s0f0</tt> is located on NUMA node 0 and is local to the logical cores 0-7 and 16-23. You should choose worker cores from this list.</p></div>
+
+<p>In this example <tt>enp9s0f0</tt> is located on NUMA node 0 and is local to the logical cores 0-7 and 16-23.  You should choose worker cores from this list.</p></div>
 <div class="section">
 <h3><a name="CPU_Isolation"></a>CPU Isolation</h3>
-<p>Once you have chosen the logical cores to use that are local to the ethernet device, it also beneficial to isolate those cores so that the Linux kernel scheduler does not attempt to run tasks on them. This can be done using the <tt>isolcpus</tt> kernel boot parameter.</p>
+<p>Once you have chosen the logical cores to use that are local to the ethernet device, it also beneficial to isolate those cores so that the Linux kernel scheduler does not attempt to run tasks on them.  This can be done using the <tt>isolcpus</tt> kernel boot parameter.</p>
 
-<div class="source">
-<div class="source">
-<pre>isolcpus=0,1,2,3,4,5,6,7
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">isolcpus=0,1,2,3,4,5,6,7
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Device_Limitations"></a>Device Limitations</h3>
-<p>Check the output of running the probe to ensure that there are no device limitations that you are not aware of. While you may have specified 16 receive queues on the command line, your device may not support that number. This is especially true for the number of receive queues and descriptors.</p>
-<p>The following example shows the output when the number of receive descriptors requested is greater than what can be supported by the device. In many cases the probe will not terminate, but will choose the maximum allowable value and continue. This behavior is specific to the underlying device driver in use.</p>
+<p>Check the output of running the probe to ensure that there are no device limitations that you are not aware of.  While you may have specified 16 receive queues on the command line, your device may not support that number.  This is especially true for the number of receive queues and descriptors.</p>
+<p>The following example shows the output when the number of receive descriptors requested is greater than what can be supported by the device.  In many cases the probe will not terminate, but will choose the maximum allowable value and continue.  This behavior is specific to the underlying device driver in use.</p>
 
-<div class="source">
-<div class="source">
-<pre>PMD: rte_enic_pmd: Rq 0 Scatter rx mode enabled
+<div>
+<div>
+<pre class="source">PMD: rte_enic_pmd: Rq 0 Scatter rx mode enabled
 PMD: rte_enic_pmd: Rq 0 Scatter rx mode not being used
 PMD: rte_enic_pmd: Number of rx_descs too high, adjusting to maximum
 PMD: rte_enic_pmd: Using 512 rx descriptors (sop 512, data 0)
@@ -1011,7 +772,8 @@ PMD: rte_enic_pmd: Number of rx_descs too high, adjusting to maximum
 PMD: rte_enic_pmd: Using 512 rx descriptors (sop 512, data 0)
 PMD: rte_enic_pmd: TX Queues - effective number of descs:32
 PMD: rte_enic_pmd: vNIC resources used:  wq 1 rq 4 cq 3 intr 0
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="More_Information"></a>More Information</h3>
 <p>More information on this topic can be found in <a class="externalLink" href="http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html">DPDK&#x2019;s Getting Started Guide</a>.</p></div></div>
@@ -1021,81 +783,78 @@ PMD: rte_enic_pmd: vNIC resources used:  wq 1 rq 4 cq 3 intr 0
 <h3><a name="No_free_hugepages_reported"></a>No free hugepages reported</h3>
 <p>Problem: When executing <tt>fastcapa</tt> it fails with the following error message.</p>
 
-<div class="source">
-<div class="source">
-<pre>EAL: No free hugepages reported in hugepages-1048576kB
+<div>
+<div>
+<pre class="source">EAL: No free hugepages reported in hugepages-1048576kB
 PANIC in rte_eal_init():
 Cannot get hugepage information
 </pre></div></div>
-<p>Solution: This can occur if any process that has been allocated THPs crashes and fails to return the resources.</p>
 
+<p>Solution: This can occur if any process that has been allocated THPs crashes and fails to return the resources.</p>
 <ul>
-  
+
 <li>
+
 <p>Delete the THP files that are not in use.</p>
-  
-<div class="source">
-<div class="source">
-<pre>rm -f /mnt/huge_1GB/rtemap_*
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">rm -f /mnt/huge_1GB/rtemap_*
+</pre></div></div>
+</li>
 <li>
+
 <p>If the first option does not work, re-mount the <tt>hugetlbfs</tt> file system.</p>
-  
-<div class="source">
-<div class="source">
-<pre>umount -a -t hugetlbfs
+
+<div>
+<div>
+<pre class="source">umount -a -t hugetlbfs
 mount -a
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ul></div>
 <div class="section">
 <h3><a name="No_ethernet_ports_detected"></a>No ethernet ports detected</h3>
 <p>Problem: When executing <tt>fastcapa</tt> it fails with the following error message.</p>
 
-<div class="source">
-<div class="source">
-<pre>EAL: Error - exiting with code: 1
+<div>
+<div>
+<pre class="source">EAL: Error - exiting with code: 1
   Cause: No ethernet ports detected.
 </pre></div></div>
 
 <ul>
-  
+
 <li>Solution 1: The <tt>uio_pci_generic</tt> kernel module has not been loaded.</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>modprobe uio_pci_generic
+<div>
+<div>
+<pre class="source">modprobe uio_pci_generic
 </pre></div></div>
 
 <ul>
-  
+
 <li>Solution 2: Ensure that the ethernet device is bound. Re-bind if necessary.</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre> dpdk-devbind --unbind &quot;09:00.0&quot;
+<div>
+<div>
+<pre class="source"> dpdk-devbind --unbind &quot;09:00.0&quot;
  dpdk-devbind --bind=uio_pci_generic &quot;09:00.0&quot;
  dpdk-devbind --status
 </pre></div></div></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-sensors/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-sensors/index.html b/site/current-book/metron-sensors/index.html
index 8aff298..a40e51d 100644
--- a/site/current-book/metron-sensors/index.html
+++ b/site/current-book/metron-sensors/index.html
@@ -1,238 +1,127 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-sensors/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Metron Sensors</title>
-    <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../css/site.css" />
     <link rel="stylesheet" href="../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Metron Sensors</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron Sensors</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                              
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Sensors</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../metron-sensors/fastcapa/index.html" title="Fastcapa">
-          <i class="none"></i>
-        Fastcapa</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-sensors/pycapa/index.html" title="Pycapa">
-          <i class="none"></i>
-        Pycapa</a>
-            </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Sensors</a>
+    <ul class="nav nav-list">
+    <li><a href="../metron-sensors/fastcapa/index.html" title="Fastcapa"><span class="none"></span>Fastcapa</a></li>
+    <li><a href="../metron-sensors/pycapa/index.html" title="Pycapa"><span class="none"></span>Pycapa</a></li>
+    </ul>
+</li>
+    <li><a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <div class="section">
-<h2><a name="Metron_Sensors"></a>Metron Sensors</h2>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
 
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<div class="section">
+<h2><a name="Metron_Sensors"></a>Metron Sensors</h2>
 <ul>
-  
+
 <li>
-<p><a class="externalLink" href="https://github.com/apache/metron-bro-plugin-kafka"><tt>metron-bro-plugin-kafka</tt></a>: Provides integration between <a class="externalLink" href="https://www.bro.org/">Bro</a> and Kafka. A Bro plugin that sends logging output to Kafka. This provides a convenient means for tools in the Hadoop ecosystem, such as Storm, Spark, and others to process the data generated by Bro.</p></li>
-  
+
+<p><a class="externalLink" href="https://github.com/apache/metron-bro-plugin-kafka"><tt>metron-bro-plugin-kafka</tt></a>: Provides integration between <a class="externalLink" href="https://www.bro.org/">Bro</a> and Kafka.  A Bro plugin that sends logging output to Kafka.  This provides a convenient means for tools in the Hadoop ecosystem, such as Storm, Spark, and others to process the data generated by Bro.</p>
+</li>
 <li>
-<p><a href="fastcapa/index.html"><tt>fastcapa</tt></a>: Performs fast network packet capture by leveraging Linux kernel-bypass and user space networking technology. The probe will bind to a network interface, capture network packets, and send the raw packet data to Kafka. This provides a scalable mechanism for ingesting high-volumes of network packet data.</p></li>
-  
+
+<p><a href="fastcapa/index.html"><tt>fastcapa</tt></a>: Performs fast network packet capture by leveraging Linux kernel-bypass and user space networking technology.  The probe will bind to a network interface, capture network packets, and send the raw packet data to Kafka.  This provides a scalable mechanism for ingesting high-volumes of network packet data.</p>
+</li>
 <li>
-<p><a href="pycapa/index.html"><tt>pycapa</tt></a>: Performs lightweight network packet capture, retrieves network packets from Kafka, generates <tt>libpcap</tt>-compliant files, and enables integration with third-party tools like Wireshark.</p></li>
-</ul></div>
-                  </div>
-            </div>
-          </div>
 
+<p><a href="pycapa/index.html"><tt>pycapa</tt></a>: Performs lightweight network packet capture, retrieves network packets from Kafka, generates <tt>libpcap</tt>-compliant files, and enables integration with third-party tools like Wireshark.</p>
+</li>
+</ul></div>
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[46/50] [abbrv] metron git commit: METRON-1572 Enhance KAFKA_PUT function (nickwallen) closes apache/metron#1024

Posted by mm...@apache.org.
METRON-1572 Enhance KAFKA_PUT function (nickwallen) closes apache/metron#1024


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/40796c06
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/40796c06
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/40796c06

Branch: refs/heads/feature/METRON-1554-pcap-query-panel
Commit: 40796c06ad96ae45dd853925fbae8c26509f6c2f
Parents: ae1d3eb
Author: nickwallen <ni...@nickallen.org>
Authored: Fri Jun 8 08:39:20 2018 -0400
Committer: nickallen <ni...@apache.org>
Committed: Fri Jun 8 08:39:20 2018 -0400

----------------------------------------------------------------------
 .../metron/management/KafkaFunctions.java       | 87 +++++++++++++++-----
 .../KafkaFunctionsIntegrationTest.java          | 21 +++++
 2 files changed, 89 insertions(+), 19 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/40796c06/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java b/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
index 316e19d..a0c92eb 100644
--- a/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
+++ b/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
@@ -22,6 +22,7 @@ import org.apache.kafka.clients.consumer.ConsumerRecord;
 import org.apache.kafka.clients.consumer.KafkaConsumer;
 import org.apache.kafka.clients.producer.KafkaProducer;
 import org.apache.kafka.clients.producer.ProducerRecord;
+import org.apache.kafka.clients.producer.RecordMetadata;
 import org.apache.kafka.common.PartitionInfo;
 import org.apache.kafka.common.TopicPartition;
 import org.apache.kafka.common.serialization.StringDeserializer;
@@ -37,7 +38,7 @@ import org.slf4j.LoggerFactory;
 
 import java.lang.invoke.MethodHandles;
 import java.util.ArrayList;
-import java.util.Arrays;
+import java.util.Collections;
 import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
@@ -46,6 +47,9 @@ import java.util.Optional;
 import java.util.Properties;
 import java.util.Set;
 import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
 
 import static java.lang.String.format;
 import static org.apache.metron.stellar.dsl.Context.Capabilities.GLOBAL_CONFIG;
@@ -332,18 +336,26 @@ public class KafkaFunctions {
   /**
    * KAFKA_PUT
    *
-   * Sends messages to a Kafka topic.
+   * <p>Sends messages to a Kafka topic.
    *
-   * Example: Put two messages on the topic 'topic'.
+   * <p>Example: Put two messages on the topic 'topic'.
+   * <pre>
+   *  {@code
    *  KAFKA_PUT('topic', ["message1", "message2"])
+   *  }
+   * </pre>
    *
-   * Example: Put a message on a topic and also define an alternative Kafka broker.
+   * <p>Example: Put a message on a topic and also define an alternative Kafka broker.
+   * <pre>
+   *  {@code
    *  KAFKA_PUT('topic', ["message1"], { "bootstrap.servers": "kafka-broker-1:6667" })
+   *  }
+   * </pre>
    */
   @Stellar(
           namespace = "KAFKA",
           name = "PUT",
-          description = "Sends messages to a Kafka topic.",
+          description = "Sends messages to a Kafka topic. ",
           params = {
                   "topic - The name of the Kafka topic.",
                   "messages - A list of messages to write.",
@@ -355,45 +367,82 @@ public class KafkaFunctions {
 
     @Override
     public Object apply(List<Object> args, Context context) throws ParseException {
-
       String topic = ConversionUtils.convert(args.get(0), String.class);
-      List<String> messages = ConversionUtils.convert(args.get(1), List.class);
 
-      // build the properties for kafka
+      List<String> messages;
+      if(args.get(1) instanceof String) {
+        // a single message needs sent
+        String msg = ConversionUtils.convert(args.get(1), String.class);
+        messages = Collections.singletonList(msg);
+
+      } else {
+        // a list of messages; all need sent
+        messages = ConversionUtils.convert(args.get(1), List.class);
+      }
+
+      // are there any overrides?
       Map<String, String> overrides = new HashMap<>();
       if(args.size() > 2) {
         overrides = ConversionUtils.convert(args.get(2), Map.class);
       }
-      Properties properties = buildKafkaProperties(overrides, context);
 
       // send the messages
-      try {
-        send(topic, messages, properties);
-
-      } catch(InterruptedException | ExecutionException e) {
-        throw new ParseException(e.getMessage(), e);
-      }
+      Properties properties = buildKafkaProperties(overrides, context);
+      putMessages(topic, messages, properties);
 
       return null;
     }
 
     /**
-     * Send each message synchronously.
+     * Put messages to a Kafka topic.
+     *
+     * <p>Sends each message synchronously.
+     *
      * @param topic The topic to send messages to.
      * @param messages The messages to send.
      * @param properties The properties to use with Kafka.
      */
-    private void send(String topic, List<String> messages, Properties properties) throws InterruptedException, ExecutionException {
+    private void putMessages(String topic, List<String> messages, Properties properties) {
+      LOG.debug("KAFKA_PUT sending messages; topic={}, count={}", topic, messages.size());
       try (KafkaProducer<String, String> producer = new KafkaProducer<>(properties)) {
 
-        // send each message synchronously, hence the get()
+        List<Future<RecordMetadata>> futures = new ArrayList<>();
+
+        // send each message
         for(String msg : messages) {
-          producer.send(new ProducerRecord<>(topic, msg)).get();
+          Future<RecordMetadata> future = producer.send(new ProducerRecord<>(topic, msg));
+          futures.add(future);
+        }
+
+        // wait for the sends to complete
+        for(Future<RecordMetadata> future : futures) {
+          waitForResponse(future, properties);
         }
+
         producer.flush();
       }
     }
 
+    /**
+     * Wait for response to the message being sent.
+     *
+     * @param future The future for the message being sent.
+     * @param properties The configuration properties.
+     * @return
+     */
+    private void waitForResponse(Future<RecordMetadata> future, Properties properties) {
+      int maxWait = getMaxWait(properties);
+      try {
+        // wait for the record and then render it for the user
+        RecordMetadata record = future.get(maxWait, TimeUnit.MILLISECONDS);
+        LOG.debug("KAFKA_PUT message sent; topic={}, partition={}, offset={}",
+                record.topic(), record.partition(), record.offset());
+
+      } catch(TimeoutException | InterruptedException | ExecutionException e) {
+        LOG.error("KAFKA_PUT message send failure", e);
+      }
+    }
+
     @Override
     public void initialize(Context context) {
       // no initialization required

http://git-wip-us.apache.org/repos/asf/metron/blob/40796c06/metron-platform/metron-management/src/test/java/org/apache/metron/management/KafkaFunctionsIntegrationTest.java
----------------------------------------------------------------------
diff --git a/metron-platform/metron-management/src/test/java/org/apache/metron/management/KafkaFunctionsIntegrationTest.java b/metron-platform/metron-management/src/test/java/org/apache/metron/management/KafkaFunctionsIntegrationTest.java
index 74c6705..ad45b52 100644
--- a/metron-platform/metron-management/src/test/java/org/apache/metron/management/KafkaFunctionsIntegrationTest.java
+++ b/metron-platform/metron-management/src/test/java/org/apache/metron/management/KafkaFunctionsIntegrationTest.java
@@ -161,6 +161,26 @@ public class KafkaFunctionsIntegrationTest extends BaseIntegrationTest {
   }
 
   /**
+   * KAFKA_PUT should be able to write a message passed as a String, rather than a List.
+   */
+  @Test
+  public void testKafkaPutOneMessagePassedAsString() {
+
+    // use a unique topic name for this test
+    final String topicName = testName.getMethodName();
+    variables.put("topic", topicName);
+
+    // put a message onto the topic - the message is just a string, not a list
+    run("KAFKA_PUT(topic, message1)");
+
+    // get a message from the topic
+    Object actual = run("KAFKA_GET(topic)");
+
+    // validate
+    assertEquals(Collections.singletonList(message1), actual);
+  }
+
+  /**
    * KAFKA_PUT should be able to write multiple messages passed as a List.
    * KAFKA_GET should be able to read multiple messages at once.
    */
@@ -373,3 +393,4 @@ public class KafkaFunctionsIntegrationTest extends BaseIntegrationTest {
     }
   }
 }
+


[19/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-data-management/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-data-management/index.html b/site/current-book/metron-platform/metron-data-management/index.html
index df34389..dea600c 100644
--- a/site/current-book/metron-platform/metron-data-management/index.html
+++ b/site/current-book/metron-platform/metron-data-management/index.html
@@ -1,359 +1,190 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-data-management/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Resource Data Management</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Resource Data Management</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Resource Data Management</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Data-management</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Resource Data Management</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Resource Data Management</h1>
 <p><a name="Resource_Data_Management"></a></p>
-<p>This project is a collection of classes to assist with loading of various enrichment and threat intelligence sources into Metron.</p>
 <div class="section">
-<h2><a name="Simple_HBase_EnrichmentsThreat_Intelligence"></a>Simple HBase Enrichments/Threat Intelligence</h2>
-<p>The vast majority of enrichments and threat intelligence processing tend toward the following pattern:</p>
+<h2><a name="Table_of_Contents"></a>Table of Contents</h2>
+<ul>
 
+<li><a href="#Overview">Overview</a></li>
+<li><a href="#Simple_HBase_EnrichmentsThreat_Intelligence">Simple HBase Enrichments/Threat Intelligence</a></li>
+<li><a href="#Extractor_Framework">Extractor Framework</a></li>
+<li><a href="#Enrichment_Config">Enrichment Config</a></li>
+<li><a href="#Loading_Utilities">Loading Utilities</a></li>
+<li><a href="#Pruning_Data_from_Elasticsearch">Pruning Data from Elasticsearch</a></li>
+</ul></div>
+<div class="section">
+<h2><a name="Overview"></a>Overview</h2>
+<p>This project is a collection of classes to assist with loading of various enrichment and threat intelligence sources into Metron.</p></div>
+<div class="section">
+<h2><a name="Simple_HBase_Enrichments.2FThreat_Intelligence"></a>Simple HBase Enrichments/Threat Intelligence</h2>
+<p>The vast majority of enrichments and threat intelligence processing tend toward the following pattern:</p>
 <ul>
-  
+
 <li>Take a field</li>
-  
 <li>Look up the field in a key/value store</li>
-  
 <li>If the key exists, then either it&#x2019;s a threat to be alerted or it should be enriched with the value associated with the key.</li>
 </ul>
-<p>As such, we have created this capability as a default threat intel and enrichment adapter. The basic primitive for simple enrichments and threat intelligence sources is a complex key containing the following:</p>
-
+<p>As such, we have created this capability as a default threat intel and enrichment adapter.  The basic primitive for simple enrichments and threat intelligence sources is a complex key containing the following:</p>
 <ul>
-  
+
 <li>Type : The type of threat intel or enrichment (e.g. malicious_ip)</li>
-  
 <li>Indicator : The indicator in question</li>
-  
-<li>Value : The value to associate with the type, indicator pair. This is a JSON map.</li>
+<li>Value : The value to associate with the type, indicator pair.  This is a JSON map.</li>
 </ul>
 <p>At present, all of the dataloads utilities function by converting raw data sources to this primitive key (type, indicator) and value to be placed in HBase.</p>
 <p>In the case of threat intel, a hit on the threat intel table will result in:</p>
-
 <ul>
-  
+
 <li>The <tt>is_alert</tt> field being set to <tt>true</tt> in the index</li>
-  
 <li>A field named <tt>threatintels.hbaseThreatIntel.$field.$threatintel_type</tt> is set to <tt>alert</tt>
-  
 <ul>
-    
+
 <li><tt>$field</tt> is the field in the original document that was a match (e.g. <tt>src_ip_addr</tt>)</li>
-    
 <li><tt>$threatintel_type</tt> is the type of threat intel imported (defined in the Extractor configuration below).</li>
-  </ul></li>
 </ul>
-<p>In the case of simple hbase enrichment, a hit on the enrichments table will result in the following new field for each key in the value:<tt>enrichments.hbaseEnrichment.$field.$enrichment_type.$key</tt> </p>
-
+</li>
+</ul>
+<p>In the case of simple hbase enrichment, a hit on the enrichments table will result in the following new field for each key in the value:<tt>enrichments.hbaseEnrichment.$field.$enrichment_type.$key</tt></p>
 <ul>
-  
-<li><tt>$field</tt> is the field in the original document that was a match (e.g. <tt>src_ip_addr</tt>)</li>
-  
+
+<li><tt>$field</tt> is the field in the original document that was a match (e.g.  <tt>src_ip_addr</tt>)</li>
 <li><tt>$enrichment_type</tt> is the type of enrichment imported (defined in the Extractor configuration below).</li>
-  
 <li><tt>$key</tt> is a key in the JSON map associated with the row in HBase.</li>
 </ul>
 <p>For instance, in the situation where we had the following very silly key/value in HBase in the enrichment table:</p>
-
 <ul>
-  
+
 <li>indicator: <tt>127.0.0.1</tt></li>
-  
 <li>type : <tt>important_addresses</tt></li>
-  
 <li>value: <tt>{ &quot;name&quot; : &quot;localhost&quot;, &quot;location&quot; : &quot;home&quot; }</tt></li>
 </ul>
 <p>If we had a document whose <tt>ip_src_addr</tt> came through with a value of <tt>127.0.0.1</tt>, we would have the following fields added to the indexed document:</p>
-
 <ul>
-  
+
 <li><tt>enrichments.hbaseEnrichment.ip_src_addr.important_addresses.name</tt> : <tt>localhost</tt></li>
-  
 <li><tt>enrichments.hbaseEnrichment.ip_src_addr.important_addresses.location</tt> : <tt>home</tt></li>
 </ul></div>
 <div class="section">
 <h2><a name="Extractor_Framework"></a>Extractor Framework</h2>
-<p>For the purpose of ingesting data of a variety of formats, we have created an Extractor framework which allows for common data formats to be interpreted as enrichment or threat intelligence sources. The formats supported at present are:</p>
-
+<p>For the purpose of ingesting data of a variety of formats, we have created an Extractor framework which allows for common data formats to be interpreted as enrichment or threat intelligence sources.  The formats supported at present are:</p>
 <ul>
-  
+
 <li>CSV (both threat intel and enrichment)</li>
-  
 <li>STIX (threat intel only)</li>
-  
 <li>Custom (pass your own class)</li>
 </ul>
-<p>All of the current utilities take a JSON file to configure how to interpret input data. This JSON describes the type of data and the schema if necessary for the data if it is not fixed (as in STIX, e.g.).</p>
+<p>All of the current utilities take a JSON file to configure how to interpret input data.  This JSON describes the type of data and the schema if necessary for the data if it is not fixed (as in STIX, e.g.).</p>
 <div class="section">
 <h3><a name="CSV_Extractor"></a>CSV Extractor</h3>
 <p>Consider the following example configuration file which describes how to process a CSV file.</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;config&quot; : {
     &quot;columns&quot; : {
          &quot;ip&quot; : 0
@@ -366,129 +197,86 @@
   ,&quot;extractor&quot; : &quot;CSV&quot;
 }
 </pre></div></div>
-<p>In this example, we have instructed the extractor of the schema (i.e. the columns field), two columns at the first and third position. We have indicated that the <tt>ip</tt> column is the indicator type and that the enrichment type is named <tt>malicious_ip</tt>. We have also indicated that the extractor to use is the CSV Extractor. The other option is the STIX extractor or a fully qualified classname for your own extractor.</p>
-<p>The meta column values will show up in the value in HBase because it is called out as a non-indicator column. The key for the value will be &#x2018;meta&#x2019;. For instance, given an input string of <tt>123.45.123.12,something,the grapevine</tt>, the following key, value would be extracted:</p>
 
+<p>In this example, we have instructed the extractor of the schema (i.e. the columns field), two columns at the first and third position.  We have indicated that the <tt>ip</tt> column is the indicator type and that the enrichment type is named <tt>malicious_ip</tt>.  We have also indicated that the extractor to use is the CSV Extractor. The other option is the STIX extractor or a fully qualified classname for your own extractor.</p>
+<p>The meta column values will show up in the value in HBase because it is called out as a non-indicator column.  The key for the value will be &#x2018;meta&#x2019;.  For instance, given an input string of <tt>123.45.123.12,something,the grapevine</tt>, the following key, value would be extracted:</p>
 <ul>
-  
+
 <li>Indicator : <tt>123.45.123.12</tt></li>
-  
 <li>Type : <tt>malicious_ip</tt></li>
-  
 <li>Value : <tt>{ &quot;ip&quot; : &quot;123.45.123.12&quot;, &quot;source&quot; : &quot;the grapevine&quot; }</tt></li>
 </ul></div>
 <div class="section">
 <h3><a name="STIX_Extractor"></a>STIX Extractor</h3>
-<p>Consider the following config for importing STIX documents. This is a threat intelligence interchange format, so it is particularly relevant and attractive data to import for our purposes. Because STIX is a standard format, there is no need to specify the schema or how to interpret the documents.</p>
+<p>Consider the following config for importing STIX documents.  This is a threat intelligence interchange format, so it is particularly relevant and attractive data to import for our purposes.  Because STIX is a standard format, there is no need to specify the schema or how to interpret the documents.</p>
 <p>We support the versions of Stix and Cybox supported by <a class="externalLink" href="https://github.com/STIXProject/java-stix/tree/v1.2.0.2">java-stix</a>:</p>
-
 <ul>
-  
+
 <li>Stix - <a class="externalLink" href="https://github.com/STIXProject/schemas/blob/356cc4f6b06625465f0808388eb166807313b4e0/stix_core.xsd">1.2</a> and earlier</li>
-  
 <li>Cybox - <a class="externalLink" href="https://github.com/CybOXProject/schemas/blob/97beb32c376a9223e91b52cb3e4c8d2af6baf786/cybox_core.xsd">2.1</a> and earlier</li>
 </ul>
 <p>We support a subset of STIX messages for importation:</p>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>STIX Type </th>
-      
-<th>Specific Type </th>
-      
-<th>Enrichment Type Name </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> STIX Type </th>
+<th> Specific Type </th>
+<th> Enrichment Type Name </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>Address </td>
-      
-<td>IPV_4_ADDR </td>
-      
-<td>address:IPV_4_ADDR </td>
-    </tr>
-    
+<td> Address   </td>
+<td> IPV_4_ADDR    </td>
+<td> address:IPV_4_ADDR   </td></tr>
 <tr class="a">
-      
-<td>Address </td>
-      
-<td>IPV_6_ADDR </td>
-      
-<td>address:IPV_6_ADDR </td>
-    </tr>
-    
+<td> Address   </td>
+<td> IPV_6_ADDR    </td>
+<td> address:IPV_6_ADDR   </td></tr>
 <tr class="b">
-      
-<td>Address </td>
-      
-<td>E_MAIL </td>
-      
-<td>address:E_MAIL </td>
-    </tr>
-    
+<td> Address   </td>
+<td> E_MAIL        </td>
+<td> address:E_MAIL       </td></tr>
 <tr class="a">
-      
-<td>Address </td>
-      
-<td>MAC </td>
-      
-<td>address:MAC </td>
-    </tr>
-    
+<td> Address   </td>
+<td> MAC           </td>
+<td> address:MAC          </td></tr>
 <tr class="b">
-      
-<td>Domain </td>
-      
-<td>FQDN </td>
-      
-<td>domain:FQDN </td>
-    </tr>
-    
+<td> Domain    </td>
+<td> FQDN          </td>
+<td> domain:FQDN          </td></tr>
 <tr class="a">
-      
-<td>Hostname </td>
-      
-<td> </td>
-      
-<td>hostname </td>
-    </tr>
-    
+<td> Hostname  </td>
+<td>               </td>
+<td> hostname             </td></tr>
 <tr class="b">
-      
-<td>URI </td>
-      
-<td> </td>
-      
-<td>uriobjecttype </td>
-    </tr>
-  </tbody>
+<td> URI       </td>
+<td>               </td>
+<td> uriobjecttype        </td></tr>
+</tbody>
 </table>
 <p>NOTE: The enrichment type will be used as the type above.</p>
 <p>Consider the following configuration for an Extractor</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;config&quot; : {
     &quot;stix_address_categories&quot; : &quot;IPV_4_ADDR&quot;
   }
   ,&quot;extractor&quot; : &quot;STIX&quot;
 }
 </pre></div></div>
-<p>In here, we&#x2019;re configuring the STIX extractor to load from a series of STIX files, however we only want to bring in IPv4 addresses from the set of all possible addresses. Note that if no categories are specified for import, all are assumed. Also, only address and domain types allow filtering via <tt>stix_address_categories</tt> and <tt>stix_domain_categories</tt> config parameters.</p></div>
+
+<p>In here, we&#x2019;re configuring the STIX extractor to load from a series of STIX files, however we only want to bring in IPv4 addresses from the set of all possible addresses.  Note that if no categories are specified for import, all are assumed. Also, only address and domain types allow filtering via <tt>stix_address_categories</tt> and <tt>stix_domain_categories</tt> config parameters.</p></div>
 <div class="section">
 <h3><a name="Common_Extractor_Properties"></a>Common Extractor Properties</h3>
 <p>Users also have the ability to transform and filter enrichment and threat intel data using Stellar as it is loaded into HBase. This feature is available to all extractor types.</p>
 <p>As an example, we will be providing a CSV list of top domains as an enrichment and filtering the value metadata, as well as the indicator column, with Stellar expressions.</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;config&quot; : {
     &quot;zk_quorum&quot; : &quot;node1:2181&quot;,
     &quot;columns&quot; : {
@@ -510,154 +298,117 @@
   &quot;extractor&quot; : &quot;CSV&quot;
 }
 </pre></div></div>
-<p>There are 2 property maps that work with full Stellar expressions, and 2 properties that will work with Stellar predicates.</p>
 
+<p>There are 2 property maps that work with full Stellar expressions, and 2 properties that will work with Stellar predicates.</p>
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
+<tr class="a">
+<th> Property             </th>
+<th> Description</th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td> <tt>value_transform</tt>    </td>
+<td> Transform fields defined in the &#x201c;columns&#x201d; mapping with Stellar transformations. New keys introduced in the transform will be added to the key metadata.</td></tr>
 <tr class="a">
-      
-<th>Property </th>
-      
-<th>Description</th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<td> <tt>value_filter</tt>       </td>
+<td> Allows additional filtering with Stellar predicates based on results from the value transformations. In this example, records whose domain property is empty after removing the TLD will be omitted.</td></tr>
 <tr class="b">
-      
-<td>value_transform </td>
-      
-<td>Transform fields defined in the &#x201c;columns&#x201d; mapping with Stellar transformations. New keys introduced in the transform will be added to the key metadata.</td>
-    </tr>
-    
+<td> <tt>indicator_transform</tt></td>
+<td> Transform the indicator column independent of the value transformations. You can refer to the original indicator value by using &#x201c;indicator&#x201d; as the variable name, as shown in the example above. In addition, if you prefer to piggyback your transformations, you can refer to the variable &#x201c;domain&#x201d;, which will allow your indicator transforms to inherit transformations done to this value during the value transformations.</td></tr>
 <tr class="a">
-      
-<td>value_filter </td>
-      
-<td>Allows additional filtering with Stellar predicates based on results from the value transformations. In this example, records whose domain property is empty after removing the TLD will be omitted.</td>
-    </tr>
-    
+<td> <tt>indicator_filter</tt>   </td>
+<td> Allows additional filtering with Stellar predicates based on results from the value transformations. In this example, records whose indicator value is empty after removing the TLD will be omitted.</td></tr>
 <tr class="b">
-      
-<td>indicator_transform </td>
-      
-<td>Transform the indicator column independent of the value transformations. You can refer to the original indicator value by using &#x201c;indicator&#x201d; as the variable name, as shown in the example above. In addition, if you prefer to piggyback your transformations, you can refer to the variable &#x201c;domain&#x201d;, which will allow your indicator transforms to inherit transformations done to this value during the value transformations.</td>
-    </tr>
-    
+<td> <tt>state_init</tt>         </td>
+<td> Allows a state object to be initialized.  This is a string, so a single expression is created.  The output of this expression will be available as the <tt>state</tt> variable.  This is to be used with the <tt>flatfile_summarizer.sh</tt> rather than the loader.</td></tr>
 <tr class="a">
-      
-<td>indicator_filter </td>
-      
-<td>Allows additional filtering with Stellar predicates based on results from the value transformations. In this example, records whose indicator value is empty after removing the TLD will be omitted.</td>
-    </tr>
-  </tbody>
+<td> <tt>state_update</tt>       </td>
+<td> Allows a state object to be updated.  This is a map, so you can have temporary variables here.  Note that you can reference the <tt>state</tt> variable from this.  This is to be used with the <tt>flatfile_summarizer.sh</tt> rather than the loader.</td></tr>
+<tr class="b">
+<td> <tt>state_merge</tt>        </td>
+<td> Allows a list of states to be merged. This is a string, so a single expression.  There is a special field called <tt>states</tt> available, which is a list of the states (one per thread).  This is to be used with the <tt>flatfile_summarizer.sh</tt> rather than the loader.</td></tr>
+</tbody>
 </table>
 <p>top-list.csv</p>
 
-<div class="source">
-<div class="source">
-<pre>1,google.com
+<div>
+<div>
+<pre class="source">1,google.com
 2,youtube.com
 ...
 </pre></div></div>
-<p>Running a file import with the above data and extractor configuration would result in the following 2 extracted data records:</p>
 
+<p>Running a file import with the above data and extractor configuration would result in the following 2 extracted data records:</p>
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Indicator </th>
-      
-<th>Type </th>
-      
-<th>Value </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Indicator </th>
+<th> Type        </th>
+<th> Value                                 </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>google </td>
-      
-<td>top_domains </td>
-      
-<td>{ &#x201c;rank&#x201d; : &#x201c;1&#x201d;, &#x201c;domain&#x201d; : &#x201c;google&#x201d; } </td>
-    </tr>
-    
+<td> google    </td>
+<td> top_domains </td>
+<td> { &#x201c;rank&#x201d; : &#x201c;1&#x201d;, &#x201c;domain&#x201d; : &#x201c;google&#x201d; } </td></tr>
 <tr class="a">
-      
-<td>yahoo </td>
-      
-<td>top_domains </td>
-      
-<td>{ &#x201c;rank&#x201d; : &#x201c;2&#x201d;, &#x201c;domain&#x201d; : &#x201c;yahoo&#x201d; } </td>
-    </tr>
-  </tbody>
+<td> yahoo     </td>
+<td> top_domains </td>
+<td> { &#x201c;rank&#x201d; : &#x201c;2&#x201d;, &#x201c;domain&#x201d; : &#x201c;yahoo&#x201d; }  </td></tr>
+</tbody>
 </table>
 <p>Similar to the parser framework, providing a Zookeeper quorum via the zk_quorum property will enable Stellar to access properties that reside in the global config. Expanding on our example above, if the global config looks as follows:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
     &quot;global_property&quot; : &quot;metron-ftw&quot;
 }
 </pre></div></div>
+
 <p>And we expand our value_tranform:</p>
 
-<div class="source">
-<div class="source">
-<pre>...
+<div>
+<div>
+<pre class="source">...
     &quot;value_transform&quot; : {
        &quot;domain&quot; : &quot;DOMAIN_REMOVE_TLD(domain)&quot;,
        &quot;a-new-prop&quot; : &quot;global_property&quot;
     },
 ...
 </pre></div></div>
-<p>The resulting value data would look like the following:</p>
 
+<p>The resulting value data would look like the following:</p>
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Indicator </th>
-      
-<th>Type </th>
-      
-<th>Value </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Indicator </th>
+<th> Type        </th>
+<th> Value                                                              </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>google </td>
-      
-<td>top_domains </td>
-      
-<td>{ &#x201c;rank&#x201d; : &#x201c;1&#x201d;, &#x201c;domain&#x201d; : &#x201c;google&#x201d;, &#x201c;a-new-prop&#x201d; : &#x201c;metron-ftw&#x201d; } </td>
-    </tr>
-    
+<td> google    </td>
+<td> top_domains </td>
+<td> { &#x201c;rank&#x201d; : &#x201c;1&#x201d;, &#x201c;domain&#x201d; : &#x201c;google&#x201d;, &#x201c;a-new-prop&#x201d; : &#x201c;metron-ftw&#x201d; } </td></tr>
 <tr class="a">
-      
-<td>yahoo </td>
-      
-<td>top_domains </td>
-      
-<td>{ &#x201c;rank&#x201d; : &#x201c;2&#x201d;, &#x201c;domain&#x201d; : &#x201c;yahoo&#x201d;, &#x201c;a-new-prop&#x201d; : &#x201c;metron-ftw&#x201d; } </td>
-    </tr>
-  </tbody>
+<td> yahoo     </td>
+<td> top_domains </td>
+<td> { &#x201c;rank&#x201d; : &#x201c;2&#x201d;, &#x201c;domain&#x201d; : &#x201c;yahoo&#x201d;, &#x201c;a-new-prop&#x201d; : &#x201c;metron-ftw&#x201d; }  </td></tr>
+</tbody>
 </table></div></div>
 <div class="section">
 <h2><a name="Enrichment_Config"></a>Enrichment Config</h2>
-<p>In order to automatically add new enrichment and threat intel types to existing, running enrichment topologies, you will need to add new fields and new types to the zookeeper configuration. A convenience parameter has been made to assist in this when doing an import. Namely, you can specify the enrichment configs and how they associate with the fields of the documents flowing through the enrichment topology.</p>
-<p>Consider the following Enrichment Configuration JSON. This one is for a threat intelligence type:</p>
+<p>In order to automatically add new enrichment and threat intel types to existing, running enrichment topologies, you will need to add new fields and new types to the zookeeper configuration.  A convenience parameter has been made to assist in this when doing an import.  Namely, you can specify the enrichment configs and how they associate with the fields of the documents flowing through the enrichment topology.</p>
+<p>Consider the following Enrichment Configuration JSON.  This one is for a threat intelligence type:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;zkQuorum&quot; : &quot;localhost:2181&quot;
  ,&quot;sensorToFieldList&quot; : {
     &quot;bro&quot; : {
@@ -670,35 +421,32 @@
                         }
 }
 </pre></div></div>
-<p>We have to specify the following:</p>
 
+<p>We have to specify the following:</p>
 <ul>
-  
+
 <li>The zookeeper quorum which holds the cluster configuration</li>
-  
 <li>The mapping between the fields in the enriched documents and the enrichment types.</li>
 </ul>
 <p>This configuration allows the ingestion tools to update zookeeper post-ingestion so that the enrichment topology can take advantage immediately of the new type.</p></div>
 <div class="section">
 <h2><a name="Loading_Utilities"></a>Loading Utilities</h2>
 <p>The two configurations above are used in the three separate ingestion tools:</p>
-
 <ul>
-  
+
 <li>Taxii Loader</li>
-  
 <li>Bulk load from HDFS via MapReduce</li>
-  
 <li>Flat File ingestion</li>
 </ul>
 <div class="section">
 <h3><a name="Taxii_Loader"></a>Taxii Loader</h3>
-<p>The shell script <tt>$METRON_HOME/bin/threatintel_taxii_load.sh</tt> can be used to poll a Taxii server for STIX documents and ingest them into HBase.<br />It is quite common for this Taxii server to be an aggregation server such as Soltra Edge.</p>
-<p>In addition to the Enrichment and Extractor configs described above, this loader requires a configuration file describing the connection information to the Taxii server. An illustrative example of such a configuration file is:</p>
+<p>The shell script <tt>$METRON_HOME/bin/threatintel_taxii_load.sh</tt> can be used to poll a Taxii server for STIX documents and ingest them into HBase.<br />
+It is quite common for this Taxii server to be an aggregation server such as Soltra Edge.</p>
+<p>In addition to the Enrichment and Extractor configs described above, this loader requires a configuration file describing the connection information to the Taxii server.  An illustrative example of such a configuration file is:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
    &quot;endpoint&quot; : &quot;http://localhost:8282/taxii-discovery-service&quot;
   ,&quot;type&quot; : &quot;DISCOVER&quot;
   ,&quot;collection&quot; : &quot;guest.Abuse_ch&quot;
@@ -707,363 +455,379 @@
   ,&quot;allowedIndicatorTypes&quot; : [ &quot;domainname:FQDN&quot;, &quot;address:IPV_4_ADDR&quot; ]
 }
 </pre></div></div>
-<p>As you can see, we are specifying the following information:</p>
 
+<p>As you can see, we are specifying the following information:</p>
 <ul>
-  
+
 <li>endpoint : The URL of the endpoint</li>
-  
 <li>type : <tt>POLL</tt> or <tt>DISCOVER</tt> depending on the endpoint.</li>
-  
 <li>collection : The Taxii collection to ingest</li>
-  
 <li>table : The HBase table to import into</li>
-  
 <li>columnFamily : The column family to import into</li>
-  
 <li>allowedIndicatorTypes : an array of acceptable threat intel types (see the &#x201c;Enrichment Type Name&#x201d; column of the Stix table above for the possibilities).</li>
 </ul>
 <p>The parameters for the utility are as follows:</p>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Short Code </th>
-      
-<th>Long Code </th>
-      
-<th>Is Required? </th>
-      
-<th>Description </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Short Code </th>
+<th> Long Code                 </th>
+<th> Is Required? </th>
+<th> Description                                                                                                                                        </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>-h </td>
-      
-<td> </td>
-      
-<td>No </td>
-      
-<td>Generate the help screen/set of options </td>
-    </tr>
-    
+<td> -h         </td>
+<td>                           </td>
+<td> No           </td>
+<td> Generate the help screen/set of options                                                                                                            </td></tr>
 <tr class="a">
-      
-<td>-e </td>
-      
-<td>&#x2013;extractor_config </td>
-      
-<td>Yes </td>
-      
-<td>JSON Document describing the extractor for this input data source </td>
-    </tr>
-    
+<td> -e         </td>
+<td> &#x2013;extractor_config        </td>
+<td> Yes          </td>
+<td> JSON Document describing the extractor for this input data source                                                                                  </td></tr>
 <tr class="b">
-      
-<td>-c </td>
-      
-<td>&#x2013;taxii_connection_config </td>
-      
-<td>Yes </td>
-      
-<td>The JSON config file to configure the connection </td>
-    </tr>
-    
+<td> -c         </td>
+<td> &#x2013;taxii_connection_config </td>
+<td> Yes          </td>
+<td> The JSON config file to configure the connection                                                                                                   </td></tr>
 <tr class="a">
-      
-<td>-p </td>
-      
-<td>&#x2013;time_between_polls </td>
-      
-<td>No </td>
-      
-<td>The time between polling the Taxii server in milliseconds. (default: 1 hour) </td>
-    </tr>
-    
+<td> -p         </td>
+<td> &#x2013;time_between_polls      </td>
+<td> No           </td>
+<td> The time between polling the Taxii server in milliseconds. (default: 1 hour)                                                                       </td></tr>
 <tr class="b">
-      
-<td>-b </td>
-      
-<td>&#x2013;begin_time </td>
-      
-<td>No </td>
-      
-<td>Start time to poll the Taxii server (all data from that point will be gathered in the first pull). The format for the date is yyyy-MM-dd HH:mm:ss </td>
-    </tr>
-    
+<td> -b         </td>
+<td> &#x2013;begin_time              </td>
+<td> No           </td>
+<td> Start time to poll the Taxii server (all data from that point will be gathered in the first pull).  The format for the date is yyyy-MM-dd HH:mm:ss </td></tr>
 <tr class="a">
-      
-<td>-l </td>
-      
-<td>&#x2013;log4j </td>
-      
-<td>No </td>
-      
-<td>The Log4j Properties to load </td>
-    </tr>
-    
+<td> -l         </td>
+<td> &#x2013;log4j                   </td>
+<td> No           </td>
+<td> The Log4j Properties to load                                                                                                                       </td></tr>
 <tr class="b">
-      
-<td>-n </td>
-      
-<td>&#x2013;enrichment_config </td>
-      
-<td>No </td>
-      
-<td>The JSON document describing the enrichments to configure. Unlike other loaders, this is run first if specified. </td>
-    </tr>
-  </tbody>
+<td> -n         </td>
+<td> &#x2013;enrichment_config       </td>
+<td> No           </td>
+<td> The JSON document describing the enrichments to configure.  Unlike other loaders, this is run first if specified.                                  </td></tr>
+</tbody>
 </table></div>
 <div class="section">
 <h3><a name="Flatfile_Loader"></a>Flatfile Loader</h3>
-<p>The shell script <tt>$METRON_HOME/bin/flatfile_loader.sh</tt> will read data from local disk, HDFS or URLs and load the enrichment or threat intel data into an HBase table.<br />Note: This utility works for enrichment as well as threat intel due to the underlying infrastructure being the same.</p>
+<p>The shell script <tt>$METRON_HOME/bin/flatfile_loader.sh</tt> will read data from local disk, HDFS or URLs and load the enrichment or threat intel data into an HBase table.<br />
+Note: This utility works for enrichment as well as threat intel due to the underlying infrastructure being the same.</p>
 <p>One special thing to note here is that there is a special configuration parameter to the Extractor config that is only considered during this loader:</p>
-
 <ul>
-  
-<li>inputFormat : This specifies how to consider the data. The two implementations are <tt>BY_LINE</tt> and <tt>WHOLE_FILE</tt>.</li>
+
+<li>inputFormat : This specifies how to consider the data.  The two implementations are <tt>BY_LINE</tt> and <tt>WHOLE_FILE</tt>.</li>
 </ul>
 <p>The default is <tt>BY_LINE</tt>, which makes sense for a list of CSVs where each line indicates a unit of information which can be imported. However, if you are importing a set of STIX documents, then you want each document to be considered as input to the Extractor.</p>
 <p>The parameters for the utility are as follows:</p>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Short Code </th>
-      
-<th>Long Code </th>
-      
-<th>Is Required? </th>
-      
-<th>Description </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Short Code </th>
+<th> Long Code           </th>
+<th> Is Required? </th>
+<th> Description                                                                                                                                                                         </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>-h </td>
-      
-<td> </td>
-      
-<td>No </td>
-      
-<td>Generate the help screen/set of options </td>
-    </tr>
-    
+<td> -h         </td>
+<td>                     </td>
+<td> No           </td>
+<td> Generate the help screen/set of options                                                                                                                                             </td></tr>
 <tr class="a">
-      
-<td>-q </td>
-      
-<td>&#x2013;quiet </td>
-      
-<td>No </td>
-      
-<td>Do not update progress </td>
-    </tr>
-    
+<td> -q         </td>
+<td> &#x2013;quiet             </td>
+<td> No           </td>
+<td> Do not update progress                                                                                                                                                              </td></tr>
 <tr class="b">
-      
-<td>-e </td>
-      
-<td>&#x2013;extractor_config </td>
-      
-<td>Yes </td>
-      
-<td>JSON Document describing the extractor for this input data source </td>
-    </tr>
-    
+<td> -e         </td>
+<td> &#x2013;extractor_config  </td>
+<td> Yes          </td>
+<td> JSON Document describing the extractor for this input data source                                                                                                                   </td></tr>
 <tr class="a">
-      
-<td>-m </td>
-      
-<td>&#x2013;import_mode </td>
-      
-<td>No </td>
-      
-<td>The Import mode to use: LOCAL, MR. Default: LOCAL </td>
-    </tr>
-    
+<td> -m         </td>
+<td> &#x2013;import_mode       </td>
+<td> No           </td>
+<td> The Import mode to use: LOCAL, MR.  Default: LOCAL                                                                                                                                  </td></tr>
 <tr class="b">
-      
-<td>-t </td>
-      
-<td>&#x2013;hbase_table </td>
-      
-<td>Yes </td>
-      
-<td>The HBase table to import into </td>
-    </tr>
-    
+<td> -t         </td>
+<td> &#x2013;hbase_table       </td>
+<td> Yes          </td>
+<td> The HBase table to import into                                                                                                                                                      </td></tr>
 <tr class="a">
-      
-<td>-c </td>
-      
-<td>&#x2013;hbase_cf </td>
-      
-<td>Yes </td>
-      
-<td>The HBase table column family to import into </td>
-    </tr>
-    
+<td> -c         </td>
+<td> &#x2013;hbase_cf          </td>
+<td> Yes          </td>
+<td> The HBase table column family to import into                                                                                                                                        </td></tr>
 <tr class="b">
-      
-<td>-i </td>
-      
-<td>&#x2013;input </td>
-      
-<td>Yes </td>
-      
-<td>The input data location on local disk. If this is a file, then that file will be loaded. If this is a directory, then the files will be loaded recursively under that directory. </td>
-    </tr>
-    
+<td> -i         </td>
+<td> &#x2013;input             </td>
+<td> Yes          </td>
+<td> The input data location on local disk.  If this is a file, then that file will be loaded.  If this is a directory, then the files will be loaded recursively under that directory.  </td></tr>
 <tr class="a">
-      
-<td>-l </td>
-      
-<td>&#x2013;log4j </td>
-      
-<td>No </td>
-      
-<td>The log4j properties file to load </td>
-    </tr>
-    
+<td> -l         </td>
+<td> &#x2013;log4j             </td>
+<td> No           </td>
+<td> The log4j properties file to load                                                                                                                                                   </td></tr>
 <tr class="b">
-      
-<td>-n </td>
-      
-<td>&#x2013;enrichment_config </td>
-      
-<td>No </td>
-      
-<td>The JSON document describing the enrichments to configure. Unlike other loaders, this is run first if specified. </td>
-    </tr>
-    
+<td> -n         </td>
+<td> &#x2013;enrichment_config </td>
+<td> No           </td>
+<td> The JSON document describing the enrichments to configure.  Unlike other loaders, this is run first if specified.                                                                   </td></tr>
 <tr class="a">
-      
-<td>-p </td>
-      
-<td>&#x2013;threads </td>
-      
-<td>No </td>
-      
-<td>The number of threads to use when extracting data. The default is the number of cores. </td>
-    </tr>
-    
+<td> -p         </td>
+<td> &#x2013;threads           </td>
+<td> No           </td>
+<td> The number of threads to use when extracting data.  The default is the number of cores.                                                                                             </td></tr>
 <tr class="b">
-      
-<td>-b </td>
-      
-<td>&#x2013;batchSize </td>
-      
-<td>No </td>
-      
-<td>The batch size to use for HBase puts </td>
-    </tr>
-  </tbody>
+<td> -b         </td>
+<td> &#x2013;batchSize         </td>
+<td> No           </td>
+<td> The batch size to use for HBase puts                                                                                                                                                </td></tr>
+</tbody>
 </table></div>
 <div class="section">
 <h3><a name="GeoLite2_Loader"></a>GeoLite2 Loader</h3>
 <p>The shell script <tt>$METRON_HOME/bin/geo_enrichment_load.sh</tt> will retrieve MaxMind GeoLite2 data and load data into HDFS, and update the configuration.</p>
-<p>THIS SCRIPT WILL NOT UPDATE AMBARI&#x2019;S GLOBAL.JSON, JUST THE ZK CONFIGS. CHANGES WILL GO INTO EFFECT, BUT WILL NOT PERSIST PAST AN AMBARI RESTART UNTIL UPDATED THERE.</p>
+<p>THIS SCRIPT WILL NOT UPDATE AMBARI&#x2019;S GLOBAL.JSON, JUST THE ZK CONFIGS.  CHANGES WILL GO INTO EFFECT, BUT WILL NOT PERSIST PAST AN AMBARI RESTART UNTIL UPDATED THERE.</p>
 <p>The parameters for the utility are as follows:</p>
+<table border="0" class="table table-striped">
+<thead>
 
+<tr class="a">
+<th> Short Code </th>
+<th> Long Code           </th>
+<th> Is Required? </th>
+<th> Description                                                                                      </th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td> -h         </td>
+<td>                     </td>
+<td> No           </td>
+<td> Generate the help screen/set of options                                                          </td></tr>
+<tr class="a">
+<td> -g         </td>
+<td> &#x2013;geo_url           </td>
+<td> No           </td>
+<td> GeoIP URL - defaults to <a class="externalLink" href="http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz">http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz</a> </td></tr>
+<tr class="b">
+<td> -r         </td>
+<td> &#x2013;remote_dir        </td>
+<td> No           </td>
+<td> HDFS directory to land formatted GeoIP file - defaults to /apps/metron/geo/&lt;epoch millis&gt;/     </td></tr>
+<tr class="a">
+<td> -t         </td>
+<td> &#x2013;tmp_dir           </td>
+<td> No           </td>
+<td> Directory for landing the temporary GeoIP data - defaults to /tmp                                </td></tr>
+<tr class="b">
+<td> -z         </td>
+<td> &#x2013;zk_quorum         </td>
+<td> Yes          </td>
+<td> Zookeeper Quorum URL (zk1:port,zk2:port,&#x2026;)                                                     </td></tr>
+</tbody>
+</table></div>
+<div class="section">
+<h3><a name="Flatfile_Summarizer"></a>Flatfile Summarizer</h3>
+<p>The shell script <tt>$METRON_HOME/bin/flatfile_summarizer.sh</tt> will read data from local disk, HDFS or URLs and generate a summary object. The object will be serialized and written to disk, either HDFS or local disk depending on the output mode specified.</p>
+<p>It should be noted that this utility uses the same extractor config as the <tt>flatfile_loader.sh</tt>, but as the output target is not a key value store (but rather a summary object), it is not necessary to specify certain configs:</p>
+<ul>
+
+<li><tt>indicator</tt>, <tt>indicator_filter</tt> and <tt>indicator_transform</tt> are not required, but will be executed if present. As in the loader, there will be an indicator field available if you so specify it (by using <tt>indicator</tt> in the config).</li>
+<li><tt>type</tt> is neither required nor used</li>
+</ul>
+<p>Indeed, some new configs are expected:</p>
+<ul>
+
+<li><tt>state_init</tt> : Executed once to initialize the state object (the object written out).</li>
+<li><tt>state_update</tt>: Called once per message.  The fields available are the fields for the row as well as
+<ul>
+
+<li><tt>indicator</tt> - the indicator value if you&#x2019;ve specified it in the config</li>
+<li><tt>state</tt> - the current state.  Useful for adding to the state (e.g. <tt>BLOOM_ADD(state, val)</tt> where <tt>val</tt> is the name of a field).</li>
+</ul>
+</li>
+<li><tt>state_merge</tt> : If you are running this multi-threaded and your objects can be merged, this is the statement that will merge the state objects created per thread.  There is a special field available to this config:
+<ul>
+
+<li><tt>states</tt> - a list of the state objects</li>
+</ul>
+</li>
+</ul>
+<p>One special thing to note here is that there is a special configuration parameter to the Extractor config that is only considered during this loader:</p>
+<ul>
+
+<li>inputFormat : This specifies how to consider the data.  The two implementations are <tt>BY_LINE</tt> and <tt>WHOLE_FILE</tt>.</li>
+</ul>
+<p>The default is <tt>BY_LINE</tt>, which makes sense for a list of CSVs where each line indicates a unit of information which can be imported. However, if you are importing a set of STIX documents, then you want each document to be considered as input to the Extractor.</p>
+<div class="section">
+<h4><a name="Example"></a>Example</h4>
+<p>Consider the possibility that you want to generate a bloom filter with all of the domains in a CSV structured similarly to the Alexa top 1M domains, so the columns are:</p>
+<ul>
+
+<li>rank</li>
+<li>domain name</li>
+</ul>
+<p>You want to generate a bloom filter with just the domains, not considering the TLD. You would execute the following to:</p>
+<ul>
+
+<li>read data from <tt>./top-1m.csv</tt></li>
+<li>write data to <tt>./filter.ser</tt></li>
+<li>use 5 threads</li>
+</ul>
+
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/flatfile_summarizer.sh -i ./top-1m.csv -o ./filter.ser -e ./extractor.json -p 5 -b 128
+</pre></div></div>
+
+<p>To configure this, <tt>extractor.json</tt> would look like:</p>
+
+<div>
+<div>
+<pre class="source">{
+  &quot;config&quot; : {
+    &quot;columns&quot; : {
+      &quot;rank&quot; : 0,
+      &quot;domain&quot; : 1
+    },
+    &quot;value_transform&quot; : {
+      &quot;domain&quot; : &quot;DOMAIN_REMOVE_TLD(domain)&quot;
+    },
+    &quot;value_filter&quot; : &quot;LENGTH(domain) &gt; 0&quot;,
+    &quot;state_init&quot; : &quot;BLOOM_INIT()&quot;,
+    &quot;state_update&quot; : {
+      &quot;state&quot; : &quot;BLOOM_ADD(state, domain)&quot;
+    },
+    &quot;state_merge&quot; : &quot;BLOOM_MERGE(states)&quot;,
+    &quot;separator&quot; : &quot;,&quot;
+  },
+  &quot;extractor&quot; : &quot;CSV&quot;
+}
+</pre></div></div>
+</div>
+<div class="section">
+<h4><a name="Parameters"></a>Parameters</h4>
+<p>The parameters for the utility are as follows:</p>
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Short Code </th>
-      
-<th>Long Code </th>
-      
-<th>Is Required? </th>
-      
-<th>Description </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Short Code </th>
+<th> Long Code           </th>
+<th> Is Required? </th>
+<th> Description                                                                                                                                                                         </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>-h </td>
-      
-<td> </td>
-      
-<td>No </td>
-      
-<td>Generate the help screen/set of options </td>
-    </tr>
-    
+<td> -h         </td>
+<td>                     </td>
+<td> No           </td>
+<td> Generate the help screen/set of options                                                                                                                                             </td></tr>
 <tr class="a">
-      
-<td>-g </td>
-      
-<td>&#x2013;geo_url </td>
-      
-<td>No </td>
-      
-<td>GeoIP URL - defaults to <a class="externalLink" href="http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz">http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz</a> </td>
-    </tr>
-    
+<td> -q         </td>
+<td> &#x2013;quiet             </td>
+<td> No           </td>
+<td> Do not update progress                                                                                                                                                              </td></tr>
 <tr class="b">
-      
-<td>-r </td>
-      
-<td>&#x2013;remote_dir </td>
-      
-<td>No </td>
-      
-<td>HDFS directory to land formatted GeoIP file - defaults to /apps/metron/geo/&lt;epoch millis&gt;/ </td>
-    </tr>
-    
+<td> -e         </td>
+<td> &#x2013;extractor_config  </td>
+<td> Yes          </td>
+<td> JSON Document describing the extractor for this input data source                                                                                                                   </td></tr>
 <tr class="a">
-      
-<td>-t </td>
-      
-<td>&#x2013;tmp_dir </td>
-      
-<td>No </td>
-      
-<td>Directory for landing the temporary GeoIP data - defaults to /tmp </td>
-    </tr>
-    
+<td> -m         </td>
+<td> &#x2013;import_mode       </td>
+<td> No           </td>
+<td> The Import mode to use: LOCAL, MR.  Default: LOCAL                                                                                                                                  </td></tr>
 <tr class="b">
-      
-<td>-z </td>
-      
-<td>&#x2013;zk_quorum </td>
-      
-<td>Yes </td>
-      
-<td>Zookeeper Quorum URL (zk1:port,zk2:port,&#x2026;) </td>
-    </tr>
-  </tbody>
-</table></div></div>
-                  </div>
-            </div>
-          </div>
+<td> -om        </td>
+<td> &#x2013;output_mode       </td>
+<td> No           </td>
+<td> The Output mode to use: LOCAL, HDFS.  Default: LOCAL                                                                                                                                  </td></tr>
+<tr class="a">
+<td> -i         </td>
+<td> &#x2013;input             </td>
+<td> Yes          </td>
+<td> The input data location on local disk.  If this is a file, then that file will be loaded.  If this is a directory, then the files will be loaded recursively under that directory.  </td></tr>
+<tr class="b">
+<td> -o         </td>
+<td> &#x2013;output            </td>
+<td> Yes          </td>
+<td> The output data location.    </td></tr>
+<tr class="a">
+<td> -l         </td>
+<td> &#x2013;log4j             </td>
+<td> No           </td>
+<td> The log4j properties file to load                                                                                                                                                   </td></tr>
+<tr class="b">
+<td> -p         </td>
+<td> &#x2013;threads           </td>
+<td> No           </td>
+<td> The number of threads to use when extracting data.  The default is the number of cores.                                                                                             </td></tr>
+<tr class="a">
+<td> -b         </td>
+<td> &#x2013;batchSize         </td>
+<td> No           </td>
+<td> The batch size to use for HBase puts                                                                                                                                                </td></tr>
+</tbody>
+</table></div></div></div>
+<div class="section">
+<h2><a name="Pruning_Data_from_Elasticsearch"></a>Pruning Data from Elasticsearch</h2>
+<p><b>Note</b> - As of the Metron upgrade from Elasticsearch 2.3.3 to 5.6.2, the included Data Pruner is no longer supported. It is replaced in favor of the Curator utility provided by Elasticsearch. The current Curator version is 5.4 as of this version of Metron and does not match exactly with ES and Kibana.</p>
+<p>Elasticsearch provides tooling to prune index data through <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/client/curator/5.4/index.html">Curator</a>.</p>
+<p>Here is a sample invocation that you can configure through Cron to prune indexes based on timestamp in the index name.</p>
+
+<div>
+<div>
+<pre class="source">/opt/elasticsearch-curator/curator_cli --host localhost delete_indices --filter_list '
+    {
+      &quot;filtertype&quot;: &quot;age&quot;,
+      &quot;source&quot;: &quot;name&quot;,
+      &quot;timestring&quot;: &quot;%Y.%m.%d&quot;,
+      &quot;unit&quot;: &quot;days&quot;,
+      &quot;unit_count&quot;: 10,
+      &quot;direction&quot;: &quot;older&#x201d;
+    }'
+</pre></div></div>
 
-    <hr/>
+<p>From the ES documentation:</p>
+<blockquote>
+
+<p>Using name as the source tells Curator to look for a timestring within the index or snapshot name, and convert that into an epoch timestamp (epoch implies UTC).</p>
+</blockquote>
+<p>You can also provide multiple filters as an array of JSON objects to filter_list if you want finer-grained control over the indexes that will be pruned. There is an implicit logical AND when chaining multiple filters.</p>
+
+<div>
+<div>
+<pre class="source">--filter_list '[{&quot;filtertype&quot;:&quot;age&quot;,&quot;source&quot;:&quot;creation_date&quot;,&quot;direction&quot;:&quot;older&quot;,&quot;unit&quot;:&quot;days&quot;,&quot;unit_count&quot;:13},{&quot;filtertype&quot;:&quot;pattern&quot;,&quot;kind&quot;:&quot;prefix&quot;,&quot;value&quot;:&quot;logstash&quot;}]'
+</pre></div></div>
 
+<div class="section">
+<h3><a name="Reference"></a>Reference</h3>
+<ul>
+
+<li><a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/client/curator/5.4/index.html">https://www.elastic.co/guide/en/elasticsearch/client/curator/5.4/index.html</a></li>
+<li><a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/client/curator/5.4/filtertype_age.html">https://www.elastic.co/guide/en/elasticsearch/client/curator/5.4/filtertype_age.html</a></li>
+<li><a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/client/curator/5.4/singleton-cli.html">https://www.elastic.co/guide/en/elasticsearch/client/curator/5.4/singleton-cli.html</a></li>
+</ul></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[23/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html b/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html
deleted file mode 100644
index f72df78..0000000
--- a/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html
+++ /dev/null
@@ -1,406 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Quick Development Platform</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../../css/site.css" />
-    <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Quick Development Platform</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                                                                                
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-down"></i>
-        Vagrant</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/codelab-platform/index.html" title="Codelab-platform">
-          <i class="none"></i>
-        Codelab-platform</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/fastcapa-test-platform/index.html" title="Fastcapa-test-platform">
-          <i class="none"></i>
-        Fastcapa-test-platform</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/full-dev-platform/index.html" title="Full-dev-platform">
-          <i class="none"></i>
-        Full-dev-platform</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Quick-dev-platform</a>
-          </li>
-              </ul>
-        </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                                            
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Quick Development Platform</h1>
-<p>This project fully automates the provisioning and deployment of Apache Metron and all necessary prerequisites on a single, virtualized host running on Virtualbox. </p>
-<p>This image is designed for quick deployment of a single node Metron cluster running on Virtualbox. This platform is ideal for use by Metron developers. It uses a base image that has been pre-loaded with Ambari and HDP.</p>
-<p>Metron is composed of many components and installing all of these on a single host, especially a virtualized one, will greatly stress the resources of the host. The host will require at least 8 GB of RAM and a fair amount of patience. It is highly recommended that you shut down all unnecessary services. To that end the vagrant file configuration defaults to disabling solr and yaf.</p>
-<div class="section">
-<h2><a name="Getting_Started"></a>Getting Started</h2>
-<div class="section">
-<h3><a name="Prerequisites"></a>Prerequisites</h3>
-<p>As with the Full Development Platform (<tt>metron-deployment/vagrant/full-dev-platform</tt>), the computer used to deploy Apache Metron will need the following components installed.</p>
-
-<ul>
-  
-<li><a class="externalLink" href="https://github.com/ansible/ansible">Ansible</a> (2.0.0.2 or 2.2.2.0)</li>
-  
-<li><a class="externalLink" href="https://www.docker.com/community-edition">Docker</a></li>
-  
-<li><a class="externalLink" href="https://www.vagrantup.com">Vagrant</a> 1.8.1</li>
-  
-<li><a class="externalLink" href="https://github.com/devopsgroup-io/vagrant-hostmanager">Vagrant Hostmanager Plugin</a> <tt>vagrant plugin install vagrant-hostmanager</tt></li>
-  
-<li><a class="externalLink" href="https://virtualbox.org">Virtualbox</a> 5.0.16</li>
-  
-<li>Python 2.7.11</li>
-  
-<li>Maven 3.3.9</li>
-</ul>
-<div class="section">
-<h4><a name="macOS"></a>macOS</h4>
-<p>Any platform that supports these tools is suitable, but the following instructions cover installation on macOS. The easiest means of installing these tools on a Mac is to use the excellent <a class="externalLink" href="http://brew.sh/">Homebrew</a> project.</p>
-
-<ol style="list-style-type: decimal">
-  
-<li>
-<p>Install Homebrew by following the instructions at <a class="externalLink" href="http://brew.sh/">Homebrew</a>.</p></li>
-  
-<li>
-<p>Run the following command in a terminal to install all of the required tools.</p>
-  
-<div class="source">
-<div class="source">
-<pre> brew cask install vagrant virtualbox java docker
- brew install maven git
-</pre></div></div></li>
-  
-<li>
-<p>Install Ansible by following the instructions <a class="externalLink" href="http://docs.ansible.com/ansible/intro_installation.html#latest-releases-via-pip">here</a>.</p></li>
-</ol></div></div>
-<div class="section">
-<h3><a name="Deploy_Metron"></a>Deploy Metron</h3>
-
-<ol style="list-style-type: decimal">
-  
-<li>
-<p>Ensure that the Docker service is running.</p></li>
-  
-<li>
-<p>Deploy Metron</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron-deployment/vagrant/quick-dev-platform
-vagrant up
-</pre></div></div>
-<p>Should the process fail before completing the deployment, the following command will continue the deployment process without re-instantiating the host.</p>
-  
-<div class="source">
-<div class="source">
-<pre>vagrant provision
-</pre></div></div></li>
-</ol></div>
-<div class="section">
-<h3><a name="Explore_Metron"></a>Explore Metron</h3>
-<p>Navigate to the following resources to explore your newly minted Apache Metron environment.</p>
-
-<ul>
-  
-<li><a class="externalLink" href="http://node1:8080">Ambari</a></li>
-  
-<li><a class="externalLink" href="http://node1:5000">Metron</a></li>
-  
-<li><a class="externalLink" href="http://node1:2812">Services</a></li>
-</ul></div>
-<div class="section">
-<h3><a name="Working_with_Metron"></a>Working with Metron</h3>
-<p>As you build out new capabilities for Metron, you will need to re-deploy the Storm topologies. To do so, first HALT the running Storm topologies and then run the following command.</p>
-
-<div class="source">
-<div class="source">
-<pre>./run_enrichment_role.sh
-</pre></div></div>
-<p>Connecting to the host through SSH is as simple as running the following command.</p>
-
-<div class="source">
-<div class="source">
-<pre>vagrant ssh
-</pre></div></div></div></div>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-interface/metron-alerts/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-interface/metron-alerts/index.html b/site/current-book/metron-interface/metron-alerts/index.html
index cf30176..9459bd1 100644
--- a/site/current-book/metron-interface/metron-alerts/index.html
+++ b/site/current-book/metron-interface/metron-alerts/index.html
@@ -1,210 +1,108 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-interface/metron-alerts/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; </title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class=""></li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active "></li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Alerts</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <ul>
-  
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<ul>
+
 <li><a href="#Caveats">Caveats</a></li>
-  
 <li><a href="#Prerequisites">Prerequisites</a></li>
-  
 <li><a href="#Development_Setup">Development Setup</a></li>
-  
 <li><a href="#E2E_Tests">E2E Tests</a></li>
-  
 <li><a href="#Mpack_Integration">Mpack Integration</a></li>
-  
 <li><a href="#Installing_on_an_existing_Cluster">Installing on an existing Cluster</a></li>
 </ul>
 <div class="section">
 <h2><a name="Caveats"></a>Caveats</h2>
 <div class="section">
 <h3><a name="Local_Storage"></a>Local Storage</h3>
-<p>UI uses local storage to save all the data. A middleware needs to be designed and developed for persisting the data</p></div>
+<p>UI uses local storage to save all the data.  A middleware needs to be designed and developed for persisting the data</p></div>
 <div class="section">
 <h3><a name="Search_for_Alert_GUIDs"></a>Search for Alert GUIDs</h3>
 <p>Alert GUIDs must be double-quoted when being searched on to ensure correctness of results, e.g. guid:&#x201c;id1&#x201d;.</p></div>
@@ -213,170 +111,172 @@
 <p>Users cannot search for the contents of the comment&#x2019;s in the Alerts-UI</p></div>
 <div class="section">
 <h3><a name="Meta_alerts"></a>Meta alerts</h3>
-<p>Grouping/faceting requests and other aggregations do not return meta alerts. This is because it&#x2019;s not clear what the intended results should be when there are multiple matching items.</p>
+<p>Grouping/faceting requests and other aggregations do not return meta alerts.  This is because it&#x2019;s not clear what the intended results should be when there are multiple matching items.</p>
 <p>Sorting has a similar caveat, in that if we are matching on multiple alerts, there is no well defined sort.</p>
 <p>Alerts that are contained in a a meta alert are generally excluded from search results, because a user has already grouped them in a meaningful way.</p></div></div>
 <div class="section">
 <h2><a name="Prerequisites"></a>Prerequisites</h2>
-
 <ul>
-  
+
 <li>The Metron REST application should be up and running and Elasticsearch should have some alerts populated by Metron topologies</li>
-  
 <li>The Management UI should be installed (which includes <a class="externalLink" href="https://expressjs.com/">Express</a>)</li>
-  
 <li>The alerts can be populated using Full Dev or any other setup</li>
-  
 <li>UI is developed using angular4 and uses angular-cli</li>
-  
 <li>node.JS &gt;= 7.8.0</li>
 </ul></div>
 <div class="section">
 <h2><a name="Installation"></a>Installation</h2>
 <div class="section">
 <h3><a name="From_Source"></a>From Source</h3>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Package the application with Maven:</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron-interface/metron-alerts
+
+<div>
+<div>
+<pre class="source">cd metron-interface/metron-alerts
 mvn clean package
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Untar the archive in the $METRON_HOME directory. The directory structure will look like:</p>
-  
-<div class="source">
-<div class="source">
-<pre>bin
+
+<p>Untar the archive in the $METRON_HOME directory.  The directory structure will look like:</p>
+
+<div>
+<div>
+<pre class="source">bin
   metron-alerts-ui
 web
   expressjs
     alerts-server.js
   alerts-ui
     web assets (html, css, js, ...)
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Copy the <tt>$METRON_HOME/bin/metron-alerts-ui</tt> script to <tt>/etc/init.d/metron-alerts-ui</tt></p></li>
-  
+
+<p>Copy the <tt>$METRON_HOME/bin/metron-alerts-ui</tt> script to <tt>/etc/init.d/metron-alerts-ui</tt></p>
+</li>
 <li>
-<p><a class="externalLink" href="https://expressjs.com/">Express</a> is installed at <tt>$METRON_HOME/web/expressjs/</tt> as part of the Management UI installation process. The Management UI should be installed first on the same host as the Alerts UI.</p></li>
+
+<p><a class="externalLink" href="https://expressjs.com/">Express</a> is installed at <tt>$METRON_HOME/web/expressjs/</tt> as part of the Management UI installation process.  The Management UI should be installed first on the same host as the Alerts UI.</p>
+</li>
 </ol></div>
 <div class="section">
 <h3><a name="From_Package_Manager"></a>From Package Manager</h3>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Deploy the RPM at <tt>/metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-alerts-$METRON_VERSION-*.noarch.rpm</tt></p></li>
-  
+
+<p>Deploy the RPM at <tt>/metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-alerts-$METRON_VERSION-*.noarch.rpm</tt></p>
+</li>
 <li>
+
 <p>Install the RPM with:</p>
-  
-<div class="source">
-<div class="source">
-<pre>rpm -ih metron-alerts-$METRON_VERSION-*.noarch.rpm
-</pre></div></div></li>
+
+<div>
+<div>
+<pre class="source">rpm -ih metron-alerts-$METRON_VERSION-*.noarch.rpm
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h3><a name="From_Ambari_MPack"></a>From Ambari MPack</h3>
-<p>The Alerts UI is included in the Metron Ambari MPack. It can be accessed through the Quick Links in the Metron service.</p></div></div>
+<p>The Alerts UI is included in the Metron Ambari MPack.  It can be accessed through the Quick Links in the Metron service.</p></div></div>
 <div class="section">
 <h2><a name="Configuration"></a>Configuration</h2>
-<p>The Alerts UI is configured in the <tt>$METRON_HOME/config/alerts_ui.yml</tt> file. Create this file and set the values to match your environment:</p>
+<p>The Alerts UI is configured in the <tt>$METRON_HOME/config/alerts_ui.yml</tt> file.  Create this file and set the values to match your environment:</p>
 
-<div class="source">
-<div class="source">
-<pre>port: port the alerts UI will run on
+<div>
+<div>
+<pre class="source">port: port the alerts UI will run on
 
 rest:
   host: REST application host
   port: REST applciation port
-</pre></div></div></div>
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Global_Configuration_Properties"></a>Global Configuration Properties</h2>
+<div class="section">
+<h3><a name="source.type.field"></a><tt>source.type.field</tt></h3>
+<p>The source type format used. Defaults to <tt>source:type</tt>.</p></div></div>
 <div class="section">
 <h2><a name="Usage"></a>Usage</h2>
 <p>After configuration is complete, the Management UI can be managed as a service:</p>
 
-<div class="source">
-<div class="source">
-<pre>service metron-alerts-ui start
+<div>
+<div>
+<pre class="source">service metron-alerts-ui start
 </pre></div></div>
-<p>The application will be available at <a class="externalLink" href="http://host:4201">http://host:4201</a> assuming the port is set to <tt>4201</tt>. Logs can be found at <tt>/var/log/metron/metron-alerts-ui.log</tt>.</p></div>
+
+<p>The application will be available at <a class="externalLink" href="http://host:4201">http://host:4201</a> assuming the port is set to <tt>4201</tt>.  Logs can be found at <tt>/var/log/metron/metron-alerts-ui.log</tt>.</p></div>
 <div class="section">
 <h2><a name="Development_Setup"></a>Development Setup</h2>
-
 <ol style="list-style-type: decimal">
-  
-<li>
-<p>Install all the dependent node_modules using the following command</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron/metron-interface/metron-alerts
+
+<li>Install all the dependent node_modules using the following command
+
+<div>
+<div>
+<pre class="source">cd metron/metron-interface/metron-alerts
 npm install
-</pre></div></div></li>
-  
-<li>
-<p>UI can be run by using the following command</p>
-  
-<div class="source">
-<div class="source">
-<pre>./scripts/start-dev.sh
-</pre></div></div></li>
-  
-<li>You can view the GUI @http://localhost:4201. The default credentials for login are admin/password</li>
+</pre></div></div>
+</li>
+<li>UI can be run by using the following command
+
+<div>
+<div>
+<pre class="source">./scripts/start-dev.sh
+</pre></div></div>
+</li>
+<li>You can view the GUI @<a class="externalLink" href="http://localhost:4201">http://localhost:4201</a>. The default credentials for login are admin/password</li>
 </ol>
 <p><b>NOTE</b>: <i>In the development mode ui by default connects to REST at <a class="externalLink" href="http://node1:8082">http://node1:8082</a> for fetching data. If you wish to change it you can change the REST url at metron/metron-interface/metron-alerts/proxy.conf.json</i></p></div>
 <div class="section">
 <h2><a name="E2E_Tests"></a>E2E Tests</h2>
 <p>An expressjs server is available for mocking the elastic search api.</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Run e2e webserver :</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron/metron-interface/metron-alerts
+
+<div>
+<div>
+<pre class="source">cd metron/metron-interface/metron-alerts
 sh ./scripts/start-server-for-e2e.sh
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>run e2e test using the following command</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron/metron-interface/metron-alerts
+
+<div>
+<div>
+<pre class="source">cd metron/metron-interface/metron-alerts
 npm run e2e
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>E2E tests uses data from full-dev wherever applicable. The tests assume rest-api&#x2019;s are available @http://node1:8082</p></li>
+
+<p>E2E tests uses data from full-dev wherever applicable. The tests assume rest-api&#x2019;s are available @<a class="externalLink" href="http://node1:8082">http://node1:8082</a></p>
+</li>
 </ol>
 <p><b>NOTE</b>: <i>e2e tests covers all the general workflows and we will extend them as we need</i></p></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-interface/metron-config/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-interface/metron-config/index.html b/site/current-book/metron-interface/metron-config/index.html
index e231b51..0c4c073 100644
--- a/site/current-book/metron-interface/metron-config/index.html
+++ b/site/current-book/metron-interface/metron-config/index.html
@@ -1,225 +1,127 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-interface/metron-config/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Metron Management UI</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Metron Management UI</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron Management UI</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Config</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Metron Management UI</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron Management UI</h1>
 <p><a name="Metron_Management_UI"></a></p>
 <p>This module provides a user interface for management functions in Metron.</p>
 <div class="section">
 <h2><a name="Prerequisites"></a>Prerequisites</h2>
-
 <ul>
-  
+
 <li>A network accessible Metron REST application</li>
-  
 <li>nodejs v6.9+ (nodejs can be installed on Full Dev with <tt>curl --silent --location https://rpm.nodesource.com/setup_6.x | bash - &amp;&amp; yum install -y nodejs</tt>)</li>
 </ul></div>
 <div class="section">
 <h2><a name="Installation"></a>Installation</h2>
 <div class="section">
 <h3><a name="From_Source"></a>From Source</h3>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Package the application with Maven:</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron-interface/metron-config
+
+<div>
+<div>
+<pre class="source">cd metron-interface/metron-config
 mvn clean package
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Untar the archive in the $METRON_HOME directory. The directory structure will look like:</p>
-  
-<div class="source">
-<div class="source">
-<pre>bin
+
+<p>Untar the archive in the $METRON_HOME directory.  The directory structure will look like:</p>
+
+<div>
+<div>
+<pre class="source">bin
   metron-management-ui
 web
   expressjs
@@ -227,122 +129,127 @@ web
     server.js
   management-ui
     web assets (html, css, js, ...)
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Copy the <tt>$METRON_HOME/bin/metron-management-ui</tt> script to <tt>/etc/init.d/metron-management-ui</tt></p></li>
-  
+
+<p>Copy the <tt>$METRON_HOME/bin/metron-management-ui</tt> script to <tt>/etc/init.d/metron-management-ui</tt></p>
+</li>
 <li>
+
 <p>Install the <a class="externalLink" href="https://expressjs.com/">Express</a> web framework from the <tt>package.json</tt> file in <tt>$METRON_HOME/web/expressjs</tt>:</p>
-  
-<div class="source">
-<div class="source">
-<pre>npm --prefix $METRON_HOME/web/expressjs/ install
-</pre></div></div></li>
+
+<div>
+<div>
+<pre class="source">npm --prefix $METRON_HOME/web/expressjs/ install
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h3><a name="From_Package_Manager"></a>From Package Manager</h3>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Deploy the RPM at <tt>/metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-config-$METRON_VERSION-*.noarch.rpm</tt></p></li>
-  
+
+<p>Deploy the RPM at <tt>/metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-config-$METRON_VERSION-*.noarch.rpm</tt></p>
+</li>
 <li>
+
 <p>Install the RPM with:</p>
-  
-<div class="source">
-<div class="source">
-<pre>rpm -ih metron-config-$METRON_VERSION-*.noarch.rpm
-</pre></div></div></li>
+
+<div>
+<div>
+<pre class="source">rpm -ih metron-config-$METRON_VERSION-*.noarch.rpm
+</pre></div></div>
+</li>
 </ol></div></div>
 <div class="section">
 <h2><a name="Configuration"></a>Configuration</h2>
-<p>The Managment UI is configured in the <tt>$METRON_HOME/config/management_ui.yml</tt> file. Create this file and set the values to match your environment:</p>
+<p>The Managment UI is configured in the <tt>$METRON_HOME/config/management_ui.yml</tt> file.  Create this file and set the values to match your environment:</p>
 
-<div class="source">
-<div class="source">
-<pre>port: port the managment UI will run on
+<div>
+<div>
+<pre class="source">port: port the managment UI will run on
 
 rest:
   host: REST application host
   port: REST applciation port
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Usage"></a>Usage</h2>
 <p>After configuration is complete, the Management UI can be managed as a service:</p>
 
-<div class="source">
-<div class="source">
-<pre>service metron-management-ui start
+<div>
+<div>
+<pre class="source">service metron-management-ui start
 </pre></div></div>
-<p>The application will be available at <a class="externalLink" href="http://host:4200">http://host:4200</a> assuming the port is set to <tt>4200</tt>. Logs can be found at <tt>/var/log/metron/metron-management-ui.log</tt>.</p></div>
+
+<p>The application will be available at <a class="externalLink" href="http://host:4200">http://host:4200</a> assuming the port is set to <tt>4200</tt>.  Logs can be found at <tt>/var/log/metron/metron-management-ui.log</tt>.</p></div>
 <div class="section">
 <h2><a name="Development"></a>Development</h2>
-<p>The Management UI can also be started in development mode. This allows changes to web assets to be seen interactively.</p>
-
+<p>The Management UI can also be started in development mode.  This allows changes to web assets to be seen interactively.</p>
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Install the application with dev dependencies:</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron-interface/metron-config
+
+<div>
+<div>
+<pre class="source">cd metron-interface/metron-config
 npm install
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Start the application:</p>
-  
-<div class="source">
-<div class="source">
-<pre>./scripts/start_dev.sh
-</pre></div></div></li>
+
+<div>
+<div>
+<pre class="source">./scripts/start_dev.sh
+</pre></div></div>
+</li>
 </ol>
-<p>The application will be available at <a class="externalLink" href="http://localhost:4200/">http://localhost:4200/</a>. The REST application url defaults to <tt>http://localhost:8080</tt> but can be changed in the <tt>proxy.conf.json</tt> file.</p></div>
+<p>The application will be available at <a class="externalLink" href="http://localhost:4200/">http://localhost:4200/</a>.  The REST application url defaults to <tt>http://localhost:8080</tt> but can be changed in the <tt>proxy.conf.json</tt> file.</p></div>
 <div class="section">
 <h2><a name="Testing"></a>Testing</h2>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Install the application with dev dependencies:</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron-interface/metron-config
+
+<div>
+<div>
+<pre class="source">cd metron-interface/metron-config
 npm install
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Unit tests can be run with:</p>
-  
-<div class="source">
-<div class="source">
-<pre>npm test
-</pre></div></div></li>
+
+<div>
+<div>
+<pre class="source">npm test
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h2><a name="License"></a>License</h2>
-<p>This projects bundles Font Awesome which is available under the SIL Open Font License. See <a class="externalLink" href="http://fontawesome.io/license/">http://fontawesome.io/license/</a> for more details.</p></div>
-                  </div>
-            </div>
-          </div>
-
+<p>This projects bundles Font Awesome which is available under the SIL Open Font License.  See <a class="externalLink" href="http://fontawesome.io/license/">http://fontawesome.io/license/</a> for more details.</p></div>
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[17/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-enrichment/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-enrichment/index.html b/site/current-book/metron-platform/metron-enrichment/index.html
index 584c36f..de1e4fa 100644
--- a/site/current-book/metron-platform/metron-enrichment/index.html
+++ b/site/current-book/metron-platform/metron-enrichment/index.html
@@ -1,291 +1,165 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-enrichment/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Enrichment</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Enrichment</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Enrichment</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Enrichment</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Enrichment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/metron-enrichment/Performance.html" title="Performance"><span class="none"></span>Performance</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Enrichment</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Enrichment</h1>
 <p><a name="Enrichment"></a></p>
 <div class="section">
 <h2><a name="Introduction"></a>Introduction</h2>
-<p>The <tt>enrichment</tt> topology is a topology dedicated to taking the data from the parsing topologies that have been normalized into the Metron data format (e.g. a JSON Map structure with <tt>original_message</tt> and <tt>timestamp</tt>) and </p>
-
+<p>The <tt>enrichment</tt> topology is a topology dedicated to taking the data from the parsing topologies that have been normalized into the Metron data format (e.g. a JSON Map structure with <tt>original_message</tt> and <tt>timestamp</tt>) and</p>
 <ul>
-  
-<li>Enriching messages with external data from data stores (e.g. hbase) by  adding new fields based on existing fields in the messages.</li>
-  
+
+<li>Enriching messages with external data from data stores (e.g. hbase) by adding new fields based on existing fields in the messages.</li>
 <li>Marking messages as threats based on data in external data stores</li>
-  
-<li>Marking threat alerts with a numeric triage level based on a set of  Stellar rules.</li>
+<li>Marking threat alerts with a numeric triage level based on a set of Stellar rules.</li>
 </ul></div>
 <div class="section">
 <h2><a name="Enrichment_Architecture"></a>Enrichment Architecture</h2>
-<p><img src="../../images/enrichment_arch.png" alt="Architecture" /></p></div>
+<p><img src="../../images/enrichment_arch.png" alt="Architecture" /></p>
+<div class="section">
+<h3><a name="Unified_Enrichment_Topology"></a>Unified Enrichment Topology</h3>
+<p>There is an experimental unified enrichment topology which is shipped. Currently the architecture, as described above, has a split/join in order to perform enrichments in parallel.  This poses some issues in terms of ease of tuning and reasoning about performance.</p>
+<p>In order to deal with these issues, there is an alternative enrichment topology which uses data parallelism as opposed to the split/join task parallelism. This architecture uses a worker pool to fully enrich any message within a worker.  This results in</p>
+<ul>
+
+<li>Fewer bolts in the topology</li>
+<li>Each bolt fully operates on a message.</li>
+<li>Fewer network hops</li>
+</ul>
+<p><img src="unified_enrichment_arch.svg" alt="Unified Architecture" /></p>
+<p>This architecture is fully backwards compatible; the only difference is how the enrichment will operate on each message (in one bolt where the split/join is done in a threadpool as opposed to split across multiple bolts).</p>
+<div class="section">
+<h4><a name="Using_It"></a>Using It</h4>
+<p>In order to use this, you will need to</p>
+<ul>
+
+<li>Edit <tt>$METRON_HOME/bin/start_enrichment_topology.sh</tt> and adjust it to use <tt>remote-unified.yaml</tt> instead of <tt>remote.yaml</tt></li>
+<li>Restart the enrichment topology.</li>
+</ul></div>
+<div class="section">
+<h4><a name="Configuring_It"></a>Configuring It</h4>
+<p>There are two parameters which you might want to tune in this topology. Both of them are topology configuration adjustable in the flux file <tt>$METRON_HOME/config/flux/enrichment/remote-unified.yaml</tt>:</p>
+<ul>
+
+<li><tt>metron.threadpool.size</tt> : The size of the threadpool.  This can take a number or a multiple of the number of cores (e.g. <tt>5C</tt> to 5 times the number of cores).  The default is <tt>2C</tt>.</li>
+<li><tt>metron.threadpool.type</tt> : The type of threadpool. (note: descriptions taken from <a class="externalLink" href="https://zeroturnaround.com/rebellabs/fixedthreadpool-cachedthreadpool-or-forkjoinpool-picking-correct-java-executors-for-background-tasks/">here</a>).
+<ul>
+
+<li><tt>FIXED</tt> is a fixed threadpool of size <tt>n</tt>. <tt>n</tt> threads will process tasks at the time, when the pool is saturated, new tasks will get added to a queue without a limit on size. Good for CPU intensive tasks.  This is the default.</li>
+<li><tt>WORK_STEALING</tt> is a work stealing threadpool.  This will create and shut down threads dynamically to accommodate the required parallelism level. It also tries to reduce the contention on the task queue, so can be really good in heavily loaded environments. Also good when your tasks create more tasks for the executor, like recursive tasks.</li>
+</ul>
+</li>
+</ul>
+<p>In order to configure the parallelism for the enrichment bolt and threat intel bolt, the configurations will be taken from the respective join bolt parallelism.  When proper ambari support for this is added, we will add its own property.</p></div></div></div>
 <div class="section">
 <h2><a name="Enrichment_Configuration"></a>Enrichment Configuration</h2>
 <p>The configuration for the <tt>enrichment</tt> topology, the topology primarily responsible for enrichment and threat intelligence enrichment, is defined by JSON documents stored in zookeeper.</p>
-<p>There are two types of configurations at the moment, <tt>global</tt> and <tt>sensor</tt> specific. </p></div>
+<p>There are two types of configurations at the moment, <tt>global</tt> and <tt>sensor</tt> specific.</p></div>
 <div class="section">
 <h2><a name="Global_Configuration"></a>Global Configuration</h2>
 <p>There are a few enrichments which have independent configurations, such as from the global config.</p>
@@ -295,69 +169,49 @@
 <p>Metron supports enrichment of IP information using <a class="externalLink" href="https://dev.maxmind.com/geoip/geoip2/geolite2/">GeoLite2</a>. The location of the file is managed in the global config.</p>
 <div class="section">
 <h4><a name="geo.hdfs.file"></a><tt>geo.hdfs.file</tt></h4>
-<p>The location on HDFS of the GeoLite2 database file to use for GeoIP lookups. This file will be localized on the storm supervisors running the topology and used from there. This is lazy, so if this property changes in a running topology, the file will be localized from HDFS upon first time the file is used via the geo enrichment. </p></div></div></div>
+<p>The location on HDFS of the GeoLite2 database file to use for GeoIP lookups.  This file will be localized on the storm supervisors running the topology and used from there. This is lazy, so if this property changes in a running topology, the file will be localized from HDFS upon first time the file is used via the geo enrichment.</p></div></div></div>
 <div class="section">
 <h2><a name="Sensor_Enrichment_Configuration"></a>Sensor Enrichment Configuration</h2>
 <p>The sensor specific configuration is intended to configure the individual enrichments and threat intelligence enrichments for a given sensor type (e.g. <tt>snort</tt>).</p>
 <p>Just like the global config, the format is a JSON stored in zookeeper. The configuration is a complex JSON object with the following top level fields:</p>
-
 <ul>
-  
+
 <li><tt>enrichment</tt> : A complex JSON object representing the configuration of the enrichments</li>
-  
 <li><tt>threatIntel</tt> : A complex JSON object representing the configuration of the threat intelligence enrichments</li>
 </ul>
 <div class="section">
 <h3><a name="The_enrichment_Configuration"></a>The <tt>enrichment</tt> Configuration</h3>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Field </th>
-      
-<th>Description </th>
-      
-<th>Example </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Field            </th>
+<th> Description                                                                                                                                                                                                                   </th>
+<th> Example                                                          </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td><tt>fieldToTypeMap</tt> </td>
-      
-<td>In the case of a simple HBase enrichment (i.e. a key/value lookup), the mapping between fields and the enrichment types associated with those fields must be known. This enrichment type is used as part of the HBase key. Note: applies to hbaseEnrichment only. </td>
-      
-<td><tt>&quot;fieldToTypeMap&quot; : { &quot;ip_src_addr&quot; : [ &quot;asset_enrichment&quot; ] }</tt> </td>
-    </tr>
-    
+<td> <tt>fieldToTypeMap</tt> </td>
+<td> In the case of a simple HBase enrichment (i.e. a key/value lookup), the mapping between fields and the enrichment types associated with those fields must be known.  This enrichment type is used as part of the HBase key. Note: applies to hbaseEnrichment only. </td>
+<td> <tt>&quot;fieldToTypeMap&quot; : { &quot;ip_src_addr&quot; : [ &quot;asset_enrichment&quot; ] }</tt>  </td></tr>
 <tr class="a">
-      
-<td><tt>fieldMap</tt> </td>
-      
-<td>The map of enrichment bolts names to configuration handlers which know how to split the message up. The simplest of which is just a list of fields. More complex examples would be the stellar enrichment which provides stellar statements. Each field listed in the array arg is sent to the enrichment referenced in the key. Cardinality of fields to enrichments is many-to-many. </td>
-      
-<td><tt>&quot;fieldMap&quot;: {&quot;hbaseEnrichment&quot;: [&quot;ip_src_addr&quot;,&quot;ip_dst_addr&quot;]}</tt> </td>
-    </tr>
-    
+<td> <tt>fieldMap</tt>       </td>
+<td> The map of enrichment bolts names to configuration handlers which know how to split the message up.  The simplest of which is just a list of fields.  More complex examples would be the stellar enrichment which provides stellar statements. Each field listed in the array arg is sent to the enrichment referenced in the key. Cardinality of fields to enrichments is many-to-many. </td>
+<td> <tt>&quot;fieldMap&quot;: {&quot;hbaseEnrichment&quot;: [&quot;ip_src_addr&quot;,&quot;ip_dst_addr&quot;]}</tt> </td></tr>
 <tr class="b">
-      
-<td><tt>config</tt> </td>
-      
-<td>The general configuration for the enrichment </td>
-      
-<td><tt>&quot;config&quot;: {&quot;typeToColumnFamily&quot;: { &quot;asset_enrichment&quot; : &quot;cf&quot; } }</tt> </td>
-    </tr>
-  </tbody>
+<td> <tt>config</tt>         </td>
+<td> The general configuration for the enrichment                                                                                                                                                                                  </td>
+<td> <tt>&quot;config&quot;: {&quot;typeToColumnFamily&quot;: { &quot;asset_enrichment&quot; : &quot;cf&quot; } }</tt> </td></tr>
+</tbody>
 </table>
 <p>The <tt>config</tt> map is intended to house enrichment specific configuration. For instance, for the <tt>hbaseEnrichment</tt>, the mappings between the enrichment types to the column families is specified.</p>
-<p>The <tt>fieldMap</tt>contents are of interest because they contain the routing and configuration information for the enrichments.<br />When we say &#x2018;routing&#x2019;, we mean how the messages get split up and sent to the enrichment adapter bolts.<br />The simplest, by far, is just providing a simple list as in</p>
+<p>The <tt>fieldMap</tt>contents are of interest because they contain the routing and configuration information for the enrichments.<br />
+When we say &#x2018;routing&#x2019;, we mean how the messages get split up and sent to the enrichment adapter bolts.<br />
+The simplest, by far, is just providing a simple list as in</p>
 
-<div class="source">
-<div class="source">
-<pre>    &quot;fieldMap&quot;: {
+<div>
+<div>
+<pre class="source">    &quot;fieldMap&quot;: {
       &quot;geo&quot;: [
         &quot;ip_src_addr&quot;,
         &quot;ip_dst_addr&quot;
@@ -372,15 +226,16 @@
       ]
       }
 </pre></div></div>
-<p>Based on this sample config, both <tt>ip_src_addr</tt> and <tt>ip_dst_addr</tt> will go to the <tt>geo</tt>, <tt>host</tt>, and <tt>hbaseEnrichment</tt> adapter bolts. </p>
+
+<p>Based on this sample config, both <tt>ip_src_addr</tt> and <tt>ip_dst_addr</tt> will go to the <tt>geo</tt>, <tt>host</tt>, and <tt>hbaseEnrichment</tt> adapter bolts.</p>
 <div class="section">
 <h4><a name="Stellar_Enrichment_Configuration"></a>Stellar Enrichment Configuration</h4>
-<p>For the <tt>geo</tt>, <tt>host</tt> and <tt>hbaseEnrichment</tt>, this is sufficient. However, more complex enrichments may contain their own configuration. Currently, the <tt>stellar</tt> enrichment is more adaptable and thus requires a more nuanced configuration.</p>
-<p>At its most basic, we want to take a message and apply a couple of enrichments, such as converting the <tt>hostname</tt> field to lowercase. We do this by specifying the transformation inside of the <tt>config</tt> for the <tt>stellar</tt> fieldMap. There are two syntaxes that are supported, specifying the transformations as a map with the key as the field and the value the stellar expression:</p>
+<p>For the <tt>geo</tt>, <tt>host</tt> and <tt>hbaseEnrichment</tt>, this is sufficient. However, more complex enrichments may contain their own configuration.  Currently, the <tt>stellar</tt> enrichment is more adaptable and thus requires a more nuanced configuration.</p>
+<p>At its most basic, we want to take a message and apply a couple of enrichments, such as converting the <tt>hostname</tt> field to lowercase. We do this by specifying the transformation inside of the <tt>config</tt> for the <tt>stellar</tt> fieldMap.  There are two syntaxes that are supported, specifying the transformations as a map with the key as the field and the value the stellar expression:</p>
 
-<div class="source">
-<div class="source">
-<pre>    &quot;fieldMap&quot;: {
+<div>
+<div>
+<pre class="source">    &quot;fieldMap&quot;: {
        ...
       &quot;stellar&quot; : {
         &quot;config&quot; : {
@@ -389,11 +244,12 @@
       }
     }
 </pre></div></div>
+
 <p>Another approach is to make the transformations as a list with the same <tt>var := expr</tt> syntax as is used in the Stellar REPL, such as:</p>
 
-<div class="source">
-<div class="source">
-<pre>    &quot;fieldMap&quot;: {
+<div>
+<div>
+<pre class="source">    &quot;fieldMap&quot;: {
        ...
       &quot;stellar&quot; : {
         &quot;config&quot; : [
@@ -402,12 +258,13 @@
       }
     }
 </pre></div></div>
-<p>Sometimes arbitrary stellar enrichments may take enough time that you would prefer to split some of them into groups and execute the groups of stellar enrichments in parallel. Take, for instance, if you wanted to do an HBase enrichment and a profiler call which were independent of one another. This usecase is supported by splitting the enrichments up as groups.</p>
+
+<p>Sometimes arbitrary stellar enrichments may take enough time that you would prefer to split some of them into groups and execute the groups of stellar enrichments in parallel.  Take, for instance, if you wanted to do an HBase enrichment and a profiler call which were independent of one another.  This usecase is supported by splitting the enrichments up as groups.</p>
 <p>Consider the following example:</p>
 
-<div class="source">
-<div class="source">
-<pre>    &quot;fieldMap&quot;: {
+<div>
+<div>
+<pre class="source">    &quot;fieldMap&quot;: {
        ...
       &quot;stellar&quot; : {
         &quot;config&quot; : {
@@ -428,143 +285,93 @@
       }
     }
 </pre></div></div>
-<p>Here we want to perform two enrichments that hit HBase and we would rather not run in sequence. These enrichments are entirely independent of one another (i.e. neither relies on the output of the other). In this case, we&#x2019;ve created a group called <tt>malicious_domain_enrichment</tt> to inquire about whether the destination address exists in the HBase enrichment table in the <tt>malicious_domains</tt> enrichment type. This is a simple enrichment, so we can express the enrichment group as a map with the new field <tt>is_bad_domain</tt> being a key and the stellar expression associated with that operation being the associated value.</p>
-<p>In contrast, the stellar enrichment group <tt>login_profile</tt> is interacting with the profiler, has multiple temporary expressions (i.e. <tt>profile_window</tt>, <tt>global_login_profile</tt>, and <tt>stats</tt>) that are useful only within the context of this group of stellar expressions. In this case, we would need to ensure that we use the list construct when specifying the group and remember to set the temporary variables to <tt>null</tt> so they are not passed along.</p>
-<p>In general, things to note from this section are as follows:</p>
 
+<p>Here we want to perform two enrichments that hit HBase and we would rather not run in sequence.  These enrichments are entirely independent of one another (i.e. neither relies on the output of the other).  In this case, we&#x2019;ve created a group called <tt>malicious_domain_enrichment</tt> to inquire about whether the destination address exists in the HBase enrichment table in the <tt>malicious_domains</tt> enrichment type.  This is a simple enrichment, so we can express the enrichment group as a map with the new field <tt>is_bad_domain</tt> being a key and the stellar expression associated with that operation being the associated value.</p>
+<p>In contrast, the stellar enrichment group <tt>login_profile</tt> is interacting with the profiler, has multiple temporary expressions (i.e. <tt>profile_window</tt>, <tt>global_login_profile</tt>, and <tt>stats</tt>) that are useful only within the context of this group of stellar expressions.  In this case, we would need to ensure that we use the list construct when specifying the group and remember to set the temporary variables to <tt>null</tt> so they are not passed along.</p>
+<p>In general, things to note from this section are as follows:</p>
 <ul>
-  
+
 <li>The stellar enrichments for the <tt>stellar</tt> enrichment adapter are specified in the <tt>config</tt> for the <tt>stellar</tt> enrichment adapter in the <tt>fieldMap</tt></li>
-  
 <li>Groups of independent (i.e. no expression in any group depend on the output of an expression from an other group) may be executed in parallel</li>
-  
-<li>If you have the need to use temporary variables, you may use the list construct. Ensure that you assign the variables to <tt>null</tt> before the end of the group.</li>
-  
+<li>If you have the need to use temporary variables, you may use the list construct.  Ensure that you assign the variables to <tt>null</tt> before the end of the group.</li>
 <li><b>Ensure that you do not assign a field to a stellar expression which returns an object which JSON cannot represent.</b></li>
-  
 <li>Fields assigned to Maps as part of stellar enrichments have the maps unfolded, similar to the HBase Enrichment
-  
 <ul>
-    
-<li>For example the stellar enrichment for field <tt>foo</tt> which assigns a map such as <tt>foo := { 'grok' : 1, 'bar' : 'baz'}</tt>  would yield the following fields:
-    
+
+<li>For example the stellar enrichment for field <tt>foo</tt> which assigns a map such as <tt>foo := { 'grok' : 1, 'bar' : 'baz'}</tt> would yield the following fields:
 <ul>
-      
+
 <li><tt>foo.grok</tt> == <tt>1</tt></li>
-      
 <li><tt>foo.bar</tt> == <tt>'baz'</tt></li>
-    </ul></li>
-  </ul></li>
+</ul>
+</li>
+</ul>
+</li>
 </ul></div></div>
 <div class="section">
 <h3><a name="The_threatIntel_Configuration"></a>The <tt>threatIntel</tt> Configuration</h3>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Field </th>
-      
-<th>Description </th>
-      
-<th>Example </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Field            </th>
+<th> Description                                                                                                                                                                                                                                   </th>
+<th> Example                                                                  </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td><tt>fieldToTypeMap</tt> </td>
-      
-<td>In the case of a simple HBase threat intel enrichment (i.e. a key/value lookup), the mapping between fields and the enrichment types associated with those fields must be known. This enrichment type is used as part of the HBase key. Note: applies to hbaseThreatIntel only. </td>
-      
-<td><tt>&quot;fieldToTypeMap&quot; : { &quot;ip_src_addr&quot; : [ &quot;malicious_ips&quot; ] }</tt> </td>
-    </tr>
-    
+<td> <tt>fieldToTypeMap</tt> </td>
+<td> In the case of a simple HBase threat intel enrichment (i.e. a key/value lookup), the mapping between fields and the enrichment types associated with those fields must be known.  This enrichment type is used as part of the HBase key. Note: applies to hbaseThreatIntel only. </td>
+<td> <tt>&quot;fieldToTypeMap&quot; : { &quot;ip_src_addr&quot; : [ &quot;malicious_ips&quot; ] }</tt>             </td></tr>
 <tr class="a">
-      
-<td><tt>fieldMap</tt> </td>
-      
-<td>The map of threat intel enrichment bolts names to fields in the JSON messages. Each field is sent to the threat intel enrichment bolt referenced in the key. Each field listed in the array arg is sent to the enrichment referenced in the key. Cardinality of fields to enrichments is many-to-many. </td>
-      
-<td><tt>&quot;fieldMap&quot;: {&quot;hbaseThreatIntel&quot;: [&quot;ip_src_addr&quot;,&quot;ip_dst_addr&quot;]}</tt> </td>
-    </tr>
-    
+<td> <tt>fieldMap</tt>       </td>
+<td> The map of threat intel enrichment bolts names to fields in the JSON messages. Each field is sent to the threat intel enrichment bolt referenced in the key. Each field listed in the array arg is sent to the enrichment referenced in the key. Cardinality of fields to enrichments is many-to-many.                                                     </td>
+<td> <tt>&quot;fieldMap&quot;: {&quot;hbaseThreatIntel&quot;: [&quot;ip_src_addr&quot;,&quot;ip_dst_addr&quot;]}</tt>        </td></tr>
 <tr class="b">
-      
-<td><tt>triageConfig</tt> </td>
-      
-<td>The configuration of the threat triage scorer. In the situation where a threat is detected, a score is assigned to the message and embedded in the indexed message. </td>
-      
-<td><tt>&quot;riskLevelRules&quot; : { &quot;IN_SUBNET(ip_dst_addr, '192.168.0.0/24')&quot; : 10 }</tt> </td>
-    </tr>
-    
+<td> <tt>triageConfig</tt>   </td>
+<td> The configuration of the threat triage scorer.  In the situation where a threat is detected, a score is assigned to the message and embedded in the indexed message.                                                                    </td>
+<td> <tt>&quot;riskLevelRules&quot; : { &quot;IN_SUBNET(ip_dst_addr, '192.168.0.0/24')&quot; : 10 }</tt> </td></tr>
 <tr class="a">
-      
-<td><tt>config</tt> </td>
-      
-<td>The general configuration for the Threat Intel </td>
-      
-<td><tt>&quot;config&quot;: {&quot;typeToColumnFamily&quot;: { &quot;malicious_ips&quot;,&quot;cf&quot; } }</tt> </td>
-    </tr>
-  </tbody>
+<td> <tt>config</tt>         </td>
+<td> The general configuration for the Threat Intel                                                                                                                                                                                                </td>
+<td> <tt>&quot;config&quot;: {&quot;typeToColumnFamily&quot;: { &quot;malicious_ips&quot;,&quot;cf&quot; } }</tt>            </td></tr>
+</tbody>
 </table>
-<p>The <tt>config</tt> map is intended to house threat intel specific configuration. For instance, for the <tt>hbaseThreatIntel</tt> threat intel adapter, the mappings between the enrichment types to the column families is specified. The <tt>fieldMap</tt> configuration is similar to the <tt>enrichment</tt> configuration in that the adapters available are the same.</p>
+<p>The <tt>config</tt> map is intended to house threat intel specific configuration. For instance, for the <tt>hbaseThreatIntel</tt> threat intel adapter, the mappings between the enrichment types to the column families is specified.  The <tt>fieldMap</tt> configuration is similar to the <tt>enrichment</tt> configuration in that the adapters available are the same.</p>
 <p>The <tt>triageConfig</tt> field is also a complex field and it bears some description:</p>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Field </th>
-      
-<th>Description </th>
-      
-<th>Example </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Field            </th>
+<th> Description                                                                                                                                             </th>
+<th> Example                                                                  </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td><tt>riskLevelRules</tt> </td>
-      
-<td>This is a list of rules (represented as Stellar expressions) associated with scores with optional names and comments </td>
-      
-<td>see below</td>
-    </tr>
-    
+<td> <tt>riskLevelRules</tt> </td>
+<td> This is a list of rules (represented as Stellar expressions) associated with scores with optional names and comments                                    </td>
+<td>  see below</td></tr>
 <tr class="a">
-      
-<td><tt>aggregator</tt> </td>
-      
-<td>An aggregation function that takes all non-zero scores representing the matching queries from <tt>riskLevelRules</tt> and aggregates them into a single score. </td>
-      
-<td><tt>&quot;MAX&quot;</tt> </td>
-    </tr>
-  </tbody>
+<td> <tt>aggregator</tt>     </td>
+<td> An aggregation function that takes all non-zero scores representing the matching queries from <tt>riskLevelRules</tt> and aggregates them into a single score. </td>
+<td> <tt>&quot;MAX&quot;</tt>                                                                  </td></tr>
+</tbody>
 </table>
 <p>A risk level rule is of the following format:</p>
-
 <ul>
-  
+
 <li><tt>name</tt> : The name of the threat triage rule</li>
-  
 <li><tt>comment</tt> : A comment describing the rule</li>
-  
 <li><tt>rule</tt> : The rule, represented as a Stellar statement</li>
-  
 <li><tt>score</tt> : Associated threat triage score for the rule</li>
-  
 <li><tt>reason</tt> : Reason the rule tripped. Can be represented as a Stellar statement</li>
 </ul>
 <p>An example of a rule is as follows:</p>
 
-<div class="source">
-<div class="source">
-<pre>    &quot;riskLevelRules&quot; : [ 
+<div>
+<div>
+<pre class="source">    &quot;riskLevelRules&quot; : [ 
         { 
           &quot;name&quot; : &quot;is internal&quot;
         , &quot;comment&quot; : &quot;determines if the destination is internal.&quot;
@@ -574,27 +381,23 @@
         }
                        ]
 </pre></div></div>
-<p>The supported aggregation functions are:</p>
 
+<p>The supported aggregation functions are:</p>
 <ul>
-  
+
 <li><tt>MAX</tt> : The max of all of the associated values for matching queries</li>
-  
 <li><tt>MIN</tt> : The min of all of the associated values for matching queries</li>
-  
 <li><tt>MEAN</tt> : The mean of all of the associated values for matching queries</li>
-  
 <li><tt>SUM</tt> : The sum of all the associated values for matching queries</li>
-  
 <li><tt>POSITIVE_MEAN</tt> : The mean of the positive associated values for the matching queries.</li>
 </ul></div>
 <div class="section">
 <h3><a name="Example_Configuration"></a>Example Configuration</h3>
 <p>An example configuration for the YAF sensor is as follows:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;enrichment&quot;: {
     &quot;fieldMap&quot;: {
       &quot;geo&quot;: [
@@ -646,6 +449,7 @@
   }
 }
 </pre></div></div>
+
 <p>ThreatIntel alert levels are emitted as a new field &#x201c;threat.triage.level.&#x201d; So for the example above, an incoming message that trips the <tt>ip_src_addr</tt> rule will have a new field threat.triage.level=10.</p>
 <p><a name="Example_Enrichment_via_Stellar"></a></p>
 <h1>Example Enrichment via Stellar</h1>
@@ -653,113 +457,95 @@
 <div class="section">
 <h2><a name="Install_Prerequisites"></a>Install Prerequisites</h2>
 <p>Now let&#x2019;s install some prerequisites:</p>
-
 <ul>
-  
+
 <li>Squid client via <tt>yum install squid</tt></li>
-  
 <li>ES Head plugin via <tt>/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head</tt></li>
 </ul>
 <p>Start Squid via <tt>service squid start</tt></p></div>
 <div class="section">
 <h2><a name="Adjust_Enrichment_Configurations_for_Squid_to_Call_Stellar"></a>Adjust Enrichment Configurations for Squid to Call Stellar</h2>
 <p>Let&#x2019;s adjust the configurations for the Squid topology to annotate the messages using some Stellar functions.</p>
-
 <ul>
-  
-<li>
-<p>Edit the squid enrichment configuration at <tt>$METRON_HOME/config/zookeeper/enrichments/squid.json</tt> (this file will not exist, so create a new one) to add some new fields based on stellar queries:</p>
-  
-<div class="source">
-<div class="source">
-<pre>{
+
+<li>Edit the squid enrichment configuration at <tt>$METRON_HOME/config/zookeeper/enrichments/squid.json</tt> (this file will not exist, so create a new one) to add some new fields based on stellar queries:</li>
+</ul>
+
+<div>
+<div>
+<pre class="source">{
   &quot;enrichment&quot; : {
-&quot;fieldMap&quot;: {
-  &quot;stellar&quot; : {
-    &quot;config&quot; : {
-      &quot;numeric&quot; : {
-                  &quot;foo&quot;: &quot;1 + 1&quot;
-                  }
-      ,&quot;ALL_CAPS&quot; : &quot;TO_UPPER(source.type)&quot;
-    }
-  }
- }
+    &quot;fieldMap&quot;: {
+      &quot;stellar&quot; : {
+        &quot;config&quot; : {
+          &quot;numeric&quot; : {
+                      &quot;foo&quot;: &quot;1 + 1&quot;
+                      }
+          ,&quot;ALL_CAPS&quot; : &quot;TO_UPPER(source.type)&quot;
+        }
+      }
+     }
   },
   &quot;threatIntel&quot; : {
-&quot;fieldMap&quot;:{
- &quot;stellar&quot; : {
-    &quot;config&quot; : {
-      &quot;bar&quot; : &quot;TO_UPPER(source.type)&quot;
+    &quot;fieldMap&quot;:{
+     &quot;stellar&quot; : {
+        &quot;config&quot; : {
+          &quot;bar&quot; : &quot;TO_UPPER(source.type)&quot;
+        }
+      } 
+    },
+    &quot;triageConfig&quot; : {
     }
-  } 
-},
-&quot;triageConfig&quot; : {
-}
   }
 }
 </pre></div></div>
-<p>We have added the following fields as part of the enrichment phase of the enrichment topology:</p></li>
-  
-<li>
-<p><tt>foo</tt> == 2</p></li>
-  
+
+<p>We have added the following fields as part of the enrichment phase of the enrichment topology:</p>
+<ul>
+
+<li><tt>foo</tt> ==  2</li>
 <li><tt>ALL_CAPS</tt> == SQUID</li>
 </ul>
 <p>We have added the following as part of the threat intel:</p>
-
 <ul>
-  
+
 <li><tt>bar</tt> == SQUID</li>
 </ul>
 <p>Please note that foo and ALL_CAPS will be applied in separate workers due to them being in separate groups.</p>
-
 <ul>
-  
+
 <li>Upload new configs via <tt>$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -i $METRON_HOME/config/zookeeper -z node1:2181</tt></li>
-  
 <li>Make the Squid topic in kafka via <tt>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper node1:2181 --create --topic squid --partitions 1 --replication-factor 1</tt></li>
 </ul></div>
 <div class="section">
 <h2><a name="Start_Topologies_and_Send_Data"></a>Start Topologies and Send Data</h2>
 <p>Now we need to start the topologies and send some data:</p>
-
 <ul>
-  
+
 <li>Start the squid topology via <tt>$METRON_HOME/bin/start_parser_topology.sh -k node1:6667 -z node1:2181 -s squid</tt></li>
-  
 <li>Generate some data via the squid client:
-  
 <ul>
-    
+
 <li><tt>squidclient http://yahoo.com</tt></li>
-    
 <li><tt>squidclient http://cnn.com</tt></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Send the data to kafka via <tt>cat /var/log/squid/access.log | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic squid</tt></li>
-  
 <li>Browse the data in elasticsearch via the ES Head plugin @ <a class="externalLink" href="http://node1:9200/_plugin/head/">http://node1:9200/_plugin/head/</a> and verify that in the squid index you have two documents</li>
-  
 <li>Ensure that the documents have new fields <tt>foo</tt>, <tt>bar</tt> and <tt>ALL_CAPS</tt> with values as described above.</li>
 </ul>
 <p>Note that we could have used any Stellar statements here, including calling out to HBase via <tt>ENRICHMENT_GET</tt> and <tt>ENRICHMENT_EXISTS</tt> or even calling a machine learning model via <a href="../../metron-analytics/metron-maas-service/index.html">Model as a Service</a>.</p></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[22/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-interface/metron-rest/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-interface/metron-rest/index.html b/site/current-book/metron-interface/metron-rest/index.html
index f673ca6..eb378b3 100644
--- a/site/current-book/metron-interface/metron-rest/index.html
+++ b/site/current-book/metron-interface/metron-rest/index.html
@@ -1,253 +1,157 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-interface/metron-rest/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Metron REST</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Metron REST</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron REST</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Rest</a>
-          </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Metron REST</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron REST</h1>
 <p><a name="Metron_REST"></a></p>
 <p>This module provides a RESTful API for interacting with Metron.</p>
 <div class="section">
 <h2><a name="Prerequisites"></a>Prerequisites</h2>
-
 <ul>
-  
+
 <li>A running Metron cluster</li>
-  
 <li>Java 8 installed</li>
-  
 <li>Storm CLI and Metron topology scripts (start_parser_topology.sh, start_enrichment_topology.sh, start_elasticsearch_topology.sh) installed</li>
-  
 <li>A relational database</li>
 </ul></div>
 <div class="section">
 <h2><a name="Installation"></a>Installation</h2>
 <div class="section">
 <h3><a name="From_Source"></a>From Source</h3>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Package the application with Maven:</p>
-  
-<div class="source">
-<div class="source">
-<pre>mvn clean package
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">mvn clean package
+</pre></div></div>
+</li>
 <li>
-<p>Untar the archive in the $METRON_HOME directory. The directory structure will look like:</p>
-  
-<div class="source">
-<div class="source">
-<pre>config
+
+<p>Untar the archive in the $METRON_HOME directory.  The directory structure will look like:</p>
+
+<div>
+<div>
+<pre class="source">config
   rest_application.yml
 bin
   metron-rest
 lib
   metron-rest-$METRON_VERSION.jar
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Copy the <tt>$METRON_HOME/bin/metron-rest</tt> script to <tt>/etc/init.d/metron-rest</tt></p></li>
+
+<p>Copy the <tt>$METRON_HOME/bin/metron-rest</tt> script to <tt>/etc/init.d/metron-rest</tt></p>
+</li>
 </ol></div>
 <div class="section">
 <h3><a name="From_Package_Manager"></a>From Package Manager</h3>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Deploy the RPM at <tt>/metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-rest-$METRON_VERSION-*.noarch.rpm</tt></p></li>
-  
+
+<p>Deploy the RPM at <tt>/metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-rest-$METRON_VERSION-*.noarch.rpm</tt></p>
+</li>
 <li>
+
 <p>Install the RPM with:</p>
-  
-<div class="source">
-<div class="source">
-<pre>rpm -ih metron-rest-$METRON_VERSION-*.noarch.rpm
-</pre></div></div></li>
+
+<div>
+<div>
+<pre class="source">rpm -ih metron-rest-$METRON_VERSION-*.noarch.rpm
+</pre></div></div>
+</li>
 </ol></div></div>
 <div class="section">
 <h2><a name="Configuration"></a>Configuration</h2>
@@ -255,245 +159,137 @@ lib
 <div class="section">
 <h3><a name="REQUIRED"></a>REQUIRED</h3>
 <p>No optional parameter has a default.</p>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Environment Variable </th>
-      
-<th>Description</th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Environment Variable                  </th>
+<th> Description</th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>METRON_JDBC_DRIVER </td>
-      
-<td>JDBC driver class</td>
-    </tr>
-    
+<td> METRON_JDBC_DRIVER                    </td>
+<td> JDBC driver class</td></tr>
 <tr class="a">
-      
-<td>METRON_JDBC_URL </td>
-      
-<td>JDBC url</td>
-    </tr>
-    
+<td> METRON_JDBC_URL                       </td>
+<td> JDBC url</td></tr>
 <tr class="b">
-      
-<td>METRON_JDBC_USERNAME </td>
-      
-<td>JDBC username</td>
-    </tr>
-    
+<td> METRON_JDBC_USERNAME                  </td>
+<td> JDBC username</td></tr>
 <tr class="a">
-      
-<td>METRON_JDBC_PLATFORM </td>
-      
-<td>JDBC platform (one of h2, mysql, postgres, oracle</td>
-    </tr>
-    
+<td> METRON_JDBC_PLATFORM                  </td>
+<td> JDBC platform (one of h2, mysql, postgres, oracle</td></tr>
 <tr class="b">
-      
-<td>ZOOKEEPER </td>
-      
-<td>Zookeeper quorum (ex. node1:2181,node2:2181)</td>
-    </tr>
-    
+<td> ZOOKEEPER                             </td>
+<td> Zookeeper quorum (ex. node1:2181,node2:2181)</td></tr>
 <tr class="a">
-      
-<td>BROKERLIST </td>
-      
-<td>Kafka Broker list (ex. node1:6667,node2:6667)</td>
-    </tr>
-    
+<td> BROKERLIST                            </td>
+<td> Kafka Broker list (ex. node1:6667,node2:6667)</td></tr>
 <tr class="b">
-      
-<td>HDFS_URL </td>
-      
-<td>HDFS url or <tt>fs.defaultFS</tt> Hadoop setting (ex. <a class="externalLink" href="hdfs://node1:8020">hdfs://node1:8020</a>)</td>
-    </tr>
-  </tbody>
+<td> HDFS_URL                              </td>
+<td> HDFS url or <tt>fs.defaultFS</tt> Hadoop setting (ex. <a class="externalLink" href="hdfs://node1:8020">hdfs://node1:8020</a>)</td></tr>
+</tbody>
 </table></div>
 <div class="section">
 <h3><a name="Optional_-_With_Defaults"></a>Optional - With Defaults</h3>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Environment Variable </th>
-      
-<th>Description </th>
-      
-<th>Required </th>
-      
-<th>Default</th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Environment Variable                  </th>
+<th> Description                                                       </th>
+<th> Required </th>
+<th> Default</th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>METRON_LOG_DIR </td>
-      
-<td>Directory where the log file is written </td>
-      
-<td>Optional </td>
-      
-<td>/var/log/metron/</td>
-    </tr>
-    
+<td> METRON_LOG_DIR                        </td>
+<td> Directory where the log file is written                           </td>
+<td> Optional </td>
+<td> /var/log/metron/</td></tr>
 <tr class="a">
-      
-<td>METRON_PID_FILE </td>
-      
-<td>File where the pid is written </td>
-      
-<td>Optional </td>
-      
-<td>/var/run/metron/</td>
-    </tr>
-    
+<td> METRON_PID_FILE                       </td>
+<td> File where the pid is written                                     </td>
+<td> Optional </td>
+<td> /var/run/metron/</td></tr>
 <tr class="b">
-      
-<td>METRON_REST_PORT </td>
-      
-<td>REST application port </td>
-      
-<td>Optional </td>
-      
-<td>8082</td>
-    </tr>
-    
+<td> METRON_REST_PORT                      </td>
+<td> REST application port                                             </td>
+<td> Optional </td>
+<td> 8082</td></tr>
 <tr class="a">
-      
-<td>METRON_JDBC_CLIENT_PATH </td>
-      
-<td>Path to JDBC client jar </td>
-      
-<td>Optional </td>
-      
-<td>H2 is bundled</td>
-    </tr>
-    
+<td> METRON_JDBC_CLIENT_PATH               </td>
+<td> Path to JDBC client jar                                           </td>
+<td> Optional </td>
+<td> H2 is bundled</td></tr>
 <tr class="b">
-      
-<td>METRON_TEMP_GROK_PATH </td>
-      
-<td>Temporary directory used to test grok statements </td>
-      
-<td>Optional </td>
-      
-<td>./patterns/temp</td>
-    </tr>
-    
+<td> METRON_TEMP_GROK_PATH                 </td>
+<td> Temporary directory used to test grok statements                  </td>
+<td> Optional </td>
+<td> ./patterns/temp</td></tr>
 <tr class="a">
-      
-<td>METRON_DEFAULT_GROK_PATH </td>
-      
-<td>Defaults HDFS directory used to store grok statements </td>
-      
-<td>Optional </td>
-      
-<td>/apps/metron/patterns</td>
-    </tr>
-    
+<td> METRON_DEFAULT_GROK_PATH              </td>
+<td> Defaults HDFS directory used to store grok statements             </td>
+<td> Optional </td>
+<td> /apps/metron/patterns</td></tr>
 <tr class="b">
-      
-<td>SECURITY_ENABLED </td>
-      
-<td>Enables Kerberos support </td>
-      
-<td>Optional </td>
-      
-<td>false</td>
-    </tr>
-  </tbody>
+<td> SECURITY_ENABLED                      </td>
+<td> Enables Kerberos support                                          </td>
+<td> Optional </td>
+<td> false</td></tr>
+</tbody>
 </table></div>
 <div class="section">
 <h3><a name="Optional_-_Blank_Defaults"></a>Optional - Blank Defaults</h3>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Environment Variable </th>
-      
-<th>Description </th>
-      
-<th>Required</th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Environment Variable                  </th>
+<th> Description                                                       </th>
+<th> Required</th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>METRON_JVMFLAGS </td>
-      
-<td>JVM flags added to the start command </td>
-      
-<td>Optional</td>
-    </tr>
-    
+<td> METRON_JVMFLAGS                       </td>
+<td> JVM flags added to the start command                              </td>
+<td> Optional</td></tr>
 <tr class="a">
-      
-<td>METRON_SPRING_PROFILES_ACTIVE </td>
-      
-<td>Active Spring profiles (see <a href="#spring-profiles">below</a>) </td>
-      
-<td>Optional</td>
-    </tr>
-    
+<td> METRON_SPRING_PROFILES_ACTIVE         </td>
+<td> Active Spring profiles (see <a href="#spring-profiles">below</a>)            </td>
+<td> Optional</td></tr>
 <tr class="b">
-      
-<td>METRON_SPRING_OPTIONS </td>
-      
-<td>Additional Spring input parameters </td>
-      
-<td>Optional</td>
-    </tr>
-    
+<td> METRON_SPRING_OPTIONS                 </td>
+<td> Additional Spring input parameters                                </td>
+<td> Optional</td></tr>
 <tr class="a">
-      
-<td>METRON_PRINCIPAL_NAME </td>
-      
-<td>Kerberos principal for the metron user </td>
-      
-<td>Optional</td>
-    </tr>
-    
+<td> METRON_PRINCIPAL_NAME                 </td>
+<td> Kerberos principal for the metron user                            </td>
+<td> Optional</td></tr>
 <tr class="b">
-      
-<td>METRON_SERVICE_KEYTAB </td>
-      
-<td>Path to the Kerberos keytab for the metron user </td>
-      
-<td>Optional</td>
-    </tr>
-  </tbody>
+<td> METRON_SERVICE_KEYTAB                 </td>
+<td> Path to the Kerberos keytab for the metron user                   </td>
+<td> Optional</td></tr>
+</tbody>
 </table>
 <p>These are set in the <tt>/etc/default/metron</tt> file.</p></div></div>
 <div class="section">
 <h2><a name="Database_setup"></a>Database setup</h2>
-<p>The REST application persists data in a relational database and requires a dedicated database user and database (see <a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html">https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html</a> for more detail).<br />Spring uses Hibernate as the default ORM framework but another framework is needed becaused Hibernate is not compatible with the Apache 2 license. For this reason Metron uses <a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html#boot-features-embedded-database-support">EclipseLink</a>. See the <a class="externalLink" href="https://github.com/spring-projects/spring-data-examples/tree/master/jpa/eclipselink">Spring Data JPA - EclipseLink</a> project for an example on how to configure EclipseLink in Spring.</p>
+<p>The REST application persists data in a relational database and requires a dedicated database user and database (see <a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html">https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html</a> for more detail).<br />
+Spring uses Hibernate as the default ORM framework but another framework is needed becaused Hibernate is not compatible with the Apache 2 license.  For this reason Metron uses <a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html#boot-features-embedded-database-support">EclipseLink</a>.  See the <a class="externalLink" href="https://github.com/spring-projects/spring-data-examples/tree/master/jpa/eclipselink">Spring Data JPA - EclipseLink</a> project for an example on how to configure EclipseLink in Spring.</p>
 <div class="section">
 <h3><a name="Development"></a>Development</h3>
 <p>The REST application comes with <a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html#boot-features-embedded-database-support">embedded database support</a> for development purposes.</p>
 <p>For example, edit these variables in <tt>/etc/default/metron</tt> before starting the application to configure H2:</p>
 
-<div class="source">
-<div class="source">
-<pre>METRON_JDBC_DRIVER=&quot;org.h2.Driver&quot;
+<div>
+<div>
+<pre class="source">METRON_JDBC_DRIVER=&quot;org.h2.Driver&quot;
 METRON_JDBC_URL=&quot;jdbc:h2:file:~/metrondb&quot;
 METRON_JDBC_USERNAME=&quot;root&quot;
 METRON_JDBC_PLATFORM=&quot;h2&quot;
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Production"></a>Production</h3>
 <p>The REST application should be configured with a production-grade database outside of development.</p>
@@ -503,2062 +299,1695 @@ METRON_JDBC_PLATFORM=&quot;h2&quot;
 <div class="section">
 <h4><a name="Manual_Install"></a>Manual Install</h4>
 <p>The following configures the application for MySQL:</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Install MySQL if not already available (this example uses version 5.7, installation instructions can be found <a class="externalLink" href="https://dev.mysql.com/doc/refman/5.7/en/linux-installation-yum-repo.html">here</a>)</p></li>
-  
+
+<p>Install MySQL if not already available (this example uses version 5.7, installation instructions can be found <a class="externalLink" href="https://dev.mysql.com/doc/refman/5.7/en/linux-installation-yum-repo.html">here</a>)</p>
+</li>
 <li>
+
 <p>Create a metron user and REST database and permission the user for that database:</p>
-  
-<div class="source">
-<div class="source">
-<pre>CREATE USER 'metron'@'node1' IDENTIFIED BY 'Myp@ssw0rd';
+
+<div>
+<div>
+<pre class="source">CREATE USER 'metron'@'node1' IDENTIFIED BY 'Myp@ssw0rd';
 CREATE DATABASE IF NOT EXISTS metronrest;
 GRANT ALL PRIVILEGES ON metronrest.* TO 'metron'@'node1';
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
+<li>
+
+<p>Create the security tables as described in the <a class="externalLink" href="https://docs.spring.io/spring-security/site/docs/5.0.4.RELEASE/reference/htmlsingle/#user-schema">Spring Security Guide</a>.</p>
+</li>
 <li>
+
 <p>Install the MySQL JDBC client onto the REST application host and configurate the METRON_JDBC_CLIENT_PATH variable:</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd $METRON_HOME/lib
+
+<div>
+<div>
+<pre class="source">cd $METRON_HOME/lib
 wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.41.tar.gz
 tar xf mysql-connector-java-5.1.41.tar.gz
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Edit these variables in <tt>/etc/default/metron</tt> to configure the REST application for MySQL:</p>
-  
-<div class="source">
-<div class="source">
-<pre>METRON_JDBC_DRIVER=&quot;com.mysql.jdbc.Driver&quot;
+
+<div>
+<div>
+<pre class="source">METRON_JDBC_DRIVER=&quot;com.mysql.jdbc.Driver&quot;
 METRON_JDBC_URL=&quot;jdbc:mysql://mysql_host:3306/metronrest&quot;
 METRON_JDBC_USERNAME=&quot;metron&quot;
 METRON_JDBC_PLATFORM=&quot;mysql&quot;
 METRON_JDBC_CLIENT_PATH=$METRON_HOME/lib/mysql-connector-java-5.1.41/mysql-connector-java-5.1.41-bin.jar
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Switch to the metron user</p>
-  
-<div class="source">
-<div class="source">
-<pre>sudo su - metron
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">sudo su - metron
+</pre></div></div>
+</li>
 <li>
+
 <p>Start the REST API. Adjust the password as necessary.</p>
-  
-<div class="source">
-<div class="source">
-<pre>set -o allexport;
+
+<div>
+<div>
+<pre class="source">set -o allexport;
 source /etc/default/metron;
 set +o allexport;
 export METRON_JDBC_PASSWORD='Myp@ssw0rd';
 $METRON_HOME/bin/metron-rest.sh
 unset METRON_JDBC_PASSWORD;
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div></div></div>
 <div class="section">
 <h2><a name="Usage"></a>Usage</h2>
-<p>The REST application can be accessed with the Swagger UI at <a class="externalLink" href="http://host:port/swagger-ui.html#/">http://host:port/swagger-ui.html#/</a>. The default port is 8082.</p></div>
+<p>The REST application can be accessed with the Swagger UI at <a class="externalLink" href="http://host:port/swagger-ui.html#/">http://host:port/swagger-ui.html#/</a>.  The default port is 8082.</p></div>
 <div class="section">
 <h2><a name="Security"></a>Security</h2>
 <div class="section">
 <h3><a name="Authentication"></a>Authentication</h3>
-<p>The metron-rest module uses <a class="externalLink" href="http://projects.spring.io/spring-security/">Spring Security</a> for authentication and stores user credentials in the relational database configured above. The required tables are created automatically the first time the application is started so that should be done first. For example (continuing the MySQL example above), users can be added by connecting to MySQL and running:</p>
+<p>The metron-rest module uses <a class="externalLink" href="http://projects.spring.io/spring-security/">Spring Security</a> for authentication and stores user credentials in the relational database configured above.  The required tables are created automatically the first time the application is started so that should be done first.  For example (continuing the MySQL example above), users can be added by connecting to MySQL and running:</p>
 
-<div class="source">
-<div class="source">
-<pre>use metronrest;
+<div>
+<div>
+<pre class="source">use metronrest;
 insert into users (username, password, enabled) values ('your_username','your_password',1);
 insert into authorities (username, authority) values ('your_username', 'ROLE_USER');
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Kerberos"></a>Kerberos</h3>
-<p>Metron REST can be configured for a cluster with Kerberos enabled. A client JAAS file is required for Kafka and Zookeeper and a Kerberos keytab for the metron user principal is required for all other services. Configure these settings in the <tt>/etc/default/metron</tt> file:</p>
+<p>Metron REST can be configured for a cluster with Kerberos enabled.  A client JAAS file is required for Kafka and Zookeeper and a Kerberos keytab for the metron user principal is required for all other services.  Configure these settings in the <tt>/etc/default/metron</tt> file:</p>
 
-<div class="source">
-<div class="source">
-<pre>SECURITY_ENABLED=true
+<div>
+<div>
+<pre class="source">SECURITY_ENABLED=true
 METRON_JVMFLAGS=&quot;-Djava.security.auth.login.config=$METRON_HOME/client_jaas.conf&quot;
 METRON_PRINCIPAL_NAME=&quot;metron@EXAMPLE.COM&quot;
 METRON_SERVICE_KEYTAB=&quot;/etc/security/keytabs/metron.keytab&quot;
-</pre></div></div></div></div>
+</pre></div></div>
+</div></div>
 <div class="section">
 <h2><a name="Spring_Profiles"></a>Spring Profiles</h2>
 <p>The REST application comes with a few <a class="externalLink" href="http://docs.spring.io/autorepo/docs/spring-boot/current/reference/html/boot-features-profiles.html">Spring Profiles</a> to aid in testing and development.</p>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Profile </th>
-      
-<th>Description </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Profile                  </th>
+<th> Description                                   </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>test </td>
-      
-<td>sets variables to in-memory services, only used for integration testing </td>
-    </tr>
-    
+<td> test                     </td>
+<td> adds test users <tt>[user, user1, user2, admin]</tt> to the database with password &#x201c;<tt>password</tt>&#x201d;. sets variables to in-memory services, only used for integration testing </td></tr>
 <tr class="a">
-      
-<td>dev </td>
-      
-<td>adds a test user to the database with credentials <tt>user/password</tt> </td>
-    </tr>
-    
+<td> dev                      </td>
+<td> adds test users <tt>[user, user1, user2, admin]</tt> to the database with password &#x201c;<tt>password</tt>&#x201d; </td></tr>
 <tr class="b">
-      
-<td>vagrant </td>
-      
-<td>sets configuration variables to match the Metron vagrant environment </td>
-    </tr>
-    
+<td> vagrant                  </td>
+<td> sets configuration variables to match the Metron vagrant environment </td></tr>
 <tr class="a">
-      
-<td>docker </td>
-      
-<td>sets configuration variables to match the Metron docker environment </td>
-    </tr>
-  </tbody>
+<td> docker                   </td>
+<td> sets configuration variables to match the Metron docker environment </td></tr>
+</tbody>
 </table>
-<p>Setting active profiles is done with the METRON_SPRING_PROFILES_ACTIVE variable. For example, set this variable in <tt>/etc/default/metron</tt> to configure the REST application for the Vagrant environment and add a test user:</p>
+<p>Setting active profiles is done with the METRON_SPRING_PROFILES_ACTIVE variable.  For example, set this variable in <tt>/etc/default/metron</tt> to configure the REST application for the Vagrant environment and add a test user:</p>
 
-<div class="source">
-<div class="source">
-<pre>METRON_SPRING_PROFILES_ACTIVE=&quot;vagrant,dev&quot;
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">METRON_SPRING_PROFILES_ACTIVE=&quot;vagrant,dev&quot;
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="API"></a>API</h2>
-<p>Request and Response objects are JSON formatted. The JSON schemas are available in the Swagger UI.</p>
-
+<p>Request and Response objects are JSON formatted.  The JSON schemas are available in the Swagger UI.</p>
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
+<tr class="a">
+<th>            </th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td> <a href="#get-apiv1alertsuiescalate"> <tt>POST /api/v1/alerts/ui/escalate</tt></a></td></tr>
 <tr class="a">
-      
-<th> </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<td> <a href="#GET_apiv1alertsuisettings"> <tt>GET /api/v1/alerts/ui/settings</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#get-apiv1alertescalate"> <tt>POST /api/v1/alert/escalate</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1alertsuisettingsall"> <tt>GET /api/v1/alerts/ui/settings/all</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1alertprofile"> <tt>GET /api/v1/alert/profile</tt></a></td>
-    </tr>
-    
+<td> <a href="#DELETE_apiv1alertsuisettings"> <tt>DELETE /api/v1/alerts/ui/settings</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1alertprofileall"> <tt>GET /api/v1/alert/profile/all</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1alertsuisettings"> <tt>POST /api/v1/alerts/ui/settings</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#DELETE_apiv1alertprofile"> <tt>DELETE /api/v1/alert/profile</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1globalconfig"> <tt>GET /api/v1/global/config</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#POST_apiv1alertprofile"> <tt>POST /api/v1/alert/profile</tt></a></td>
-    </tr>
-    
+<td> <a href="#DELETE_apiv1globalconfig"> <tt>DELETE /api/v1/global/config</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1globalconfig"> <tt>GET /api/v1/global/config</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1globalconfig"> <tt>POST /api/v1/global/config</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#DELETE_apiv1globalconfig"> <tt>DELETE /api/v1/global/config</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1grokgetstatement"> <tt>GET /api/v1/grok/get/statement</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#POST_apiv1globalconfig"> <tt>POST /api/v1/global/config</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1groklist"> <tt>GET /api/v1/grok/list</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1grokgetstatement"> <tt>GET /api/v1/grok/get/statement</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1grokvalidate"> <tt>POST /api/v1/grok/validate</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1groklist"> <tt>GET /api/v1/grok/list</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1hdfs"> <tt>POST /api/v1/hdfs</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#POST_apiv1grokvalidate"> <tt>POST /api/v1/grok/validate</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1hdfs"> <tt>GET /api/v1/hdfs</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#POST_apiv1hdfs"> <tt>POST /api/v1/hdfs</tt></a></td>
-    </tr>
-    
+<td> <a href="#DELETE_apiv1hdfs"> <tt>DELETE /api/v1/hdfs</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1hdfs"> <tt>GET /api/v1/hdfs</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1hdfslist"> <tt>GET /api/v1/hdfs/list</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#DELETE_apiv1hdfs"> <tt>DELETE /api/v1/hdfs</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1kafkatopic"> <tt>GET /api/v1/kafka/topic</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1hdfslist"> <tt>GET /api/v1/hdfs/list</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1kafkatopic"> <tt>POST /api/v1/kafka/topic</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1kafkatopic"> <tt>GET /api/v1/kafka/topic</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1kafkatopicname"> <tt>GET /api/v1/kafka/topic/{name}</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#POST_apiv1kafkatopic"> <tt>POST /api/v1/kafka/topic</tt></a></td>
-    </tr>
-    
+<td> <a href="#DELETE_apiv1kafkatopicname"> <tt>DELETE /api/v1/kafka/topic/{name}</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1kafkatopicname"> <tt>GET /api/v1/kafka/topic/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1kafkatopicnamesample"> <tt>GET /api/v1/kafka/topic/{name}/sample</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#DELETE_apiv1kafkatopicname"> <tt>DELETE /api/v1/kafka/topic/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1kafkatopicnameproduce"> <tt>POST /api/v1/kafka/topic/{name}/produce</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1kafkatopicnamesample"> <tt>GET /api/v1/kafka/topic/{name}/sample</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1metaalertsearchByAlert"> <tt>GET /api/v1/metaalert/searchByAlert</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1metaalertsearchByAlert"> <tt>GET /api/v1/metaalert/searchByAlert</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1metaalertcreate"> <tt>GET /api/v1/metaalert/create</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1metaalertcreate"> <tt>GET /api/v1/metaalert/create</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1metaalertaddalert"> <tt>GET /api/v1/metaalert/add/alert</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1metaalertaddalert"> <tt>GET /api/v1/metaalert/add/alert</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1metaalertremovealert"> <tt>GET /api/v1/metaalert/remove/alert</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1metaalertremovealert"> <tt>GET /api/v1/metaalert/remove/alert</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1metaalertupdatestatusguidstatus"> <tt>GET /api/v1/metaalert/update/status/{guid}/{status}</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1metaalertupdatestatusguidstatus"> <tt>GET /api/v1/metaalert/update/status/{guid}/{status}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1searchsearch"> <tt>GET /api/v1/search/search</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1searchsearch"> <tt>GET /api/v1/search/search</tt></a></td>
-    </tr>
-    
+<td> <a href="#get-apiv1searchsearch"> <tt>POST /api/v1/search/search</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#get-apiv1searchsearch"> <tt>POST /api/v1/search/search</tt></a></td>
-    </tr>
-    
+<td> <a href="#get-apiv1searchgroup"> <tt>POST /api/v1/search/group</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#get-apiv1searchgroup"> <tt>POST /api/v1/search/group</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1searchfindOne"> <tt>GET /api/v1/search/findOne</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1searchfindOne"> <tt>GET /api/v1/search/findOne</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1searchcolumnmetadata"> <tt>GET /api/v1/search/column/metadata</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1searchcolumnmetadata"> <tt>GET /api/v1/search/column/metadata</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1sensorenrichmentconfig"> <tt>GET /api/v1/sensor/enrichment/config</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1sensorenrichmentconfig"> <tt>GET /api/v1/sensor/enrichment/config</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1sensorenrichmentconfiglistavailableenrichments"> <tt>GET /api/v1/sensor/enrichment/config/list/available/enrichments</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1sensorenrichmentconfiglistavailableenrichments"> <tt>GET /api/v1/sensor/enrichment/config/list/available/enrichments</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1sensorenrichmentconfiglistavailablethreattriageaggregators"> <tt>GET /api/v1/sensor/enrichment/config/list/available/threat/triage/aggregators</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1sensorenrichmentconfiglistavailablethreattriageaggregators"> <tt>GET /api/v1/sensor/enrichment/config/list/available/threat/triage/aggregators</tt></a></td>
-    </tr>
-    
+<td> <a href="#DELETE_apiv1sensorenrichmentconfigname"> <tt>DELETE /api/v1/sensor/enrichment/config/{name}</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#DELETE_apiv1sensorenrichmentconfigname"> <tt>DELETE /api/v1/sensor/enrichment/config/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1sensorenrichmentconfigname"> <tt>POST /api/v1/sensor/enrichment/config/{name}</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#POST_apiv1sensorenrichmentconfigname"> <tt>POST /api/v1/sensor/enrichment/config/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1sensorenrichmentconfigname"> <tt>GET /api/v1/sensor/enrichment/config/{name}</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1sensorenrichmentconfigname"> <tt>GET /api/v1/sensor/enrichment/config/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1sensorindexingconfig"> <tt>GET /api/v1/sensor/indexing/config</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1sensorindexingconfig"> <tt>GET /api/v1/sensor/indexing/config</tt></a></td>
-    </tr>
-    
+<td> <a href="#DELETE_apiv1sensorindexingconfigname"> <tt>DELETE /api/v1/sensor/indexing/config/{name}</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#DELETE_apiv1sensorindexingconfigname"> <tt>DELETE /api/v1/sensor/indexing/config/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1sensorindexingconfigname"> <tt>POST /api/v1/sensor/indexing/config/{name}</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#POST_apiv1sensorindexingconfigname"> <tt>POST /api/v1/sensor/indexing/config/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1sensorindexingconfigname"> <tt>GET /api/v1/sensor/indexing/config/{name}</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1sensorindexingconfigname"> <tt>GET /api/v1/sensor/indexing/config/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1sensorparserconfig"> <tt>POST /api/v1/sensor/parser/config</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#POST_apiv1sensorparserconfig"> <tt>POST /api/v1/sensor/parser/config</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1sensorparserconfig"> <tt>GET /api/v1/sensor/parser/config</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1sensorparserconfig"> <tt>GET /api/v1/sensor/parser/config</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1sensorparserconfiglistavailable"> <tt>GET /api/v1/sensor/parser/config/list/available</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1sensorparserconfiglistavailable"> <tt>GET /api/v1/sensor/parser/config/list/available</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1sensorparserconfigparseMessage"> <tt>POST /api/v1/sensor/parser/config/parseMessage</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#POST_apiv1sensorparserconfigparseMessage"> <tt>POST /api/v1/sensor/parser/config/parseMessage</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1sensorparserconfigreloadavailable"> <tt>GET /api/v1/sensor/parser/config/reload/available</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1sensorparserconfigreloadavailable"> <tt>GET /api/v1/sensor/parser/config/reload/available</tt></a></td>
-    </tr>
-    
+<td> <a href="#DELETE_apiv1sensorparserconfigname"> <tt>DELETE /api/v1/sensor/parser/config/{name}</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#DELETE_apiv1sensorparserconfigname"> <tt>DELETE /api/v1/sensor/parser/config/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1sensorparserconfigname"> <tt>GET /api/v1/sensor/parser/config/{name}</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1sensorparserconfigname"> <tt>GET /api/v1/sensor/parser/config/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1stellarapplytransformations"> <tt>POST /api/v1/stellar/apply/transformations</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#POST_apiv1stellarapplytransformations"> <tt>POST /api/v1/stellar/apply/transformations</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stellarlist"> <tt>GET /api/v1/stellar/list</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1stellarlist"> <tt>GET /api/v1/stellar/list</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stellarlistfunctions"> <tt>GET /api/v1/stellar/list/functions</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1stellarlistfunctions"> <tt>GET /api/v1/stellar/list/functions</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stellarlistsimplefunctions"> <tt>GET /api/v1/stellar/list/simple/functions</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1stellarlistsimplefunctions"> <tt>GET /api/v1/stellar/list/simple/functions</tt></a></td>
-    </tr>
-    
+<td> <a href="#POST_apiv1stellarvalidaterules"> <tt>POST /api/v1/stellar/validate/rules</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#POST_apiv1stellarvalidaterules"> <tt>POST /api/v1/stellar/validate/rules</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1storm"> <tt>GET /api/v1/storm</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1storm"> <tt>GET /api/v1/storm</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormclientstatus"> <tt>GET /api/v1/storm/client/status</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1stormclientstatus"> <tt>GET /api/v1/storm/client/status</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormenrichment"> <tt>GET /api/v1/storm/enrichment</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1stormenrichment"> <tt>GET /api/v1/storm/enrichment</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormenrichmentactivate"> <tt>GET /api/v1/storm/enrichment/activate</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1stormenrichmentactivate"> <tt>GET /api/v1/storm/enrichment/activate</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormenrichmentdeactivate"> <tt>GET /api/v1/storm/enrichment/deactivate</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1stormenrichmentdeactivate"> <tt>GET /api/v1/storm/enrichment/deactivate</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormenrichmentstart"> <tt>GET /api/v1/storm/enrichment/start</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1stormenrichmentstart"> <tt>GET /api/v1/storm/enrichment/start</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormenrichmentstop"> <tt>GET /api/v1/storm/enrichment/stop</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1stormenrichmentstop"> <tt>GET /api/v1/storm/enrichment/stop</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormindexingbatch"> <tt>GET /api/v1/storm/indexing/batch</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1stormindexing"> <tt>GET /api/v1/storm/indexing</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormindexingbatchactivate"> <tt>GET /api/v1/storm/indexing/batch/activate</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1stormindexingactivate"> <tt>GET /api/v1/storm/indexing/activate</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormindexingbatchdeactivate"> <tt>GET /api/v1/storm/indexing/batch/deactivate</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1stormindexingdeactivate"> <tt>GET /api/v1/storm/indexing/deactivate</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormindexingbatchstart"> <tt>GET /api/v1/storm/indexing/batch/start</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1stormindexingstart"> <tt>GET /api/v1/storm/indexing/start</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormindexingbatchstop"> <tt>GET /api/v1/storm/indexing/batch/stop</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1stormindexingstop"> <tt>GET /api/v1/storm/indexing/stop</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormindexingrandomaccess"> <tt>GET /api/v1/storm/indexing/randomaccess</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1stormparseractivatename"> <tt>GET /api/v1/storm/parser/activate/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormindexingrandomaccessactivate"> <tt>GET /api/v1/storm/indexing/randomaccess/activate</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1stormparserdeactivatename"> <tt>GET /api/v1/storm/parser/deactivate/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormindexingrandomaccessdeactivate"> <tt>GET /api/v1/storm/indexing/randomaccess/deactivate</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1stormparserstartname"> <tt>GET /api/v1/storm/parser/start/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormindexingrandomaccessstart"> <tt>GET /api/v1/storm/indexing/randomaccess/start</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1stormparserstopname"> <tt>GET /api/v1/storm/parser/stop/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormindexingrandomaccessstop"> <tt>GET /api/v1/storm/indexing/randomaccess/stop</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1stormname"> <tt>GET /api/v1/storm/{name}</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormparseractivatename"> <tt>GET /api/v1/storm/parser/activate/{name}</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#GET_apiv1stormsupervisors"> <tt>GET /api/v1/storm/supervisors</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormparserdeactivatename"> <tt>GET /api/v1/storm/parser/deactivate/{name}</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#PATCH_apiv1updatepatch"> <tt>PATCH /api/v1/update/patch</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormparserstartname"> <tt>GET /api/v1/storm/parser/start/{name}</tt></a></td></tr>
 <tr class="a">
-      
-<td><a href="#patch-apiv1updatereplace"> <tt>PUT /api/v1/update/replace</tt></a></td>
-    </tr>
-    
+<td> <a href="#GET_apiv1stormparserstopname"> <tt>GET /api/v1/storm/parser/stop/{name}</tt></a></td></tr>
 <tr class="b">
-      
-<td><a href="#GET_apiv1user"> <tt>GET /api/v1/user</tt></a></td>
-    </tr>
-  </tbody>
+<td> <a href="#GET_apiv1stormname"> <tt>GET /api/v1/storm/{name}</tt></a></td></tr>
+<tr class="a">
+<td> <a href="#GET_apiv1stormsupervisors"> <tt>GET /api/v1/storm/supervisors</tt></a></td></tr>
+<tr class="b">
+<td> <a href="#PATCH_apiv1updatepatch"> <tt>PATCH /api/v1/update/patch</tt></a></td></tr>
+<tr class="a">
+<td> <a href="#patch-apiv1updatereplace"> <tt>PUT /api/v1/update/replace</tt></a></td></tr>
+<tr class="b">
+<td> <a href="#GET_apiv1user"> <tt>GET /api/v1/user</tt></a></td></tr>
+</tbody>
 </table>
 <div class="section">
-<h3><a name="POST_apiv1alertescalate"></a><tt>POST /api/v1/alert/escalate</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Falerts.2Fui.2Fescalate"></a><tt>POST /api/v1/alerts/ui/escalate</tt></h3>
 <ul>
-  
+
 <li>Description: Escalates a list of alerts by producing it to the Kafka escalate topic</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>alerts - The alerts to be escalated</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Alerts were escalated</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1alertprofile"></a><tt>GET /api/v1/alert/profile</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Falerts.2Fui.2Fsettings"></a><tt>GET /api/v1/alerts/ui/settings</tt></h3>
 <ul>
-  
-<li>Description: Retrieves the current user&#x2019;s alerts profile</li>
-  
+
+<li>Description: Retrieves the current user&#x2019;s settings</li>
 <li>Returns:
-  
 <ul>
-    
-<li>200 - Alerts profile</li>
-    
-<li>404 - The current user does not have an alerts profile</li>
-  </ul></li>
+
+<li>200 - User settings</li>
+<li>404 - he current user does not have settings</li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1alertprofileall"></a><tt>GET /api/v1/alert/profile/all</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Falerts.2Fui.2Fsettings.2Fall"></a><tt>GET /api/v1/alerts/ui/settings/all</tt></h3>
 <ul>
-  
-<li>Description: Retrieves all users&#x2019; alerts profiles. Only users that are part of the &#x201c;ROLE_ADMIN&#x201d; role are allowed to get all alerts profiles.</li>
-  
+
+<li>Description: Retrieves all users&#x2019; settings.  Only users that are part of the &#x201c;ROLE_ADMIN&#x201d; role are allowed to get all user settings.</li>
 <li>Returns:
-  
 <ul>
-    
-<li>200 - List of all alerts profiles</li>
-    
-<li>403 - The current user does not have permission to get all alerts profiles</li>
-  </ul></li>
+
+<li>200 - List of all user settings</li>
+<li>403 - The current user does not have permission to get all user settings</li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="DELETE_apiv1alertprofile"></a><tt>DELETE /api/v1/alert/profile</tt></h3>
-
+<h3><a name="DELETE_.2Fapi.2Fv1.2Falerts.2Fui.2Fsettings"></a><tt>DELETE /api/v1/alerts/ui/settings</tt></h3>
 <ul>
-  
-<li>Description: Deletes a user&#x2019;s alerts profile. Only users that are part of the &#x201c;ROLE_ADMIN&#x201d; role are allowed to delete user alerts profiles.</li>
-  
+
+<li>Description: Deletes a user&#x2019;s settings.  Only users that are part of the &#x201c;ROLE_ADMIN&#x201d; role are allowed to delete user settings.</li>
 <li>Input:
-  
 <ul>
-    
-<li>user - The user whose prolife will be deleted</li>
-  </ul></li>
-  
+
+<li>user - The user whose settings will be deleted</li>
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
-<li>200 - Alerts profile was deleted</li>
-    
-<li>403 - The current user does not have permission to delete alerts profiles</li>
-    
-<li>404 - Alerts profile could not be found</li>
-  </ul></li>
+
+<li>200 - User settings were deleted</li>
+<li>403 - The current user does not have permission to delete user settings</li>
+<li>404 - User settings could not be found</li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1alertprofile"></a><tt>POST /api/v1/alert/profile</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Falerts.2Fui.2Fsettings"></a><tt>POST /api/v1/alerts/ui/settings</tt></h3>
 <ul>
-  
-<li>Description: Creates or updates the current user&#x2019;s alerts profile</li>
-  
+
+<li>Description: Creates or updates the current user&#x2019;s settings</li>
 <li>Input:
-  
 <ul>
-    
-<li>alertsProfile - The alerts profile to be saved</li>
-  </ul></li>
-  
+
+<li>alertsUIUserSettings - The user settings to be saved</li>
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
-<li>200 - Alerts profile updated. Returns saved alerts profile.</li>
-    
-<li>201 - Alerts profile created. Returns saved alerts profile.</li>
-  </ul></li>
+
+<li>200 - User settings updated. Returns saved settings.</li>
+<li>201 - User settings created. Returns saved settings.</li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1globalconfig"></a><tt>GET /api/v1/global/config</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fglobal.2Fconfig"></a><tt>GET /api/v1/global/config</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves the current Global Config from Zookeeper</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns current Global Config JSON in Zookeeper</li>
-    
 <li>404 - Global Config JSON was not found in Zookeeper</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="DELETE_apiv1globalconfig"></a><tt>DELETE /api/v1/global/config</tt></h3>
-
+<h3><a name="DELETE_.2Fapi.2Fv1.2Fglobal.2Fconfig"></a><tt>DELETE /api/v1/global/config</tt></h3>
 <ul>
-  
+
 <li>Description: Deletes the current Global Config from Zookeeper</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Global Config JSON was deleted</li>
-    
 <li>404 - Global Config JSON was not found in Zookeeper</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1globalconfig"></a><tt>POST /api/v1/global/config</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fglobal.2Fconfig"></a><tt>POST /api/v1/global/config</tt></h3>
 <ul>
-  
+
 <li>Description: Creates or updates the Global Config in Zookeeper</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>globalConfig - The Global Config JSON to be saved</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Global Config updated. Returns saved Global Config JSON</li>
-    
 <li>201 - Global Config created. Returns saved Global Config JSON</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1grokgetstatement"></a><tt>GET /api/v1/grok/get/statement</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fgrok.2Fget.2Fstatement"></a><tt>GET /api/v1/grok/get/statement</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves a Grok statement from the classpath</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>path - Path to classpath resource</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Grok statement</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1groklist"></a><tt>GET /api/v1/grok/list</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fgrok.2Flist"></a><tt>GET /api/v1/grok/list</tt></h3>
 <ul>
-  
+
 <li>Description: Lists the common Grok statements available in Metron</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - JSON object containing pattern label/Grok statements key value pairs</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1grokvalidate"></a><tt>POST /api/v1/grok/validate</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fgrok.2Fvalidate"></a><tt>POST /api/v1/grok/validate</tt></h3>
 <ul>
-  
+
 <li>Description: Applies a Grok statement to a sample message</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>grokValidation - Object containing Grok statement and sample message</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - JSON results</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1hdfs"></a><tt>POST /api/v1/hdfs</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fhdfs"></a><tt>POST /api/v1/hdfs</tt></h3>
 <ul>
-  
-<li>Description: Writes contents to an HDFS file. Warning: this will overwrite the contents of a file if it already exists. Permissions must be set for all three groups if they are to be set. If any are missing, the default permissions will be used, and if any are invalid an exception will be thrown.</li>
-  
+
+<li>Description: Writes contents to an HDFS file.  Warning: this will overwrite the contents of a file if it already exists. Permissions must be set for all three groups if they are to be set. If any are missing, the default permissions will be used, and if any are invalid an exception will be thrown.</li>
 <li>Input:
-  
 <ul>
-    
+
 <li>path - Path to HDFS file</li>
-    
 <li>contents - File contents</li>
-    
 <li>userMode - [optional] symbolic permission string for user portion of the permissions to be set on the file written. For example &#x2018;rwx&#x2019; or read, write, execute. The symbol &#x2018;-&#x2019; is used to exclude that permission such as &#x2018;rw-&#x2019; for read, write, no execute</li>
-    
 <li>groupMode - [optional] symbolic permission string for group portion of the permissions to be set on the file written. For example &#x2018;rwx&#x2019; or read, write, execute. The symbol &#x2018;-&#x2019; is used to exclude that permission such as &#x2018;rw-&#x2019; for read, write, no execute</li>
-    
 <li>otherMode - [optional] symbolic permission string for other portion of the permissions to be set on the file written. For example &#x2018;rwx&#x2019; or read, write, execute. The symbol &#x2018;-&#x2019; is used to exclude that permission such as &#x2018;rw-&#x2019; for read, write, no execute</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Contents were written</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1hdfs"></a><tt>GET /api/v1/hdfs</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fhdfs"></a><tt>GET /api/v1/hdfs</tt></h3>
 <ul>
-  
+
 <li>Description: Reads a file from HDFS and returns the contents</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>path - Path to HDFS file</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns file contents</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="DELETE_apiv1hdfs"></a><tt>DELETE /api/v1/hdfs</tt></h3>
-
+<h3><a name="DELETE_.2Fapi.2Fv1.2Fhdfs"></a><tt>DELETE /api/v1/hdfs</tt></h3>
 <ul>
-  
+
 <li>Description: Deletes a file from HDFS</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>path - Path to HDFS file</li>
-    
 <li>recursive - Delete files recursively</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - File was deleted</li>
-    
 <li>404 - File was not found in HDFS</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1hdfslist"></a><tt>GET /api/v1/hdfs/list</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fhdfs.2Flist"></a><tt>GET /api/v1/hdfs/list</tt></h3>
 <ul>
-  
+
 <li>Description: Lists an HDFS directory</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>path - Path to HDFS directory</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - HDFS directory list</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1kafkatopic"></a><tt>GET /api/v1/kafka/topic</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fkafka.2Ftopic"></a><tt>GET /api/v1/kafka/topic</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves all Kafka topics</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns a list of all Kafka topics</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1kafkatopic"></a><tt>POST /api/v1/kafka/topic</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fkafka.2Ftopic"></a><tt>POST /api/v1/kafka/topic</tt></h3>
 <ul>
-  
+
 <li>Description: Creates a new Kafka topic</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>topic - Kafka topic</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns saved Kafka topic</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1kafkatopicname"></a><tt>GET /api/v1/kafka/topic/{name}</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fkafka.2Ftopic.2F.7Bname.7D"></a><tt>GET /api/v1/kafka/topic/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves a Kafka topic</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>name - Kafka topic name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns Kafka topic</li>
-    
 <li>404 - Kafka topic is missing</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="DELETE_apiv1kafkatopicname"></a><tt>DELETE /api/v1/kafka/topic/{name}</tt></h3>
-
+<h3><a name="DELETE_.2Fapi.2Fv1.2Fkafka.2Ftopic.2F.7Bname.7D"></a><tt>DELETE /api/v1/kafka/topic/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Deletes a Kafka topic</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>name - Kafka topic name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Kafka topic was deleted</li>
-    
 <li>404 - Kafka topic is missing</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1kafkatopicnamesample"></a><tt>GET /api/v1/kafka/topic/{name}/sample</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fkafka.2Ftopic.2F.7Bname.7D.2Fsample"></a><tt>GET /api/v1/kafka/topic/{name}/sample</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves a sample message from a Kafka topic using the most recent offset</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>name - Kafka topic name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns sample message</li>
-    
 <li>404 - Either Kafka topic is missing or contains no messages</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1metaalertsearchByAlert"></a><tt>POST /api/v1/metaalert/searchByAlert</tt></h3>
+<h3><a name="POST_.2Fapi.2Fv1.2Fkafka.2Ftopic.2F.7Bname.7D.2Fproduce"></a><tt>POST /api/v1/kafka/topic/{name}/produce</tt></h3>
+<ul>
+
+<li>Description: Produces a message to a Kafka topic</li>
+<li>Input:
+<ul>
+
+<li>name - Kafka topic name</li>
+<li>message - message to be published</li>
+</ul>
+</li>
+<li>Returns:
+<ul>
 
+<li>200 - Published</li>
+<li>404 - Kafka topic is missing</li>
+</ul>
+</li>
+</ul></div>
+<div class="section">
+<h3><a name="POST_.2Fapi.2Fv1.2Fmetaalert.2FsearchByAlert"></a><tt>POST /api/v1/metaalert/searchByAlert</tt></h3>
 <ul>
-  
+
 <li>Description: Get all meta alerts that contain an alert.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>guid - GUID of the alert</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Search results</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1metaalertcreate"></a><tt>POST /api/v1/metaalert/create</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fmetaalert.2Fcreate"></a><tt>POST /api/v1/metaalert/create</tt></h3>
 <ul>
-  
-<li>Description: Creates a new meta alert from a list of existing alerts. The meta alert status will initially be set to &#x2018;ACTIVE&#x2019; and summary statistics will be computed from the list of alerts. A list of groups included in the request are also added to the meta alert.</li>
-  
+
+<li>Description: Creates a new meta alert from a list of existing alerts.  The meta alert status will initially be set to &#x2018;ACTIVE&#x2019; and summary statistics will be computed from the list of alerts.  A list of groups included in the request are also added to the meta alert.</li>
 <li>Input:
-  
 <ul>
-    
+
 <li>request - Meta alert create request which includes a list of alert get requests and a list of custom groups used to annotate a meta alert.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - The GUID of the new meta alert</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1metaalertaddalert"></a><tt>POST /api/v1/metaalert/add/alert</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fmetaalert.2Fadd.2Falert"></a><tt>POST /api/v1/metaalert/add/alert</tt></h3>
 <ul>
-  
-<li>Description: Adds an alert to an existing meta alert. An alert will not be added if it is already contained in a meta alert.</li>
-  
+
+<li>Description: Adds an alert to an existing meta alert.  An alert will not be added if it is already contained in a meta alert.</li>
 <li>Input:
-  
 <ul>
-    
+
 <li>request - Meta alert add request which includes a meta alert GUID and list of alert get requests</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns &#x2018;true&#x2019; if the alert was added and &#x2018;false&#x2019; if the meta alert did not change.</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1metaalertremovealert"></a><tt>POST /api/v1/metaalert/remove/alert</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fmetaalert.2Fremove.2Falert"></a><tt>POST /api/v1/metaalert/remove/alert</tt></h3>
 <ul>
-  
-<li>Description: Removes an alert from an existing meta alert. If the alert to be removed is not in a meta alert, &#x2018;false&#x2019; will be returned.</li>
-  
+
+<li>Description: Removes an alert from an existing meta alert.  If the alert to be removed is not in a meta alert, &#x2018;false&#x2019; will be returned.</li>
 <li>Input:
-  
 <ul>
-    
+
 <li>request - Meta alert remove request which includes a meta alert GUID and list of alert get requests</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns &#x2018;true&#x2019; if the alert was removed and &#x2018;false&#x2019; if the meta alert did not change.</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1metaalertupdatestatusguidstatus"></a><tt>POST /api/v1/metaalert/update/status/{guid}/{status}</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fmetaalert.2Fupdate.2Fstatus.2F.7Bguid.7D.2F.7Bstatus.7D"></a><tt>POST /api/v1/metaalert/update/status/{guid}/{status}</tt></h3>
 <ul>
-  
+
 <li>Description: Updates the status of a meta alert to either &#x2018;ACTIVE&#x2019; or &#x2018;INACTIVE&#x2019;.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>guid - Meta alert GUID</li>
-    
 <li>status - Meta alert status with a value of either &#x2018;ACTIVE&#x2019; or &#x2018;INACTIVE&#x2019;</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns &#x2018;true&#x2019; if the status changed and &#x2018;false&#x2019; if it did not.</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1searchsearch"></a><tt>POST /api/v1/search/search</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fsearch.2Fsearch"></a><tt>POST /api/v1/search/search</tt></h3>
 <ul>
-  
+
 <li>Description: Searches the indexing store. GUIDs must be quoted to ensure correct results.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>searchRequest - Search request</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Search response</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1searchgroup"></a><tt>POST /api/v1/search/group</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fsearch.2Fgroup"></a><tt>POST /api/v1/search/group</tt></h3>
 <ul>
-  
-<li>Description: Searches the indexing store and returns field groups. GUIDs must be quoted to ensure correct results. Groups are hierarchical and nested in the order the fields appear in the &#x2018;groups&#x2019; request parameter. The default sorting within groups is by count descending. A groupOrder type of count will sort based on then number of documents in a group while a groupType of term will sort by the groupBy term.</li>
-  
+
+<li>Description: Searches the indexing store and returns field groups. GUIDs must be quoted to ensure correct results. Groups are hierarchical and nested in the order the fields appear in the &#x2018;groups&#x2019; request parameter. The default sorting within groups is by count descending.  A groupOrder type of count will sort based on then number of documents in a group while a groupType of term will sort by the groupBy term.</li>
 <li>Input:
-  
 <ul>
-    
+
 <li>groupRequest - Group request
-    
 <ul>
-      
+
 <li>indices - list of indices to search</li>
-      
 <li>query - lucene query</li>
-      
 <li>scoreField - field used to compute a total score for each group</li>
-      
 <li>groups - List of groups (field name and sort order)</li>
-    </ul></li>
-  </ul></li>
-  
+</ul>
+</li>
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Group response</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1searchfindOne"></a><tt>GET /api/v1/search/findOne</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsearch.2FfindOne"></a><tt>GET /api/v1/search/findOne</tt></h3>
 <ul>
-  
+
 <li>Description: Returns latest document for a guid and sensor</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>getRequest - Get request
-    
 <ul>
-      
+
 <li>guid - message UUID</li>
-      
 <li>sensorType - Sensor Type</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Example: Return <tt>bro</tt> document with UUID of <tt>000-000-0000</tt></li>
-  </ul></li>
+</ul>
+</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;guid&quot; : &quot;000-000-0000&quot;,
   &quot;sensorType&quot; : &quot;bro&quot;
 }
 </pre></div></div>
 
 <ul>
-  
+
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Document representing the output</li>
-    
 <li>404 - Document with UUID and sensor type not found</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1searchcolumnmetadata"></a><tt>GET /api/v1/search/column/metadata</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsearch.2Fcolumn.2Fmetadata"></a><tt>GET /api/v1/search/column/metadata</tt></h3>
 <ul>
-  
-<li>Description: Get index column metadata for a list of sensor types with duplicates removed. Column names and types for each sensor are retrieved from the most recent index. Columns that exist in multiple indices with different types will default to type &#x2018;other&#x2019;.</li>
-  
+
+<li>Description: Get index column metadata for a list of sensor types with duplicates removed.  Column names and types for each sensor are retrieved from the most recent index.  Columns that exist in multiple indices with different types will default to type &#x2018;other&#x2019;.</li>
 <li>Input:
-  
 <ul>
-    
+
 <li>sensorTypes - Sensor Types</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Column Metadata</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1sensorenrichmentconfig"></a><tt>GET /api/v1/sensor/enrichment/config</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig"></a><tt>GET /api/v1/sensor/enrichment/config</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves all SensorEnrichmentConfigs from Zookeeper</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns all SensorEnrichmentConfigs</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1sensorenrichmentconfiglistavailableenrichments"></a><tt>GET /api/v1/sensor/enrichment/config/list/available/enrichments</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig.2Flist.2Favailable.2Fenrichments"></a><tt>GET /api/v1/sensor/enrichment/config/list/available/enrichments</tt></h3>
 <ul>
-  
+
 <li>Description: Lists the available enrichments</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns a list of available enrichments</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1sensorenrichmentconfiglistavailablethreattriageaggregators"></a><tt>GET /api/v1/sensor/enrichment/config/list/available/threat/triage/aggregators</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig.2Flist.2Favailable.2Fthreat.2Ftriage.2Faggregators"></a><tt>GET /api/v1/sensor/enrichment/config/list/available/threat/triage/aggregators</tt></h3>
 <ul>
-  
+
 <li>Description: Lists the available threat triage aggregators</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns a list of available threat triage aggregators</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="DELETE_apiv1sensorenrichmentconfigname"></a><tt>DELETE /api/v1/sensor/enrichment/config/{name}</tt></h3>
-
+<h3><a name="DELETE_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig.2F.7Bname.7D"></a><tt>DELETE /api/v1/sensor/enrichment/config/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Deletes a SensorEnrichmentConfig from Zookeeper</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>name - SensorEnrichmentConfig name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - SensorEnrichmentConfig was deleted</li>
-    
 <li>404 - SensorEnrichmentConfig is missing</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1sensorenrichmentconfigname"></a><tt>POST /api/v1/sensor/enrichment/config/{name}</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig.2F.7Bname.7D"></a><tt>POST /api/v1/sensor/enrichment/config/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Updates or creates a SensorEnrichmentConfig in Zookeeper</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>sensorEnrichmentConfig - SensorEnrichmentConfig</li>
-    
 <li>name - SensorEnrichmentConfig name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - SensorEnrichmentConfig updated. Returns saved SensorEnrichmentConfig</li>
-    
 <li>201 - SensorEnrichmentConfig created. Returns saved SensorEnrichmentConfig</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1sensorenrichmentconfigname"></a><tt>GET /api/v1/sensor/enrichment/config/{name}</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig.2F.7Bname.7D"></a><tt>GET /api/v1/sensor/enrichment/config/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves a SensorEnrichmentConfig from Zookeeper</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>name - SensorEnrichmentConfig name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns SensorEnrichmentConfig</li>
-    
 <li>404 - SensorEnrichmentConfig is missing</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1sensorindexingconfig"></a><tt>GET /api/v1/sensor/indexing/config</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Findexing.2Fconfig"></a><tt>GET /api/v1/sensor/indexing/config</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves all SensorIndexingConfigs from Zookeeper</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns all SensorIndexingConfigs</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="DELETE_apiv1sensorindexingconfigname"></a><tt>DELETE /api/v1/sensor/indexing/config/{name}</tt></h3>
-
+<h3><a name="DELETE_.2Fapi.2Fv1.2Fsensor.2Findexing.2Fconfig.2F.7Bname.7D"></a><tt>DELETE /api/v1/sensor/indexing/config/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Deletes a SensorIndexingConfig from Zookeeper</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>name - SensorIndexingConfig name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - SensorIndexingConfig was deleted</li>
-    
 <li>404 - SensorIndexingConfig is missing</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1sensorindexingconfigname"></a><tt>POST /api/v1/sensor/indexing/config/{name}</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fsensor.2Findexing.2Fconfig.2F.7Bname.7D"></a><tt>POST /api/v1/sensor/indexing/config/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Updates or creates a SensorIndexingConfig in Zookeeper</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>sensorIndexingConfig - SensorIndexingConfig</li>
-    
 <li>name - SensorIndexingConfig name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - SensorIndexingConfig updated. Returns saved SensorIndexingConfig</li>
-    
 <li>201 - SensorIndexingConfig created. Returns saved SensorIndexingConfig</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1sensorindexingconfigname"></a><tt>GET /api/v1/sensor/indexing/config/{name}</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Findexing.2Fconfig.2F.7Bname.7D"></a><tt>GET /api/v1/sensor/indexing/config/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves a SensorIndexingConfig from Zookeeper</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>name - SensorIndexingConfig name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns SensorIndexingConfig</li>
-    
 <li>404 - SensorIndexingConfig is missing</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1sensorparserconfig"></a><tt>POST /api/v1/sensor/parser/config</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2F.7Bname.7D"></a><tt>POST /api/v1/sensor/parser/config/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Updates or creates a SensorParserConfig in Zookeeper</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>sensorParserConfig - SensorParserConfig</li>
-  </ul></li>
-  
+<li>name - SensorEnrichmentConfig name</li>
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - SensorParserConfig updated. Returns saved SensorParserConfig</li>
-    
 <li>201 - SensorParserConfig created. Returns saved SensorParserConfig</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1sensorparserconfig"></a><tt>GET /api/v1/sensor/parser/config</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig"></a><tt>GET /api/v1/sensor/parser/config</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves all SensorParserConfigs from Zookeeper</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns all SensorParserConfigs</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1sensorparserconfiglistavailable"></a><tt>GET /api/v1/sensor/parser/config/list/available</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2Flist.2Favailable"></a><tt>GET /api/v1/sensor/parser/config/list/available</tt></h3>
 <ul>
-  
+
 <li>Description: Lists the available parser classes that can be found on the classpath</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns a list of available parser classes</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1sensorparserconfigparseMessage"></a><tt>POST /api/v1/sensor/parser/config/parseMessage</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2FparseMessage"></a><tt>POST /api/v1/sensor/parser/config/parseMessage</tt></h3>
 <ul>
-  
+
 <li>Description: Parses a sample message given a SensorParserConfig</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>parseMessageRequest - Object containing a sample message and SensorParserConfig</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns parsed message</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1sensorparserconfigreloadavailable"></a><tt>GET /api/v1/sensor/parser/config/reload/available</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2Freload.2Favailable"></a><tt>GET /api/v1/sensor/parser/config/reload/available</tt></h3>
 <ul>
-  
+
 <li>Description: Scans the classpath for available parser classes and reloads the cached parser class list</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns a list of available parser classes</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="DELETE_apiv1sensorparserconfigname"></a><tt>DELETE /api/v1/sensor/parser/config/{name}</tt></h3>
-
+<h3><a name="DELETE_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2F.7Bname.7D"></a><tt>DELETE /api/v1/sensor/parser/config/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Deletes a SensorParserConfig from Zookeeper</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>name - SensorParserConfig name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - SensorParserConfig was deleted</li>
-    
 <li>404 - SensorParserConfig is missing</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1sensorparserconfigname"></a><tt>GET /api/v1/sensor/parser/config/{name}</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2F.7Bname.7D"></a><tt>GET /api/v1/sensor/parser/config/{name}</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves a SensorParserConfig from Zookeeper</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>name - SensorParserConfig name</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns SensorParserConfig</li>
-    
 <li>404 - SensorParserConfig is missing</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1stellarapplytransformations"></a><tt>POST /api/v1/stellar/apply/transformations</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fstellar.2Fapply.2Ftransformations"></a><tt>POST /api/v1/stellar/apply/transformations</tt></h3>
 <ul>
-  
+
 <li>Description: Executes transformations against a sample message</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>transformationValidation - Object containing SensorParserConfig and sample message</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns transformation results</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1stellarlist"></a><tt>GET /api/v1/stellar/list</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fstellar.2Flist"></a><tt>GET /api/v1/stellar/list</tt></h3>
 <ul>
-  
+
 <li>Description: Retrieves field transformations</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns a list field transformations</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1stellarlistfunctions"></a><tt>GET /api/v1/stellar/list/functions</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fstellar.2Flist.2Ffunctions"></a><tt>GET /api/v1/stellar/list/functions</tt></h3>
 <ul>
-  
+
 <li>Description: Lists the Stellar functions that can be found on the classpath</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns a list of Stellar functions</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="GET_apiv1stellarlistsimplefunctions"></a><tt>GET /api/v1/stellar/list/simple/functions</tt></h3>
-
+<h3><a name="GET_.2Fapi.2Fv1.2Fstellar.2Flist.2Fsimple.2Ffunctions"></a><tt>GET /api/v1/stellar/list/simple/functions</tt></h3>
 <ul>
-  
+
 <li>Description: Lists the simple Stellar functions (functions with only 1 input) that can be found on the classpath</li>
-  
 <li>Returns:
-  
 <ul>
-    
+
 <li>200 - Returns a list of simple Stellar functions</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
-<h3><a name="POST_apiv1stellarvalidaterules"></a><tt>POST /api/v1/stellar/validate/rules</tt></h3>
-
+<h3><a name="POST_.2Fapi.2Fv1.2Fstellar.2Fvalidate.2Frules"></a><tt>POST /api/v1/stellar/validate/rules</tt>

<TRUNCATED>

[27/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/packaging/ambari/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/packaging/ambari/index.html b/site/current-book/metron-deployment/packaging/ambari/index.html
index 8013381..bac5758 100644
--- a/site/current-book/metron-deployment/packaging/ambari/index.html
+++ b/site/current-book/metron-deployment/packaging/ambari/index.html
@@ -1,505 +1,331 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/ambari/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Ambari Management Pack Development</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../../css/site.css" />
     <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Ambari Management Pack Development</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Ambari Management Pack Development</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Ambari</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Ambari</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/packaging/ambari/elasticsearch-mpack/index.html" title="Elasticsearch-mpack"><span class="none"></span>Elasticsearch-mpack</a></li>
+    <li><a href="../../../metron-deployment/packaging/ambari/metron-mpack/index.html" title="Mpack"><span class="none"></span>Mpack</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Ambari Management Pack Development</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Ambari Management Pack Development</h1>
 <p><a name="Ambari_Management_Pack_Development"></a></p>
 <div class="section">
 <h2><a name="Table_of_Contents"></a>Table of Contents</h2>
-
 <ul>
-  
+
 <li><a href="#Overview">Overview</a></li>
-  
 <li><a href="#Ambari_Zookeeper_Config_Lifecycle">Ambari Zookeeper Config Lifecycle</a></li>
-  
 <li><a href="#Adding_a_new_property">Adding a new property</a></li>
-  
 <li><a href="#How_to_identify_errors_in_MPack_changes">How to identify errors in MPack changes</a></li>
-  
 <li><a href="#Testing_changes_without_cycling_Vagrant_build">Testing changes without cycling Vagrant build</a></li>
-  
 <li><a href="#Configuration_involving_dependency_services">Configuration involving dependency services</a></li>
-  
 <li><a href="#Kerberos">Kerberos</a></li>
-  
 <li><a href="#Best_practices">Best practices</a></li>
+<li><a href="#Upgrading_MPack_Services">Upgrading MPack Services</a></li>
 </ul></div>
 <div class="section">
 <h2><a name="Overview"></a>Overview</h2>
 <p>Typically, Ambari Management Pack development will be done in the Vagrant environments. These instructions are specific to Vagrant, but can be adapted for other environemnts (e.g. make sure to be on the correct nodes for server vs agent files)</p>
-<p>There is an <tt>mpack.json</tt> file which describes what services the mpack will contains, versions, etc.</p>
-<p>Alongside this are two directories, <tt>addon-services</tt> and <tt>common-services</tt>.</p>
-<p>The layout of <tt>/common-services/METRON.CURRENT</tt> is</p>
+<p>There are two MPacks:</p>
+<ul>
 
+<li>Metron - contains artifacts for deploying the Metron service</li>
+<li>Elasticsearch - contains artifacts for installing Elasticsearch and Kibana services</li>
+</ul>
+<p>There is an <tt>mpack.json</tt> file for each which describes what services the mpack will contain, versions, etc.</p>
+<p>Alongside this are two directories, <tt>addon-services</tt> and <tt>common-services</tt>. Below the Metron MPack is described, but this also applies similarly to the Elasticsearch MPack.</p>
+<p>The layout of <tt>/common-services/METRON/CURRENT</tt> is</p>
 <ul>
-  
+
 <li><tt>/configuration</tt>
-  
 <ul>
-    
+
 <li>This contains a set of <tt>*-env.xml</tt> files, relevent to particular components or the service as a whole. These are where properties are defined.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>/package</tt>
-  
 <ul>
-    
+
 <li><tt>/files</tt>
-    
 <ul>
-      
+
 <li>Contains files that get used as provided, in particular Elasticsearch templates.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li><tt>/scripts</tt>
-    
 <ul>
-      
+
 <li>A set of Python scripts that interface with Ambari to manage setup and install</li>
-      
 <li><tt>/params</tt>
-      
 <ul>
-        
+
 <li>A set of Python scripts for managing parameters from the <tt>*-env.xml</tt> files</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li><tt>/templates</tt>
-      
 <ul>
-        
+
 <li>A set of Jinja template files which can be populated with properties</li>
-      </ul></li>
-    </ul></li>
-  </ul></li>
-  
+</ul>
+</li>
+</ul>
+</li>
+</ul>
+</li>
 <li><tt>/quicklinks</tt>
-  
 <ul>
-    
+
 <li>Contains <tt>quicklinks.json</tt> to define the Ambari quicklinks that should be used in the UI</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>themes</tt>
-  
 <ul>
-    
+
 <li>Manages the Ambari UI themes blocks for organizing the configuration by relevant area.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>kerberos.json</tt>
-  
 <ul>
-    
+
 <li>Defines the keytabs and other Kerberos configuration to be used when Kerberizing a cluster</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>metainfo.xml</tt>
-  
 <ul>
-    
+
 <li>Defines the METRON service, along with required packages, services, etc.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>service_advisor.py</tt>
-  
 <ul>
-    
+
 <li>Handles component layout and validation, along with handling some configurations for other services or that needs configs from other services.</li>
-  </ul></li>
 </ul>
-<p>The layout of <tt>/addon-services/METRON.CURRENT</tt> is</p>
-
+</li>
+</ul>
+<p>The layout of <tt>/addon-services/METRON/CURRENT</tt> is</p>
 <ul>
-  
+
 <li><tt>/repos</tt>
-  
 <ul>
-    
+
 <li>Contains <tt>repoinfo.xml</tt> that defines repositories to install packages from</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>metainfo.xml</tt>
-  
 <ul>
-    
-<li>Limited info version of <tt>/common-services/METRON.CURRENT/metainfo.xml</tt></li>
-  </ul></li>
-  
+
+<li>Limited info version of <tt>/common-services/METRON/CURRENT/metainfo.xml</tt></li>
+</ul>
+</li>
 <li><tt>role_command_order.json</tt>
-  
 <ul>
-    
+
 <li>Defines the order of service startup and other actions relative to each other.</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h2><a name="Ambari_Zookeeper_Config_Lifecycle"></a>Ambari Zookeeper Config Lifecycle</h2>
 <p>The source of truth for Metron configuration resides in Zookeeper as a series of JSON documents. There are actually multiple locations that you can populate the Zookeeper configs from:</p>
-
 <ul>
-  
+
 <li>$METRON_HOME/config/zookeeper</li>
-  
 <li>Stellar REPL</li>
-  
 <li>Management UI</li>
-  
 <li>Ambari</li>
 </ul>
 <p>Any change that works with Metron configuration stored in Zookeeper should always pull the latest config from Zookeeper first before making any changes and publishing them back up to avoid potentially overwriting the latest configs with stale data. The lifecycle for configuration management with Ambari is as follows:</p>
-
 <ul>
-  
+
 <li>First start/install for parsers, enrichment, indexing, or the profiler
-  
 <ul>
-    
+
 <li>Perform a PUSH from local disk to Zookeeper</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>All component start/restart operations for parsers, enrichment, indexing, or the profiler
-  
 <ul>
-    
+
 <li>Perform a JSON patch (RFC 6902) on the configs in Zookeeper and push the patched, pretty formatted config up to Zookeeper. Patching is currently applicable to to global.json only.</li>
-    
 <li>Pull all configs to the local file system.</li>
-  </ul></li>
+</ul>
+</li>
 </ul>
 <p>The patching mechanism we introduced to Ambari for managing the global config is by default additive in nature, though it will also overwrite any existing values with the latest values from Ambari. This means that you can add any properties to the global config outside of Ambari you like, provided you are not modifying values explicitly managed by Ambari. This is important because you will not get any errors when modifying the configuration outside of Ambari. Instead, if you modified es.ip and changed global.json outsided Ambari, you would not see this change in Ambari. Meanwhile, the indexing topology would be using the new value stored in Zookeeper. Remember, Zookeeper is the source of truth. If you were to restart a Metron topology component via Ambari, that es.ip property would now be set back to the value stored in Ambari. So in summary, if it&#x2019;s managed by Ambari, only change that value via Ambari. If it&#x2019;s not, then feel free to use whatever mechanism you like,
  just be sure to pull the latest config from Zookeeper before making any changes. We make no effort to handle race conditions. It&#x2019;s last commit wins. Below are the global config properties managed by Ambari:</p>
-
 <ul>
-  
+
 <li>es.clustername</li>
-  
 <li>es.ip</li>
-  
 <li>es.date.format</li>
-  
 <li>parser.error.topic</li>
-  
 <li>update.hbase.table</li>
-  
 <li>update.hbase.cf</li>
-  
 <li>profiler.client.period.duration</li>
-  
 <li>profiler.client.period.duration.units</li>
 </ul></div>
 <div class="section">
 <h2><a name="Adding_a_new_property"></a>Adding a new property</h2>
-
 <ol style="list-style-type: decimal">
-  
-<li>
-<p>Add the property to the appropriate <tt>*-env.xml</tt> file found in <tt>METRON.CURRENT/configuration</tt>.</p>
-  
-<div class="source">
-<div class="source">
-<pre>&lt;property&gt;
+
+<li>Add the property to the appropriate <tt>*-env.xml</tt> file found in <tt>METRON.CURRENT/configuration</tt>.
+
+<div>
+<div>
+<pre class="source">&lt;property&gt;
     &lt;name&gt;new_property&lt;/name&gt;
     &lt;description&gt;New Property description&lt;/description&gt;
     &lt;value&gt;Default Value&lt;/value&gt;
     &lt;display-name&gt;New Property Pretty Name&lt;/display-name&gt;
 &lt;/property&gt;
 </pre></div></div>
-<p>The appropriate <tt>*-env.xml</tt> file should be selected based on which component depends on the property. This allows Ambari to accurately restart only the affected components when the property is changed. If a property is in <tt>metron-env.xml</tt>, Ambari will prompt you to restart all Metron components.</p></li>
-  
+</li>
+</ol>
+<p>The appropriate <tt>*-env.xml</tt> file should be selected based on which component depends on the property. This allows Ambari to accurately restart only the affected components when the property is changed. If a property is in <tt>metron-env.xml</tt>, Ambari will prompt you to restart all Metron components.</p>
+<ol style="list-style-type: decimal">
+
 <li>
-<p>Add the property to the <tt>metron_theme.json</tt> file found in <tt>METRON.CURRENT/themes</tt> if the property was added to a component-specific <tt>*-env.xml</tt> file (<tt>metron-parsers-env.xml</tt> for example) and not <tt>metron-env.xml</tt>. This is necessary for the property to be displayed in the correct tab of the Metron Configs section in the Ambari UI. Using other properties as a guide, add the property to <tt>/configuration/placement/configs</tt> for proper placement and also to <tt>/configuration/widgets</tt> for a specific widget type.</p></li>
-  
+
+<p>Add the property to the <tt>metron_theme.json</tt> file found in <tt>METRON.CURRENT/themes</tt> if the property was added to a component-specific <tt>*-env.xml</tt> file (<tt>metron-parsers-env.xml</tt> for example) and not <tt>metron-env.xml</tt>. This is necessary for the property to be displayed in the correct tab of the Metron Configs section in the Ambari UI. Using other properties as a guide, add the property to <tt>/configuration/placement/configs</tt> for proper placement and also to <tt>/configuration/widgets</tt> for a specific widget type.</p>
+</li>
 <li>
+
 <p>Reference the property in <tt>METRON.CURRENT/package/scriptes/params/params_linux.py</tt>, unless it will be used in Ambari&#x2019;s status command. It will be stored in a variable. The name doesn&#x2019;t have to match, but it&#x2019;s preferred that it does. Make sure to use replace <tt>metron-env</tt> the correct <tt>*-env</tt> file, as noted above.</p>
-  
-<div class="source">
-<div class="source">
-<pre>  new_property = config['configurations']['metron-env']['new_property']
+
+<div>
+<div>
+<pre class="source">new_property = config['configurations']['metron-env']['new_property']
 </pre></div></div>
-<p>If this property will be used in the status command, instead make this change in <tt>METRON.CURRENT/package/scriptes/params/status_params.py</tt>. Afterwards, in <tt>params_linux.py</tt>, reference the new property:</p></li>
-</ol>
 
-<div class="source">
-<div class="source">
-<pre>  new_property = status_params.new_property
+<p>If this property will be used in the status command, instead make this change in <tt>METRON.CURRENT/package/scriptes/params/status_params.py</tt>. Afterwards, in <tt>params_linux.py</tt>, reference the new property:</p>
+
+<div>
+<div>
+<pre class="source">new_property = status_params.new_property
 </pre></div></div>
-<p>This behavior is because Ambari doesn&#x2019;t send all parameters to the status, so it needs to be explicitly provided. Also note that status_params.py parameters are not automatically pulled into the params_linux.py namespace, so we explicitly choose the variables to include.  See <a class="externalLink" href="https://docs.python.org/2/howto/doanddont.html#at-module-level">https://docs.python.org/2/howto/doanddont.html#at-module-level</a> for more info.</p>
 
-<ol style="list-style-type: decimal">
-  
+<p>This behavior is because Ambari doesn&#x2019;t send all parameters to the status, so it needs to be explicitly provided. Also note that status_params.py parameters are not automatically pulled into the params_linux.py namespace, so we explicitly choose the variables to include. See <a class="externalLink" href="https://docs.python.org/2/howto/doanddont.html#at-module-level">https://docs.python.org/2/howto/doanddont.html#at-module-level</a> for more info.</p>
+</li>
 <li>
+
 <p>Ambari master services can then import the params:</p>
-  
-<div class="source">
-<div class="source">
-<pre>  from params import params
-  env.set_params(params)
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">from params import params
+env.set_params(params)
+</pre></div></div>
+</li>
 <li>
+
 <p>The <tt>*_commands.py</tt> files receive the params as an input from the master services. Once this is done, they can be accessed via the variable we set above:</p>
-  
-<div class="source">
-<div class="source">
-<pre>  self.__params.new_property
-</pre></div></div></li>
+
+<div>
+<div>
+<pre class="source">self.__params.new_property
+</pre></div></div>
+</li>
 </ol>
 <div class="section">
 <h3><a name="Env_file_property_walkthrough"></a>Env file property walkthrough</h3>
@@ -508,108 +334,113 @@
 <h4><a name="Defining_the_property"></a>Defining the property</h4>
 <p>First the property appears in the appropriate <tt>*-env.xml</tt>, in this case <tt>METRON.CURRENT/configuration/metron-env.xml</tt>.</p>
 
-<div class="source">
-<div class="source">
-<pre>&lt;property&gt;
+<div>
+<div>
+<pre class="source">&lt;property&gt;
     &lt;name&gt;metron_apps_hdfs_dir&lt;/name&gt;
     &lt;value&gt;/apps/metron&lt;/value&gt;
     &lt;description&gt;Metron apps HDFS dir&lt;/description&gt;
     &lt;display-name&gt;Metron apps HDFS dir&lt;/display-name&gt;
 &lt;/property&gt;
 </pre></div></div>
-<p>This defines several things</p>
 
+<p>This defines several things</p>
 <ol style="list-style-type: decimal">
-  
+
 <li>The name of the property we&#x2019;ll be referencing it by in our code.</li>
-  
 <li>The default value of the property.</li>
-  
 <li>The description of the property for the Ambari UI.</li>
-  
 <li>The pretty name that will be shown in Ambari for the property.</li>
 </ol></div>
 <div class="section">
 <h4><a name="Making_the_property_available_to_scripts"></a>Making the property available to scripts</h4>
 <p>Second, we set up the property to be available to the code. This happens in <tt>METRON.CURRENT/packages/scripts/params/params_linux.py</tt>. Just add the following line:</p>
 
-<div class="source">
-<div class="source">
-<pre>metron_apps_hdfs_dir = config['configurations']['metron-env']['metron_apps_hdfs_dir']
+<div>
+<div>
+<pre class="source">metron_apps_hdfs_dir = config['configurations']['metron-env']['metron_apps_hdfs_dir']
 </pre></div></div>
+
 <p>There is one catch to this. If we wanted this to be available to Ambari&#x2019;s status command, we&#x2019;d need to put that line in <tt>METRON.CURRENT/packages/scripts/params/status_params.py</tt> and reference it in <tt>params_linux.py</tt> like so:</p>
 
-<div class="source">
-<div class="source">
-<pre>metron_apps_hdfs_dir = status_params.metron_apps_hdfs_dir
+<div>
+<div>
+<pre class="source">metron_apps_hdfs_dir = status_params.metron_apps_hdfs_dir
 </pre></div></div>
+
 <p>This behavior is because Ambari doesn&#x2019;t send all parameters to the status, so it needs to be explicitly provided.</p>
 <p>In our case, we don&#x2019;t use this parameter directly (but it could be if we wanted to use it exactly). We actually append to it before use in <tt>params_linux.py</tt>:</p>
 
-<div class="source">
-<div class="source">
-<pre>from resource_management.libraries.functions import format
+<div>
+<div>
+<pre class="source">from resource_management.libraries.functions import format
 # ...
 hdfs_grok_patterns_dir = format(&quot;{metron_apps_hdfs_dir}/patterns&quot;)
 </pre></div></div>
+
 <p>The <tt>format</tt> function is a special Ambari function that will let you create a string with properties in curly braces replaced by their values. In this case, <tt>hdfs_grok_patterns_dir</tt> will be <tt>/apps/metron/patterns</tt> and will be what we now follow in this example.</p>
 <p>Note: There are instances where we deviate a bit from this pattern and use format(format(&#x201c;{{some_prop}}&#x201d;)). This is not a mistake. Ambari does not natively support nested properties, and this was a workaround that accomplished being able to initialize values from env.xml files and have them also available to users as Ambari properties later. Here&#x2019;s an example from params_linux.py:</p>
 
-<div class="source">
-<div class="source">
-<pre>metron_apps_indexed_hdfs_dir = format(format(config['configurations']['metron-indexing-env']['metron_apps_indexed_hdfs_dir']))
+<div>
+<div>
+<pre class="source">metron_apps_indexed_hdfs_dir = format(format(config['configurations']['metron-indexing-env']['metron_apps_indexed_hdfs_dir']))
 </pre></div></div>
+
 <p>and the relavant properties in metron-env.xml and metron-indexing-env.xml</p>
 <p>metron-env.xml</p>
 
-<div class="source">
-<div class="source">
-<pre>&lt;property&gt;
+<div>
+<div>
+<pre class="source">&lt;property&gt;
     &lt;name&gt;metron_apps_hdfs_dir&lt;/name&gt;
     &lt;value&gt;/apps/metron&lt;/value&gt;
     &lt;description&gt;Metron apps HDFS dir&lt;/description&gt;
     &lt;display-name&gt;Metron apps HDFS dir&lt;/display-name&gt;
 &lt;/property&gt;
 </pre></div></div>
+
 <p>metron-indexing-env.xml</p>
 
-<div class="source">
-<div class="source">
-<pre>&lt;property&gt;
+<div>
+<div>
+<pre class="source">&lt;property&gt;
     &lt;name&gt;metron_apps_indexed_hdfs_dir&lt;/name&gt;
     &lt;value&gt;{{metron_apps_hdfs_dir}}/indexing/indexed&lt;/value&gt;
     &lt;description&gt;Indexing bolts will write to this HDFS directory&lt;/description&gt;
     &lt;display-name&gt;Metron apps indexed HDFS dir&lt;/display-name&gt;
 &lt;/property&gt;
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Importing_the_parameters_into_scripts"></a>Importing the parameters into scripts</h4>
 <p><tt>hdfs_grok_patterns_dir</tt> is used in <tt>METRON.CURRENT/package/scripts/parser_commands.py</tt>, but before we can reference it, we&#x2019;ll need the params available to <tt>parser_commands.py</tt></p>
 <p>To make them available, we take them in as part of the <tt>__init__</tt></p>
 
-<div class="source">
-<div class="source">
-<pre>def __init__(self, params):
+<div>
+<div>
+<pre class="source">def __init__(self, params):
     if params is None:
         raise ValueError(&quot;params argument is required for initialization&quot;)
     self.__params = params
     # Other initialization
 </pre></div></div>
+
 <p>This init is called from various Ambari service methods in <tt>METRON.CURRENT/package/scripts/parser_master.py</tt>, e.g.:</p>
 
-<div class="source">
-<div class="source">
-<pre>def stop(self, env, upgrade_type=None):
+<div>
+<div>
+<pre class="source">def stop(self, env, upgrade_type=None):
     from params import params
     env.set_params(params)
     commands = ParserCommands(params)
     commands.stop_parser_topologies()
 </pre></div></div>
+
 <p>Once the params are available to <tt>parser_commands.py</tt>, <tt>hdfs_grok_patterns_dir</tt> is by referencing <tt>self.__params.hdfs_grok_patterns_dir)</tt> In our case, this will create and populate <tt>/apps/metron/patterns</tt> on HDFS, owned by the metron user with appropriate permissions. It&#x2019;ll also log out what it&#x2019;s doing, which is important for being able to debug.</p>
 
-<div class="source">
-<div class="source">
-<pre>def init_parsers(self):
+<div>
+<div>
+<pre class="source">def init_parsers(self):
     Logger.info(
         &quot;Copying grok patterns from local directory '{0}' to HDFS '{1}'&quot;.format(self.__params.local_grok_patterns_dir,
                                                                                 self.__params.hdfs_grok_patterns_dir))
@@ -622,7 +453,8 @@ hdfs_grok_patterns_dir = format(&quot;{metron_apps_hdfs_dir}/patterns&quot;)
                                source=self.__params.local_grok_patterns_dir)
 
     Logger.info(&quot;Done initializing parser configuration&quot;)
-</pre></div></div></div></div>
+</pre></div></div>
+</div></div>
 <div class="section">
 <h3><a name="Jinja_Templates_and_properties"></a>Jinja Templates and properties</h3>
 <p>Jinja templates allow for the ability to have most of a file defined, but allow variables to be filled in from the properties defined in our <tt>*-env.xml</tt> files.</p>
@@ -636,36 +468,39 @@ hdfs_grok_patterns_dir = format(&quot;{metron_apps_hdfs_dir}/patterns&quot;)
 <h4><a name="Defining_the_property"></a>Defining the property</h4>
 <p>First, we need the property in the configuration file:</p>
 
-<div class="source">
-<div class="source">
-<pre>&lt;property&gt;
+<div>
+<div>
+<pre class="source">&lt;property&gt;
     &lt;name&gt;metron_log_dir&lt;/name&gt;
     &lt;value&gt;/var/log/metron&lt;/value&gt;
     &lt;description&gt;Log directory for metron&lt;/description&gt;
     &lt;display-name&gt;Metron log dir&lt;/display-name&gt;
 &lt;/property&gt;
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Making_the_property_available_to_templates"></a>Making the property available to templates</h4>
 <p>This property isn&#x2019;t used in Ambari&#x2019;s status check, so it was directly added to <tt>METRON.CURRENT/package/scripts/params/params_linux.py</tt>. All we do is add the variable, and reference Ambari&#x2019;s config object appropriately, making sure to reference <tt>metron-env</tt> as the file where the property is located.</p>
 
-<div class="source">
-<div class="source">
-<pre>metron_log_dir = config['configurations']['metron-env']['metron_log_dir']
+<div>
+<div>
+<pre class="source">metron_log_dir = config['configurations']['metron-env']['metron_log_dir']
 </pre></div></div>
+
 <p>The property is referenced in <tt>metron.j2</tt>.</p>
 
-<div class="source">
-<div class="source">
-<pre>METRON_LOG_DIR=&quot;{{metron_log_dir}}&quot;
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">METRON_LOG_DIR=&quot;{{metron_log_dir}}&quot;
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Using_the_template_in_scripts"></a>Using the template in scripts</h4>
 <p>For that property to actually be used, it is referenced in <tt>rest_master.py</tt>:</p>
 
-<div class="source">
-<div class="source">
-<pre>from resource_management.core.resources.system import File
+<div>
+<div>
+<pre class="source">from resource_management.core.resources.system import File
 from resource_management.core.source import Template
 
 def configure(self, env, upgrade_type=None, config_dir=None):
@@ -677,21 +512,23 @@ def configure(self, env, upgrade_type=None, config_dir=None):
          )
     # Do stuff
 </pre></div></div>
+
 <p>This will create a file on the Ambari agent machine&#x2019;s file system, <tt>/etc/sysconfig/metron</tt>, with the content of <tt>metron.j2</tt>, after replacing <tt>{{metron_log_dir}}</tt> with the value of the property (<tt>/var/log/metron</tt>)</p>
 
-<div class="source">
-<div class="source">
-<pre>...
+<div>
+<div>
+<pre class="source">...
 METRON_LOG_DIR=&quot;/var/log/metron&quot;
 ...
-</pre></div></div></div></div>
+</pre></div></div>
+</div></div>
 <div class="section">
 <h3><a name="Defining_presentation_in_the_Ambari_UI"></a>Defining presentation in the Ambari UI</h3>
 <p>Where and how a property is displayed can be controlled in the <tt>METRON.CURRENT/themes/metron_theme.json</tt> file. Consider the <tt>enrichment_workers</tt> property that is defined in a component specific <tt>*-env.xml</tt> file, in this case <tt>METRON.CURRENT/configuration/metron-enrichment-env.xml</tt>. The property appears in <tt>METRON.CURRENT/themes/metron_theme.json</tt> in two different sections:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;configuration&quot;: {
     &quot;layouts&quot;: [...],
     &quot;placement&quot;: {
@@ -714,21 +551,20 @@ METRON_LOG_DIR=&quot;/var/log/metron&quot;
   }
 }
 </pre></div></div>
+
 <p>The first setting places the property in the &#x201c;Storm&#x201d; section of the &#x201c;Enrichment&#x201d; tab in Ambari. Sections are defined in <tt>metron_theme.json</tt> under <tt>/configuration/layouts</tt>.</p>
 <p>The second setting defines a widget type of <tt>text-field</tt>. See the <a class="externalLink" href="https://cwiki.apache.org/confluence/display/AMBARI/Enhanced+Configs">Ambari Wiki</a> for more detail on widget types.</p>
 <p>If a property is defined in <tt>metron-env.xml</tt>, it is not necessary to add it to the <tt>metron_theme.json</tt> file. By default the property will be located under the &#x201c;Advanced&#x201d; tab in the &#x201c;Advanced metron-env&#x201d; section.</p></div></div>
 <div class="section">
 <h2><a name="How_to_identify_errors_in_MPack_changes"></a>How to identify errors in MPack changes</h2>
 <p>Typically, any errors are thrown at one of two times:</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>Attempting to install Metron as a whole. These are typically service level definition errors, not property errors. Logs will often be found in <tt>/var/log/ambari-server/ambari-server.log</tt>. Often the log will indicate it was unable to load a particular service or subservice with some more detail.</li>
-  
 <li>Running the actual functionality. These typically tend to cause errors in the UI at runtime. Logs are usually found in <tt>/var/log/ambari-agent/ambari-agent.log</tt>.</li>
 </ol>
 <p>Unfortunately, because errors tend to occur at runtime, it&#x2019;s often necessary to add things like logging statements, or even just throw errors to print out in the Ambari UI.</p>
-<p>The primary solution to these is to look in the logs for exceptions, see what&#x2019;s going wrong (Property doesn&#x2019;t exist? Malformed file couldn&#x2019;t be loaded?), and adjust appropriately.</p></div>
+<p>The primary solution to these is to look in the logs for exceptions, see what&#x2019;s going wrong (Property doesn&#x2019;t exist?  Malformed file couldn&#x2019;t be loaded?), and adjust appropriately.</p></div>
 <div class="section">
 <h2><a name="Testing_changes_without_cycling_Vagrant_build"></a>Testing changes without cycling Vagrant build</h2>
 <p>There are techniques we can use to avoid spinning down and spinning back up Vagrant.</p>
@@ -738,73 +574,73 @@ METRON_LOG_DIR=&quot;/var/log/metron&quot;
 <p>Ambari stores the Python files from the service in a couple places. We&#x2019;ll want to update the files, then have Ambari pick up the updated versions and use them as the new basis. A reinstall of Metron is unnecessary for this type of testing.</p>
 <p>Specifically, the server files live in</p>
 
-<div class="source">
-<div class="source">
-<pre>/var/lib/ambari-server/resources/mpacks/metron-ambari.mpack-0.4.0.0/common-services
+<div>
+<div>
+<pre class="source">/var/lib/ambari-server/resources/mpacks/metron-ambari.mpack-0.4.0.0/common-services
+/var/lib/ambari-server/resources/mpacks/elasticsearch-ambari.mpack-0.4.0.0/common-services
 /var/lib/ambari-agent/cache/common-services
 </pre></div></div>
+
 <p>e.g. enrichment_commands.py can be found in:</p>
 
-<div class="source">
-<div class="source">
-<pre>/var/lib/ambari-server/resources/mpacks/metron-ambari.mpack-0.4.0.0/common-services/METRON/0.4.0/package/scripts/enrichment_commands.py
+<div>
+<div>
+<pre class="source">/var/lib/ambari-server/resources/mpacks/metron-ambari.mpack-0.4.0.0/common-services/METRON/0.4.0/package/scripts/enrichment_commands.py
 /var/lib/ambari-agent/cache/common-services/METRON/0.4.0/package/scripts/enrichment_commands.py
 </pre></div></div>
+
 <p>A <tt>find</tt> command can also be useful in quickly locating the exact location of a file, e.g.</p>
 
-<div class="source">
-<div class="source">
-<pre>[root@node1 ~]# find /var/lib/ -name enrichment_commands.py
+<div>
+<div>
+<pre class="source">[root@node1 ~]# find /var/lib/ -name enrichment_commands.py
 /var/lib/ambari-server/resources/mpacks/metron-ambari.mpack-0.4.0.0/common-services/METRON/0.4.0/package/scripts/enrichment_commands.py
 /var/lib/ambari-agent/cache/common-services/METRON/0.4.0/package/scripts/enrichment_commands.py
 </pre></div></div>
-<p>The steps to update, for anything affecting an Ambari agent node, e.g. setup scripts during the service install process, are:</p>
 
+<p>The steps to update, for anything affecting an Ambari agent node, e.g. setup scripts during the service install process, are:</p>
 <ol style="list-style-type: decimal">
-  
+
 <li>Stop Metron through Ambari. If the property in question is used by the topologies, we&#x2019;ll want them stopped, so they can be restarted with the new property. This can sometimes be skipped if the change doesn&#x2019;t affect the topologies themselves, but is a case by case choice.</li>
-  
 <li>Edit the file(s) with your changes. The ambari-agent file must be edited, but generally better to update both for consistency.</li>
-  
-<li>
-<p>Restart the Ambari Agent to get the cache to pick up the modified file</p>
-  
-<div class="source">
-<div class="source">
-<pre>  ambari-agent restart
-</pre></div></div></li>
-  
+<li>Restart the Ambari Agent to get the cache to pick up the modified file
+
+<div>
+<div>
+<pre class="source">ambari-agent restart
+</pre></div></div>
+</li>
 <li>Start Metron through Ambari if it was stopped.</li>
 </ol></div>
 <div class="section">
 <h3><a name="Reinstalling_the_mpack"></a>Reinstalling the mpack</h3>
 <p>After we&#x2019;ve modified files in Ambari and the mpack is working, it is a good idea to reinstall it. Fortunately this can be done without rebuilding the Vagrant environment by following these steps:</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>Stop Metron through Ambari and remove the Metron service</li>
-  
-<li>
-<p>Rebuild the mpack on your local machine and deploy it to Vagrant, ensuring that all changes made directly to files in Ambari were also made in your local environment</p>
-  
-<div class="source">
-<div class="source">
-<pre>  cd metron-deployment
-  mvn clean package
-  scp packaging/ambari/metron-mpack/target/metron_mpack-0.4.0.0.tar.gz root@node1:~
-</pre></div></div></li>
-  
-<li>
-<p>Log in to Vagrant, deploy the mpack and restart Ambari</p>
-  
-<div class="source">
-<div class="source">
-<pre>  ssh root@node1
-  ambari-server install-mpack --mpack=metron_mpack-0.4.0.0.tar.gz --verbose --force
-  ambari-server restart
-</pre></div></div></li>
-  
+<li>Rebuild the mpack on your local machine and deploy it to Vagrant, ensuring that all changes made directly to files in Ambari were also made in your local environment
+
+<div>
+<div>
+<pre class="source">cd metron-deployment
+mvn clean package
+scp packaging/ambari/metron-mpack/target/metron_mpack-0.4.0.0.tar.gz root@node1:~
+</pre></div></div>
+</li>
+<li>Log in to Vagrant, deploy the mpack and restart Ambari
+
+<div>
+<div>
+<pre class="source">ssh root@node1
+ambari-server install-mpack --mpack=metron_mpack-0.4.0.0.tar.gz --verbose --force
+ambari-server restart
+</pre></div></div>
+</li>
 <li>Install the mpack through Ambari as you normally would</li>
+<li>
+
+<p>The same steps can be followed for Elasticsearch and Kibana by similary deploying the ES MPack located in elasticsearch-mpack/target.</p>
+</li>
 </ol></div></div>
 <div class="section">
 <h2><a name="Configuration_involving_dependency_services"></a>Configuration involving dependency services</h2>
@@ -812,88 +648,350 @@ METRON_LOG_DIR=&quot;/var/log/metron&quot;
 <p>The value is defined in a map in <tt>getSTORMSiteDesiredValues(self, is_secured)</tt>. This map is used by <tt>validateSTORMSiteConfigurations</tt> to ensure that the value used by the component is what we need it to be. Generally, only the map should need to be modified.</p>
 <p>Properties from the other services can also be used and examined. We use this to build the appropriate URL for Storm, because it differs based on Kerberos security. For example, to retrieve from another service, access the configurations array, retrieve the appropriate config file, retrieve the properties, and finally the desired property.</p>
 
-<div class="source">
-<div class="source">
-<pre>stormUIServerPort = services[&quot;configurations&quot;][&quot;storm-site&quot;][&quot;properties&quot;][&quot;ui.port&quot;]
+<div>
+<div>
+<pre class="source">stormUIServerPort = services[&quot;configurations&quot;][&quot;storm-site&quot;][&quot;properties&quot;][&quot;ui.port&quot;]
 </pre></div></div>
+
 <p>The security of the cluster can be checked with</p>
 
-<div class="source">
-<div class="source">
-<pre>is_secured = self.isSecurityEnabled(services)
+<div>
+<div>
+<pre class="source">is_secured = self.isSecurityEnabled(services)
 </pre></div></div>
+
 <p>Note that the configuration expectations will be properly enforced if the cluster is upgraded (e.g. the Storm URL is properly updated) and it does not need to occur manually.</p></div>
 <div class="section">
 <h2><a name="Kerberos"></a>Kerberos</h2>
 <p>Any scripts interacting with underlying secured tech needs to have <tt>kinit</tt> run as needed. This includes anything interacting with HDFS, HBase, Kafka, Storm, etc. This <tt>kinit</tt> should be run in a conditional statement, to avoid breaking non-Kerberized clusters.</p>
 <p>Ambari can run <tt>kinit</tt>, as a given user with the appropriate keytab like so:</p>
 
-<div class="source">
-<div class="source">
-<pre>if self.__params.security_enabled:
+<div>
+<div>
+<pre class="source">if self.__params.security_enabled:
     metron_security.kinit(self.__params.kinit_path_local,
                           self.__params.metron_keytab_path,
                           self.__params.metron_principal_name,
                           execute_user=self.__params.metron_user)
 </pre></div></div>
+
 <p>The <tt>security_enabled</tt> param is already made available, along with appropriate keytabs for metron, hdfs, kafka, and hbase users.</p></div>
 <div class="section">
 <h2><a name="Best_practices"></a>Best practices</h2>
-
 <ul>
-  
+
 <li>
+
 <p>Write scripts to be idempotent. The pattern currently used is to write a file out when a task is finished, e.g. setting up ACLs or tables. For example, when indexing is configured, a file is written out and checked based on a property.</p>
-  
-<div class="source">
-<div class="source">
-<pre>  def set_configured(self):
-  File(self.__params.indexing_configured_flag_file,
-       content=&quot;&quot;,
-       owner=self.__params.metron_user,
-       mode=0755)
-</pre></div></div>
-<p>This is checked in the indexing master</p></li>
-</ul>
 
-<div class="source">
-<div class="source">
-<pre>  if not commands.is_configured():
-      commands.init_kafka_topics()
-      commands.init_hdfs_dir()
-      commands.set_configured()
+<div>
+<div>
+<pre class="source">def set_configured(self):
+    File(self.__params.indexing_configured_flag_file,
+         content=&quot;&quot;,
+         owner=self.__params.metron_user,
+         mode=0755)
 </pre></div></div>
 
+<p>This is checked in the indexing master</p>
+
+<div>
+<div>
+<pre class="source">if not commands.is_configured():
+    commands.init_kafka_topics()
+    commands.init_hdfs_dir()
+    commands.set_configured()
+</pre></div></div>
+</li>
+<li>
+
+<p>Ensure ACLs are properly managed. This includes Kafka and HBase. Often this involves a config file written out as above because this isn&#x2019;t idempotent!</p>
 <ul>
-  
-<li>Ensure ACLs are properly managed. This includes Kafka and HBase. Often this involves a config file written out as above because this isn&#x2019;t idempotent!
-  
-<ul>
-    
+
 <li>Make sure to <tt>kinit</tt> as the correct user for setting up ACLs in a secured cluster. This is usually kafka for Kafka and hbase for HBase.</li>
-    
 <li>See <tt>set_hbase_acls</tt> in <tt>METRON.CURRENT/package/scripts/enrichment_commands.py</tt> for an HBase example</li>
-    
-<li>See <tt>init_kafka_acls</tt> in <tt>METRON.CURRENT/package/scripts/enrichment_commands.py</tt> and <tt>METRON.CURRENT/package/scripts/metron_service.py</tt> for an Kafka example</li>
-  </ul></li>
+<li>See <tt>init_kafka_acls</tt> in <tt>METRON.CURRENT/package/scripts/enrichment_commands.py</tt> and  <tt>METRON.CURRENT/package/scripts/metron_service.py</tt> for an Kafka example</li>
+</ul>
+</li>
 </ul></div>
-                  </div>
-            </div>
-          </div>
+<div class="section">
+<h2><a name="Upgrading_MPack_Services"></a>Upgrading MPack Services</h2>
+<p>Apache Metron currently provides one service as part of its Metron MPack</p>
+<ul>
 
-    <hr/>
+<li>Metron</li>
+</ul>
+<p>Apache Metron currently provides two services as part of its Elasticsearch MPack</p>
+<ul>
+
+<li>Elasticsearch</li>
+<li>Kibana</li>
+</ul>
+<p>There is currently no mechanism provided for multi-version or backwards compatibility. If you upgrade a service, e.g. Elasticsearch 2.x to 5.x, that is the only version that will be supported by Ambari via MPack.</p>
+<p>The main steps for upgrading a service are split into add-on and common services for each service within the MPack as follows:</p>
+<ul>
+
+<li>Update the common services
+<ul>
+
+<li>Change the service directory to use the new product version number</li>
+<li>Update metainfo.xml</li>
+</ul>
+</li>
+<li>Update the add-on services
+<ul>
+
+<li>Change the service directory to use the new product version number</li>
+<li>Update repoinfo.xml</li>
+<li>Update metainfo.xml</li>
+</ul>
+</li>
+<li>Update mpack.json</li>
+</ul>
+<div class="section">
+<h3><a name="Update_Elasticsearch"></a>Update Elasticsearch</h3>
+<div class="section">
+<h4><a name="Update_Common_Services"></a>Update Common Services</h4>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Change service directory names for Elasticsearch to the new desired version</p>
+
+<div>
+<div>
+<pre class="source">metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/ELASTICSEARCH/${YOUR_VERSION_NUMBER_HERE}
+</pre></div></div>
+
+<p>e.g.</p>
+
+<div>
+<div>
+<pre class="source">metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/ELASTICSEARCH/5.6.2
+</pre></div></div>
+</li>
+<li>
+
+<p>Update metainfo.xml</p>
+<p>Change the version number and package name in <tt>metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/ELASTICSEARCH/${YOUR_VERSION_NUMBER_HERE}/metainfo.xml</tt>, e.g.</p>
+
+<div>
+<div>
+<pre class="source">&lt;version&gt;5.6.2&lt;/version&gt;
+...
+&lt;osSpecifics&gt;
+    &lt;osSpecific&gt;
+        &lt;osFamily&gt;any&lt;/osFamily&gt;
+        &lt;packages&gt;
+            &lt;package&gt;
+                &lt;name&gt;elasticsearch-5.6.2&lt;/name&gt;
+            &lt;/package&gt;
+        &lt;/packages&gt;
+    &lt;/osSpecific&gt;
+&lt;/osSpecifics&gt;
+</pre></div></div>
+</li>
+</ol></div>
+<div class="section">
+<h4><a name="Update_Add-on_Services"></a>Update Add-on Services</h4>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Change service directory names for Elasticsearch to the new desired version</p>
+
+<div>
+<div>
+<pre class="source">metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/ELASTICSEARCH/${YOUR_VERSION_NUMBER_HERE}
+</pre></div></div>
+
+<p>e.g.</p>
+
+<div>
+<div>
+<pre class="source">metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/ELASTICSEARCH/5.6.2
+</pre></div></div>
+</li>
+<li>
+
+<p>Update repoinfo.xml</p>
+<p>See <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/current/rpm.html">https://www.elastic.co/guide/en/elasticsearch/reference/current/rpm.html</a> for the latest info.</p>
+<p>Modify the baseurl and repoid in <tt>metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/ELASTICSEARCH/${YOUR_VERSION_NUMBER_HERE}/repos/repoinfo.xml</tt>, e.g.</p>
+
+<div>
+<div>
+<pre class="source">&lt;baseurl&gt;https://artifacts.elastic.co/packages/5.x/yum&lt;/baseurl&gt;
+&lt;repoid&gt;elasticsearch-5.x&lt;/repoid&gt;
+&lt;reponame&gt;ELASTICSEARCH&lt;/reponame&gt;
+</pre></div></div>
+</li>
+<li>
+
+<p>Update metainfo.xml</p>
+<p>Change the version number in <tt>metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/ELASTICSEARCH/${YOUR_VERSION_NUMBER_HERE}/metainfo.xml</tt>. Also make sure to update the &#x201c;extends&#x201d; version to point to the updated common-services version, e.g.</p>
+
+<div>
+<div>
+<pre class="source">&lt;name&gt;ELASTICSEARCH&lt;/name&gt;
+&lt;version&gt;5.6.2&lt;/version&gt;
+&lt;extends&gt;common-services/ELASTICSEARCH/5.6.2&lt;/extends&gt;
+</pre></div></div>
+</li>
+</ol></div>
+<div class="section">
+<h4><a name="Update_mpack.json"></a>Update mpack.json</h4>
+<ol style="list-style-type: decimal">
 
+<li>Update the corresponding service_version in the service_versions_map, e.g.
+
+<div>
+<div>
+<pre class="source">...
+&quot;service_versions_map&quot;: [
+  {
+    &quot;service_name&quot; : &quot;ELASTICSEARCH&quot;,
+    &quot;service_version&quot; : &quot;5.6.2&quot;,
+    &quot;applicable_stacks&quot; : [
+        ...
+    ]
+  },
+  ...
+ ]
+...
+</pre></div></div>
+</li>
+</ol></div></div>
+<div class="section">
+<h3><a name="Kibana"></a>Kibana</h3>
+<p><b>Note:</b> Curator is included with the Kibana service</p>
+<div class="section">
+<h4><a name="Update_Common_Services"></a>Update Common Services</h4>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Change service directory names for Kibana to the new desired version</p>
+
+<div>
+<div>
+<pre class="source">metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/KIBANA/${YOUR_VERSION_NUMBER_HERE}
+</pre></div></div>
+
+<p>e.g.</p>
+
+<div>
+<div>
+<pre class="source">metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/KIBANA/5.6.2
+</pre></div></div>
+</li>
+<li>
+
+<p>Update metainfo.xml</p>
+<p>Change the version number and package name in <tt>metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/KIBANA/${YOUR_VERSION_NUMBER_HERE}/metainfo.xml</tt>, e.g.</p>
+
+<div>
+<div>
+<pre class="source">&lt;version&gt;5.6.2&lt;/version&gt;
+...
+&lt;packages&gt;
+    ...
+    &lt;package&gt;
+        &lt;name&gt;kibana-5.6.2&lt;/name&gt;
+    &lt;/package&gt;
+&lt;/packages&gt;
+</pre></div></div>
+</li>
+</ol></div>
+<div class="section">
+<h4><a name="Update_Add-on_Services"></a>Update Add-on Services</h4>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Change service directory names for Kibana to the new desired version</p>
+
+<div>
+<div>
+<pre class="source">metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/KIBANA/${YOUR_VERSION_NUMBER_HERE}
+</pre></div></div>
+
+<p>e.g.</p>
+
+<div>
+<div>
+<pre class="source">metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/KIBANA/5.6.2
+</pre></div></div>
+</li>
+<li>
+
+<p>Update repoinfo.xml</p>
+<p><b>Note:</b> for Curator, there is a different repo for rhel 6 vs rhel 7</p>
+<p>See the following links for current repo information for Kibana and Curator.</p>
+<ul>
+
+<li><a class="externalLink" href="https://www.elastic.co/guide/en/kibana/current/rpm.html">https://www.elastic.co/guide/en/kibana/current/rpm.html</a></li>
+<li><a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/client/curator/current/yum-repository.html">https://www.elastic.co/guide/en/elasticsearch/client/curator/current/yum-repository.html</a></li>
+</ul>
+<p>Modify the baseurl&#x2019;s and repoid&#x2019;s in <tt>metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/KIBANA/${YOUR_VERSION_NUMBER_HERE}/repos/repoinfo.xml</tt>, e.g.</p>
+
+<div>
+<div>
+<pre class="source">&lt;baseurl&gt;https://artifacts.elastic.co/packages/5.x/yum&lt;/baseurl&gt;
+&lt;repoid&gt;kibana-5.x&lt;/repoid&gt;
+&lt;reponame&gt;KIBANA&lt;/reponame&gt;
+...
+&lt;baseurl&gt;http://packages.elastic.co/curator/5/centos/6&lt;/baseurl&gt;
+&lt;repoid&gt;ES-Curator-5.x&lt;/repoid&gt;
+&lt;reponame&gt;CURATOR&lt;/reponame&gt;
+</pre></div></div>
+</li>
+<li>
+
+<p>Update metainfo.xml</p>
+<p>Change the version number in <tt>metron/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/KIBANA/${YOUR_VERSION_NUMBER_HERE}/metainfo.xml</tt>. Also make sure to update the &#x201c;extends&#x201d; version to point to the updated common-services version, e.g.</p>
+
+<div>
+<div>
+<pre class="source">&lt;name&gt;KIBANA&lt;/name&gt;
+&lt;version&gt;5.6.2&lt;/version&gt;
+&lt;extends&gt;common-services/KIBANA/5.6.2&lt;/extends&gt;
+</pre></div></div>
+</li>
+</ol></div>
+<div class="section">
+<h4><a name="Update_mpack.json"></a>Update mpack.json</h4>
+<ol style="list-style-type: decimal">
+
+<li>Update the corresponding service_version in the service_versions_map, e.g.
+
+<div>
+<div>
+<pre class="source">...
+&quot;service_versions_map&quot;: [
+  {
+    &quot;service_name&quot; : &quot;KIBANA&quot;,
+    &quot;service_version&quot; : &quot;5.6.2&quot;,
+    &quot;applicable_stacks&quot; : [
+        ...
+    ]
+  },
+  ...
+ ]
+...
+</pre></div></div>
+</li>
+</ol></div></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/packaging/ambari/metron-mpack/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/packaging/ambari/metron-mpack/index.html b/site/current-book/metron-deployment/packaging/ambari/metron-mpack/index.html
new file mode 100644
index 0000000..dd97717
--- /dev/null
+++ b/site/current-book/metron-deployment/packaging/ambari/metron-mpack/index.html
@@ -0,0 +1,337 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/ambari/metron-mpack/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; </title>
+    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../../css/site.css" />
+    <link rel="stylesheet" href="../../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active "></li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-down"></span>Ambari</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/packaging/ambari/elasticsearch-mpack/index.html" title="Elasticsearch-mpack"><span class="none"></span>Elasticsearch-mpack</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Mpack</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<p>This provides a Management Pack (MPack) extension for <a class="externalLink" href="https://ambari.apache.org/">Apache Ambari</a> that simplifies the provisioning, management and monitoring of Metron on clusters of any size.</p>
+<p>This allows you to easily install Metron using a simple, guided process.  This also allows you to monitor cluster health and even secure your cluster with kerberos.</p>
+<div class="section">
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<ul>
+
+<li>
+
+<p>Ambari 2.4.2+</p>
+</li>
+<li>
+
+<p>Installable Metron packages (either RPMs or DEBs) located in a repository on each host at <tt>/localrepo</tt>.</p>
+</li>
+<li>
+
+<p>A <a class="externalLink" href="https://nodejs.org/en/download/package-manager/">Node.js</a> repository installed on the host running the Management and Alarm UI.</p>
+</li>
+</ul></div>
+<div class="section">
+<h3><a name="Quick_Start"></a>Quick Start</h3>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Build the Metron MPack. Execute the following command from the project&#x2019;s root directory.</p>
+
+<div>
+<div>
+<pre class="source">mvn clean package -Pmpack -DskipTests
+</pre></div></div>
+</li>
+<li>
+
+<p>This results in the Mpack being produced at the following location.</p>
+
+<div>
+<div>
+<pre class="source">metron-deployment/packaging/ambari/metron-mpack/target/metron_mpack-x.y.z.0.tar.gz
+</pre></div></div>
+</li>
+<li>
+
+<p>Copy the tarball to the host where Ambari Server is installed.</p>
+</li>
+<li>
+
+<p>Ensure that Ambari Server is stopped.</p>
+</li>
+<li>
+
+<p>Install the MPack.</p>
+
+<div>
+<div>
+<pre class="source">ambari-server install-mpack --mpack=metron_mpack-x.y.z.0.tar.gz --verbose
+</pre></div></div>
+</li>
+<li>
+
+<p>Install the Metron packages (RPMs or DEBs) in a local repository on each host where a Metron component is installed.  By default, the repository is expected to exist at <tt>/localrepo</tt>.</p>
+<p>On hosts where only a Metron client is installed, the local repository must exist, but it does not need to contain Metron packages.  For example to create an empty repository for an RPM-based system, run the following commands.</p>
+
+<div>
+<div>
+<pre class="source">yum install createrepo
+mkdir /localrepo
+cd /localrepo
+createrepo
+</pre></div></div>
+</li>
+<li>
+
+<p>Metron will now be available as an installable service within Ambari.</p>
+</li>
+</ol></div>
+<div class="section">
+<h3><a name="Installation_Notes"></a>Installation Notes</h3>
+<p>The MPack will make all Metron services available in Ambari in the same manner as any other services in a stack.  These can be installed using Ambari&#x2019;s user interface using &#x201c;Add Services&#x201d; or during an initial cluster install.</p>
+<div class="section">
+<h4><a name="Kerberization"></a>Kerberization</h4>
+<p>The MPack allows Metron to be automatically kerberized in two different ways.</p>
+<ul>
+
+<li>Metron can be installed on a non-kerberized cluster and then the entire cluster can be kerberized using Ambari.</li>
+<li>Alternatively, Metron can be installed on top of an already kerberized cluster.</li>
+</ul>
+<p>Using the MPack is preferred, but instructions for manually Kerberizing a cluster with Metron can be found at <a href="../../Kerberos-manual-setup.html">Kerberos-manual-setup.md</a>.</p>
+<div class="section">
+<h5><a name="Metron_Client"></a>Metron Client</h5>
+<p>A &#x201c;Metron Client&#x201d; must be installed on each supervisor node in a kerberized cluster.  This client ensures that the Metron keytab and <tt>client_jaas.conf</tt> get distributed to each node in order to allow reading and writing from Kafka.</p>
+<ul>
+
+<li>When Metron is already installed on the cluster, installation of the &#x201c;Metron Client&#x201d; should be done before Kerberizing.</li>
+<li>When adding Metron to an already Kerberized cluster, ensure that all supervisor nodes receive a Metron client.</li>
+</ul></div>
+<div class="section">
+<h5><a name="Restarts"></a>Restarts</h5>
+<p>Storm (and the Metron topologies) must be restarted after Metron is installed on an already Kerberized cluster.  The restart triggers several Storm configurations to get updated and Metron will be unable to write to Kafka without a restart.</p>
+<p>Kerberizing a cluster with a pre-existing Metron, automatically restarts all services during Kerberization.  No additional manual restart is needed in this case.</p></div></div>
+<div class="section">
+<h4><a name="Kibana_Dashboards"></a>Kibana Dashboards</h4>
+<p>The dashboards installed by the Kibana custom action are managed by two JSON files:</p>
+<ul>
+
+<li>metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/dashboard/kibana.template</li>
+<li>metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/dashboard/dashboard-bulkload.json</li>
+</ul>
+<p>The first file, <tt>kibana.template</tt>, is an Elasticsearch template that specifies the proper mapping types for the Kibana index. This configuration is necessary due to a bug in the default dynamic mappings provided by Elasticsearch for long types versus integer that are incompatible with Kibana [1]. The second file, <tt>dashboard-bulkload.json</tt>, contains all of the dashboard metadata necessary to create the Metron dashboard. It is an Elasticsearch bulk-insert formatted file [2] that contains a series of documents necessary for setting up the dashboard in Elasticsearch. The main features installed are index patterns, searches, and a variety of visualizations that are used in the Metron dashboard.</p>
+<p>Deploying the existing dashboard is easy. Once the MPack is installed, run the Metron service&#x2019;s action &#x201c;Load Template&#x201d; to install dashboards.  This will no longer overwrite the .kibana in Elasticsearch. The bulk load is configured to fail inserts for existing documents. If you want to <i>completely</i> reload the dashboard, you would need to delete the .kibana index and reload again from Ambari.</p>
+<ol style="list-style-type: decimal">
+
+<li><a class="externalLink" href="https://github.com/elastic/kibana/issues/9888#issuecomment-298096954">https://github.com/elastic/kibana/issues/9888#issuecomment-298096954</a></li>
+<li><a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/5.6/docs-bulk.html">https://www.elastic.co/guide/en/elasticsearch/reference/5.6/docs-bulk.html</a></li>
+</ol>
+<div class="section">
+<h5><a name="Modifying_Pre-Installed_Dashboards"></a>Modifying Pre-Installed Dashboards</h5>
+<p>You can modify dashboards in Kibana and bring those changes into the core MPack distribution by performing the following steps:</p>
+<ol style="list-style-type: decimal">
+
+<li>Export the .kibana index from ES</li>
+<li>Convert the data into the ES bulk load format</li>
+<li>Replace the dashboard-bulkload.json file in the Metron MPack.</li>
+</ol>
+<p>You can export the .kibana index using a tool like <a class="externalLink" href="https://github.com/taskrabbit/elasticsearch-dump">https://github.com/taskrabbit/elasticsearch-dump</a>. The important feature is to have one document per line. Here&#x2019;s an exmaple export using elasticsearch-dump</p>
+
+<div>
+<div>
+<pre class="source">elasticdump \
+  --input=http://node1:9200/.kibana \
+  --output=~/dashboard-data.json \
+  --type=data
+</pre></div></div>
+
+<p>Once you&#x2019;ve exported the data, you can now format it as a bulk load ES file by running the import/export tool located in metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/bulk/ElasticsearchImportExport.java. This tool can be run from full-dev as follows</p>
+
+<div>
+<div>
+<pre class="source">java -cp $METRON_HOME/lib/metron-elasticsearch-0.4.2-uber.jar org.apache.metron.elasticsearch.bulk.ElasticsearchImportExport \
+  ~/dashboard-data.json \
+  ~/dashboard-bulkload.json
+</pre></div></div>
+
+<p>Locate the &#x201c;create&#x201d; command for setting the default index by searching for &#x201c;5.6.2&#x201d;. Change &#x201c;create&#x201d; to &#x201c;index&#x201d; so that it modifies the existing value. It should look similar to line 1 below.</p>
+
+<div>
+<div>
+<pre class="source">{ &quot;index&quot; : { &quot;_id&quot;: &quot;5.6.2&quot;, &quot;_type&quot;: &quot;config&quot; } }
+{&quot;defaultIndex&quot;:&quot;AV-S2e81hKs1cXXnFMqN&quot;}
+</pre></div></div>
+
+<p>Now copy this file to the Kibana MPack, overwriting the existing bulk load file. That should be everything needed to backup the dashboard.</p>
+<p><b>Note</b>: the dashboard Python Pickle binary file is deprecated and no longer used for backing up and restoring Kibana dashboards. The tooling is still provided as of this version but is expected to be removed in the future. A section describing the deprecated backup process remains below.</p></div>
+<div class="section">
+<h5><a name="Deprecated_Dashboard_Install.2FBackup_Instructions"></a>Deprecated Dashboard Install/Backup Instructions</h5>
+<p>The dashboards installed by the Kibana custom action are managed by the dashboard.p file.  This file is created by exporting existing dashboards from a running Kibana instance.</p>
+<p>To create a new version of the file, make any necessary changes to Kibana (e.g. on full-dev), and export with the appropriate script.</p>
+<p><b>Script Options</b></p>
+
+<div>
+<div>
+<pre class="source">[elasticsearch_host]        ES host
+[elasticsearch_port]        ES port number
+[input_output_filename]     Filename used for reading or writing out pickle file
+[-s]                        Flag to indicate that the .kibana index should be saved locally. Not including this flag will overwrite the .kibana
+                            index completely with the contents of 'input_output_filename'. Careful with this.
+</pre></div></div>
+
+<p><b>Saving a Backup</b></p>
+
+<div>
+<div>
+<pre class="source">python packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/dashboard/dashboardindex.py \
+$ES_HOST 9200 \
+~/dashboard.p -s
+</pre></div></div>
+
+<p><b>Restoring From a Backup</b></p>
+
+<div>
+<div>
+<pre class="source">python packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/dashboard/dashboardindex.py \
+$ES_HOST 9200 \
+~/dashboard.p
+</pre></div></div>
+
+<p><b>Note</b>: This method of writing the Kibana dashboard to Elasticsearch will overwrite the entire .kibana index. Be sure to first backup the index first using either the new JSON method described above, or writing out the dashboard.p pickle file using the old method (passing -s option to dashboardindex.py) described here.</p></div></div>
+<div class="section">
+<h4><a name="Zeppelin_Import"></a>Zeppelin Import</h4>
+<p>A custom action is available in Ambari to import Zeppelin dashboards. See the <a href="../../../../metron-platform/metron-indexing/index.html">metron-indexing documentation</a> for more information.</p></div>
+<div class="section">
+<h4><a name="Offline_Installation"></a>Offline Installation</h4>
+<p>Retrieval of the GeoIP database is the only point during installation that reaches out to the internet. For an offline installation, the URL for the GeoIP database can be manually set to a local path on the file system such as  <tt>file:///home/root/geoip/GeoLite2-City.mmdb.gz</tt>.</p>
+<p>The RPMs DO NOT reach out to the internet (because there is currently no hosting for them).  They look on the local filesystem in <tt>/localrepo</tt>.</p></div>
+<div class="section">
+<h4><a name="Limitations"></a>Limitations</h4>
+<p>There are a few limitations that should be addressed to improve the Metron MPack installation.</p>
+<ul>
+
+<li>
+
+<p>There is no external hosting for Metron packages (either RPMs or DEBs).  These have to be built locally and installed on each host in a repository located at <tt>/localrepo</tt>.</p>
+</li>
+<li>
+
+<p>Several configuration parameters used when installing Metron could retrieved from Ambari rather than requiring user input.</p>
+</li>
+<li>
+
+<p>The MPack does not support upgrades.</p>
+</li>
+</ul></div></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>


[31/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/amazon-ec2/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/amazon-ec2/index.html b/site/current-book/metron-deployment/amazon-ec2/index.html
index 4e7e771..8bf69a8 100644
--- a/site/current-book/metron-deployment/amazon-ec2/index.html
+++ b/site/current-book/metron-deployment/amazon-ec2/index.html
@@ -1,536 +1,428 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/amazon-ec2/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Apache Metron on Amazon EC2</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Apache Metron on Amazon EC2</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Apache Metron on Amazon EC2</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Amazon-ec2</a>
-          </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Apache Metron on Amazon EC2</h1>
-<p>This project fully automates the provisioning of Apache Metron on Amazon EC2 infrastructure. Starting with only your Amazon EC2 credentials, this project will create a fully-functioning, end-to-end, multi-node cluster running Apache Metron.</p>
-<p>Warning: Amazon will charge for the use of their resources when running Apache Metron. The amount will vary based on the number and size of hosts, along with current Amazon pricing structure. Be sure to stop or terminate all of the hosts instantiated by Apache Metron when not in use to avoid unnecessary charges.</p>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Apache Metron on Amazon EC2</h1>
+<p>This project fully automates the provisioning of Apache Metron on Amazon EC2 infrastructure.  Starting with only your Amazon EC2 credentials, this project will create a fully-functioning, end-to-end, multi-node cluster running Apache Metron.</p>
+<p>Warning: Amazon will charge for the use of their resources when running Apache Metron.  The amount will vary based on the number and size of hosts, along with current Amazon pricing structure.  Be sure to stop or terminate all of the hosts instantiated by Apache Metron when not in use to avoid unnecessary charges.</p>
+<div class="section">
+<h2><a name="AWS_Defaults"></a>AWS Defaults</h2>
+<p>The Ansible playbook uses the following defaults for AWS deployment:</p>
+<ul>
+
+<li>Instances: 10</li>
+<li>Region: us-west-2</li>
+<li>Instance type: m4.xlarge</li>
+</ul></div>
 <div class="section">
 <h2><a name="Getting_Started"></a>Getting Started</h2>
 <div class="section">
 <h3><a name="Prerequisites"></a>Prerequisites</h3>
-<p>The host used to deploy Apache Metron will need the following software tools installed. The following versions are known to work as of the time of this writing, but by no means are these the only working versions.</p>
-
+<p>The host used to deploy Apache Metron will need the following software tools installed.  The following versions are known to work as of the time of this writing, but by no means are these the only working versions.</p>
 <ul>
-  
-<li>Ansible 2.0.0.2 or 2.2.2.0</li>
-  
+
+<li>Ansible 2.0.0.2, 2.2.2.0, or 2.5.0</li>
 <li>Python 2.7.11</li>
-  
 <li>Maven 3.3.9</li>
 </ul>
-<p>Any platform that supports these tools is suitable, but the following instructions cover only macOS. The easiest means of installing these tools on a Mac is to use the excellent <a class="externalLink" href="http://brew.sh/">Homebrew</a> project.</p>
-
+<p>Any platform that supports these tools is suitable, but the following instructions cover only macOS.  The easiest means of installing these tools on a Mac is to use the excellent <a class="externalLink" href="http://brew.sh/">Homebrew</a> project.</p>
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Install Homebrew by running the following command in a terminal. Refer to the <a class="externalLink" href="http://brew.sh/">Homebrew</a> home page for the latest installation instructions.</p>
-  
-<div class="source">
-<div class="source">
-<pre>  /usr/bin/ruby -e &quot;$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)&quot;
-</pre></div></div></li>
-  
+
+<p>Install Homebrew by running the following command in a terminal.  Refer to the  <a class="externalLink" href="http://brew.sh/">Homebrew</a> home page for the latest installation instructions.</p>
+
+<div>
+<div>
+<pre class="source">/usr/bin/ruby -e &quot;$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)&quot;
+</pre></div></div>
+</li>
 <li>
-<p>With Homebrew installed, run the following command in a terminal to install all of the required tools.</p>
-  
-<div class="source">
-<div class="source">
-<pre>  brew cask install java
-  brew install maven git
-</pre></div></div></li>
-  
+
+<p>With Homebrew installed, run the following command in a terminal to install all of the required tools and dependencies.</p>
+
+<div>
+<div>
+<pre class="source">brew update
+brew tap caskroom/versions
+brew cask install java8 vagrant virtualbox
+brew install maven git node
+</pre></div></div>
+</li>
 <li>
-<p>Install Ansible by following the instructions <a class="externalLink" href="http://docs.ansible.com/ansible/intro_installation.html#latest-releases-via-pip">here</a>.</p></li>
-  
+
+<p>Install Ansible by following the instructions <a class="externalLink" href="http://docs.ansible.com/ansible/intro_installation.html#latest-releases-via-pip">here</a>.</p>
+</li>
 <li>
+
 <p>Ensure that a public SSH key is located at <tt>~/.ssh/id_rsa.pub</tt>.</p>
-  
-<div class="source">
-<div class="source">
-<pre>  $ cat ~/.ssh/id_rsa.pub
-  ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQChv5GJxPjR39UJV7VY17ivbLVlxFrH7UHwh1Jsjem4d1eYiAtde5N2y65/HRNxWbhYli9ED8k0/MRP92ejewucEbrPNq5mytPqdC4IvZ98Ln2GbqTDwvlP3T7xa/wYFOpFsOmXXql8216wSrnrS4f3XK7ze34S6/VmY+lsBYnr3dzyj8sG/mexpJgFS/w83mWJV0e/ryf4Hd7P6DZ5fO+nmTXfKNK22ga4ctcnbZ+toYcPL+ODCh8598XCKVo97XjwF5OxN3vl1p1HHguo3cHB4H1OIaqX5mUt59gFIZcAXUME89PO6NUiZDd3RTstpf125nQVkQAHu2fvW96/f037 nick@localhost
+
+<div>
+<div>
+<pre class="source">$ cat ~/.ssh/id_rsa.pub
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQChv5GJxPjR39UJV7VY17ivbLVlxFrH7UHwh1Jsjem4d1eYiAtde5N2y65/HRNxWbhYli9ED8k0/MRP92ejewucEbrPNq5mytPqdC4IvZ98Ln2GbqTDwvlP3T7xa/wYFOpFsOmXXql8216wSrnrS4f3XK7ze34S6/VmY+lsBYnr3dzyj8sG/mexpJgFS/w83mWJV0e/ryf4Hd7P6DZ5fO+nmTXfKNK22ga4ctcnbZ+toYcPL+ODCh8598XCKVo97XjwF5OxN3vl1p1HHguo3cHB4H1OIaqX5mUt59gFIZcAXUME89PO6NUiZDd3RTstpf125nQVkQAHu2fvW96/f037 nick@localhost
 </pre></div></div>
-<p>If this file does not exist, run the following command at a terminal and accept all defaults. Only the public key, not the private key, will be uploaded to Amazon and configured on each host to enable SSH connectivity. While it is possible to create and use an alternative key those details will not be covered. </p>
-  
-<div class="source">
-<div class="source">
-<pre>  ssh-keygen -t rsa
-</pre></div></div></li>
+
+<p>If this file does not exist, run the following command at a terminal and accept all defaults.  Only the public key, not the private key, will be uploaded to Amazon and configured on each host to enable SSH connectivity.  While it is possible to create and use an alternative key those details will not be covered.</p>
+
+<div>
+<div>
+<pre class="source">ssh-keygen -t rsa
+</pre></div></div>
+</li>
+<li>
+
+<p>Ensure the JAVA_HOME environment variable is set</p>
+
+<div>
+<div>
+<pre class="source">export JAVA_HOME=&quot;/Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home&quot;
+</pre></div></div>
+
+<p>Notice: You must replace the path with the installed JDK version path</p>
+</li>
 </ol></div>
 <div class="section">
 <h3><a name="Amazon_Web_Services"></a>Amazon Web Services</h3>
 <p>If you already have an Amazon Web Services account that you have used to deploy EC2 hosts, then you should be able to skip the next few steps.</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Head over to <a class="externalLink" href="http://aws.amazon.com/">Amazon Web Services</a> and create an account. As part of the account creation process you will need to provide a credit card to cover any charges that may apply.</p></li>
-  
+
+<p>Head over to <a class="externalLink" href="http://aws.amazon.com/">Amazon Web Services</a> and create an account.  As part of the account creation process you will need to provide a credit card to cover any charges that may apply.</p>
+</li>
 <li>
-<p>Create a set of user credentials through <a class="externalLink" href="https://console.aws.amazon.com/iam/">Amazon&#x2019;s Identity and Access Management (IAM) </a> dashboard. On the IAM dashboard menu click &#x201c;Users&#x201d; and then &#x201c;Create New User&#x201d;. Provide a name and ensure that &#x201c;Generate an access key for each user&#x201d; remains checked. Download the credentials and keep them for later use.</p></li>
-  
+
+<p>Create a set of user credentials through <a class="externalLink" href="https://console.aws.amazon.com/iam/">Amazon&#x2019;s Identity and Access Management (IAM) </a> dashboard.  On the IAM dashboard menu click &#x201c;Users&#x201d; and then &#x201c;Create New User&#x201d;. Provide a name and ensure that &#x201c;Generate an access key for each user&#x201d; remains checked.  Download the credentials and keep them for later use.</p>
+</li>
 <li>
-<p>While still in <a class="externalLink" href="https://console.aws.amazon.com/iam/">Amazon&#x2019;s Identity and Access Management (IAM) </a> dashboard, click on the user that was previously created. Click the &#x201c;Permissions&#x201d; tab and then the &#x201c;Attach Policy&#x201d; button. Attach the following policies to the user.</p>
-  
+
+<p>While still in <a class="externalLink" href="https://console.aws.amazon.com/iam/">Amazon&#x2019;s Identity and Access Management (IAM) </a> dashboard, click on the user that was previously created.  Click the &#x201c;Permissions&#x201d; tab and then the &#x201c;Attach Policy&#x201d; button.  Attach the following policies to the user.</p>
 <ul>
-    
+
 <li>AmazonEC2FullAccess</li>
-    
 <li>AmazonVPCFullAccess</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>
-<p>Apache Metron uses the <a class="externalLink" href="https://aws.amazon.com/marketplace/pp/B00NQAYLWO">official, open source CentOS 6</a> Amazon Machine Image (AMI). If you have never used this AMI before then you will need to accept Amazon&#x2019;s terms and conditions. Navigate to the <a class="externalLink" href="https://aws.amazon.com/marketplace/pp/B00NQAYLWO">web page for this AMI</a> and click the &#x201c;Continue&#x201d; button. Choose the &#x201c;Manual Launch&#x201d; tab then click the &#x201c;Accept Software Terms&#x201d; button.</p></li>
+
+<p>Apache Metron uses the <a class="externalLink" href="https://aws.amazon.com/marketplace/pp/B00NQAYLWO">official, open source CentOS 6</a> Amazon Machine Image (AMI).  If you have never used this AMI before then you will need to accept Amazon&#x2019;s terms and conditions.  Navigate to the <a class="externalLink" href="https://aws.amazon.com/marketplace/pp/B00NQAYLWO">web page for this AMI</a> and click the &#x201c;Continue&#x201d; button.  Choose the &#x201c;Manual Launch&#x201d; tab then click the &#x201c;Accept Software Terms&#x201d; button.</p>
+</li>
 </ol>
-<p>Having successfully created your Amazon Web Services account, hopefully you will find that the most difficult tasks are behind us. </p></div>
+<p>Having successfully created your Amazon Web Services account, hopefully you will find that the most difficult tasks are behind us.</p></div>
 <div class="section">
 <h3><a name="Deploy_Metron"></a>Deploy Metron</h3>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Use the Amazon access key by exporting its values via the shell&#x2019;s environment. This allows Ansible to authenticate with Amazon EC2. For example:</p>
-  
-<div class="source">
-<div class="source">
-<pre>  export AWS_ACCESS_KEY_ID=&quot;AKIAI6NRFEO27E5FFELQ&quot;
-  export AWS_SECRET_ACCESS_KEY=&quot;vTDydWJQnAer7OWauUS150i+9Np7hfCXrrVVP6ed&quot;
+
+<p>Use the Amazon access key by exporting its values via the shell&#x2019;s environment.  This allows Ansible to authenticate with Amazon EC2.  For example:</p>
+
+<div>
+<div>
+<pre class="source">export AWS_ACCESS_KEY_ID=&quot;AKIAI6NRFEO27E5FFELQ&quot;
+export AWS_SECRET_ACCESS_KEY=&quot;vTDydWJQnAer7OWauUS150i+9Np7hfCXrrVVP6ed&quot;
 </pre></div></div>
-<p>Notice: You must replace the access key values above with values from your own access key.</p></li>
-  
+
+<p>Notice: You must replace the access key values above with values from your own access key.</p>
+</li>
 <li>
-<p>Start the Apache Metron deployment process. When prompted provide a unique name for your Metron environment or accept the default. </p>
-  
-<div class="source">
-<div class="source">
-<pre>  $ ./run.sh
-  Metron Environment [metron-test]: my-metron-env
-  ...
+
+<p>Start the Apache Metron deployment process.  When prompted provide a unique name for your Metron environment or accept the default.</p>
+
+<div>
+<div>
+<pre class="source">$ ./run.sh
+Metron Environment [metron-test]: my-metron-env
+...
 </pre></div></div>
-<p>The process is likely to take between 70-90 minutes. Fortunately, everything is fully automated and you should feel free to grab a coffee.</p></li>
+
+<p>The process is likely to take between 70-90 minutes.  Fortunately, everything is fully automated and you should feel free to grab a coffee.</p>
+</li>
 </ol></div>
 <div class="section">
 <h3><a name="Explore_Metron"></a>Explore Metron</h3>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>After the deployment has completed successfully, a message like the following will be displayed. Navigate to the specified resources to explore your newly minted Apache Metron environment.</p>
-  
-<div class="source">
-<div class="source">
-<pre>  TASK [debug] *******************************************************************
-  ok: [localhost] =&gt; {
-  &quot;Success&quot;: [
-      &quot;Apache Metron deployed successfully&quot;,
-      &quot;   Metron  @  http://ec2-52-37-255-142.us-west-2.compute.amazonaws.com:5000&quot;,
-      &quot;   Ambari  @  http://ec2-52-37-225-202.us-west-2.compute.amazonaws.com:8080&quot;,
-      &quot;   Sensors @  ec2-52-37-225-202.us-west-2.compute.amazonaws.com on tap0&quot;,
-      &quot;For additional information, see https://metron.apache.org/'&quot;
-  ]
-  }
-</pre></div></div></li>
-  
+
+<p>After the deployment has completed successfully, a message like the following will be displayed.  Navigate to the specified resources to explore your newly minted Apache Metron environment.</p>
+
+<div>
+<div>
+<pre class="source">TASK [debug] *******************************************************************
+ok: [localhost] =&gt; {
+    &quot;Success&quot;: [
+        &quot;Apache Metron deployed successfully&quot;,
+        &quot;   Metron  @  http://ec2-52-37-255-142.us-west-2.compute.amazonaws.com:5000&quot;,
+        &quot;   Ambari  @  http://ec2-52-37-225-202.us-west-2.compute.amazonaws.com:8080&quot;,
+        &quot;   Sensors @  ec2-52-37-225-202.us-west-2.compute.amazonaws.com on tap0&quot;,
+        &quot;For additional information, see https://metron.apache.org/'&quot;
+    ]
+}
+</pre></div></div>
+</li>
 <li>
-<p>Each of the provisioned hosts will be accessible from the internet. Connecting to one over SSH as the user <tt>centos</tt> will not require a password as it will authenticate with the pre-defined SSH key. </p>
-  
-<div class="source">
-<div class="source">
-<pre>  ssh centos@ec2-52-91-215-174.compute-1.amazonaws.com
-</pre></div></div></li>
+
+<p>Each of the provisioned hosts will be accessible from the internet. Connecting to one over SSH as the user <tt>centos</tt> will not require a password as it will authenticate with the pre-defined SSH key.</p>
+
+<div>
+<div>
+<pre class="source">ssh centos@ec2-52-91-215-174.compute-1.amazonaws.com
+</pre></div></div>
+</li>
 </ol></div></div>
 <div class="section">
 <h2><a name="Advanced_Usage"></a>Advanced Usage</h2>
 <div class="section">
 <h3><a name="Multiple_Environments"></a>Multiple Environments</h3>
-<p>This process can support provisioning of multiple, isolated environments. Simply change the <tt>env</tt> settings in <tt>conf/defaults.yml</tt>. For example, you might provision separate development, test, and production environments.</p>
+<p>This process can support provisioning of multiple, isolated environments.  Simply change the <tt>env</tt> settings in <tt>conf/defaults.yml</tt>.  For example, you might provision separate development, test, and production environments.</p>
 
-<div class="source">
-<div class="source">
-<pre>env: metron-test
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">env: metron-test
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Selective_Provisioning"></a>Selective Provisioning</h3>
-<p>To provision only subsets of the entire Metron deployment, Ansible tags can be specified. For example, to only deploy the sensors on an Amazon EC2 environment, run the following command.</p>
+<p>To provision only subsets of the entire Metron deployment, Ansible tags can be specified.  For example, to only deploy the sensors on an Amazon EC2 environment, run the following command:</p>
 
-<div class="source">
-<div class="source">
-<pre>ansible-playbook -i ec2.py playbook.yml --tags &quot;ec2,sensors&quot;
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">./run.sh --tags=&quot;ec2,sensors&quot;
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="Setting_REST_API_Profile"></a>Setting REST API Profile</h3>
+<p>By default, EC2 is deployed with the <tt>dev</tt> Spring profile. This simply instructs the REST API to automatically setup default test users <tt>[user, user1, user2, admin]</tt> with password &#x201c;<tt>password</tt>&#x201d; as opposed to requiring the user to manually create users in the REST database. You can change this default functionality by removing the <tt>metron_spring_profiles_active</tt> setting in <tt>metron-deployment/roles/ambari_config/vars/small_cluster.yml</tt>. You can view more detail on the REST Spring profiles <a href="../../metron-interface/metron-rest/index.html#spring-profiles">here</a>.</p></div>
 <div class="section">
 <h3><a name="Custom_SSH_Key"></a>Custom SSH Key</h3>
-<p>By default, the playbook will attempt to register your public SSH key <tt>~/.ssh/id_rsa.pub</tt> with each provisioned host. This enables Ansible to communicate with each host using an SSH connection. If would prefer to use another key simply add the path to the public key file to the <tt>key_file</tt> property in <tt>conf/defaults.yml</tt>.</p>
+<p>By default, the playbook will attempt to register your public SSH key <tt>~/.ssh/id_rsa.pub</tt> with each provisioned host.  This enables Ansible to communicate with each host using an SSH connection.  If would prefer to use another key simply add the path to the public key file to the <tt>key_file</tt> property in <tt>conf/defaults.yml</tt>.</p>
 <p>For example, generate a new SSH key for Metron that will be stored at <tt>~/.ssh/my-metron-key</tt>.</p>
 
-<div class="source">
-<div class="source">
-<pre>$ ssh-keygen -q -f ~/.ssh/my-metron-key
+<div>
+<div>
+<pre class="source">$ ssh-keygen -q -f ~/.ssh/my-metron-key
 Enter passphrase (empty for no passphrase):
 Enter same passphrase again:
 </pre></div></div>
+
 <p>Add the path to the newly created SSH public key to <tt>conf/defaults.yml</tt>.</p>
 
-<div class="source">
-<div class="source">
-<pre>key_file: ~/.ssh/metron-private-key.pub
-</pre></div></div></div></div>
+<div>
+<div>
+<pre class="source">key_file: ~/.ssh/metron-private-key.pub
+</pre></div></div>
+</div></div>
 <div class="section">
 <h2><a name="Common_Errors"></a>Common Errors</h2>
 <div class="section">
-<h3><a name="Error:_unsupported_operation_exception_custom_format_isnt_supported"></a>Error: [unsupported_operation_exception] custom format isn&#x2019;t supported</h3>
-<p>This error might be seen within Metron&#x2019;s default dashboard in Kibana 4. This occurs when the index templates do not exist for the Snort, Bro or YAF indices in Elasticsearch. </p>
-<p>The dashboard expects fields to be of a certain type. If the index templates have not been loaded correctly, the data types for the fields in these indices will be incorrect and the dashboard will display this error.</p>
+<h3><a name="Error:_.5Bunsupported_operation_exception.5D_custom_format_isn.E2.80.99t_supported"></a>Error: [unsupported_operation_exception] custom format isn&#x2019;t supported</h3>
+<p>This error might be seen within Metron&#x2019;s default dashboard in Kibana 4.  This occurs when the index templates do not exist for the Snort, Bro or YAF indices in Elasticsearch.</p>
+<p>The dashboard expects fields to be of a certain type.  If the index templates have not been loaded correctly, the data types for the fields in these indices will be incorrect and the dashboard will display this error.</p>
 <div class="section">
 <h4><a name="Solution"></a>Solution</h4>
-<p>If you see this error, please report your findings by creating a JIRA or dropping an email to the Metron Users mailing list. Follow these steps to work around the problem.</p>
-
+<p>If you see this error, please report your findings by creating a JIRA or dropping an email to the Metron Users mailing list.  Follow these steps to work around the problem.</p>
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Define which Elasticsearch host to interact with. Any Elasticsearch host should work.</p>
-  
-<div class="source">
-<div class="source">
-<pre>export ES_HOST=&quot;http://ec2-52-25-237-20.us-west-2.compute.amazonaws.com:9200&quot;
-</pre></div></div></li>
-  
+
+<p>Define which Elasticsearch host to interact with.  Any Elasticsearch host should work.</p>
+
+<div>
+<div>
+<pre class="source">export ES_HOST=&quot;http://ec2-52-25-237-20.us-west-2.compute.amazonaws.com:9200&quot;
+</pre></div></div>
+</li>
 <li>
-<p>Confirm the index templates are in fact missing. </p>
-  
-<div class="source">
-<div class="source">
-<pre>curl -s -XGET $ES_HOST/_template
-</pre></div></div></li>
-  
+
+<p>Confirm the index templates are in fact missing.</p>
+
+<div>
+<div>
+<pre class="source">curl -s -XGET $ES_HOST/_template
+</pre></div></div>
+</li>
 <li>
+
 <p>Manually load the index templates.</p>
-  
-<div class="source">
-<div class="source">
-<pre>cd metron-deployment
+
+<div>
+<div>
+<pre class="source">cd metron-deployment
 curl -s -XPOST $ES_HOST/_template/bro_index -d @roles/metron_elasticsearch_templates/files/es_templates/bro_index.template
 curl -s -XPOST $ES_HOST/_template/snort_index -d @roles/metron_elasticsearch_templates/files/es_templates/snort_index.template
 curl -s -XPOST $ES_HOST/_template/yaf_index -d @roles/metron_elasticsearch_templates/files/es_templates/yaf_index.template
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Delete the existing indexes. Only a new index will use the templates defined in the previous step.</p>
-  
-<div class="source">
-<div class="source">
-<pre>curl -s -XDELETE &quot;$ES_HOST/yaf_index*&quot;
+
+<p>Delete the existing indexes.  Only a new index will use the templates defined in the previous step.</p>
+
+<div>
+<div>
+<pre class="source">curl -s -XDELETE &quot;$ES_HOST/yaf_index*&quot;
 curl -s -XDELETE &quot;$ES_HOST/bro_index*&quot;
 curl -s -XDELETE &quot;$ES_HOST/snort_index*&quot;
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Open up Kibana and wait for the new indexes to be created. The dashboard should now work.</p></li>
+
+<p>Open up Kibana and wait for the new indexes to be created.  The dashboard should now work.</p>
+</li>
 </ol></div></div>
 <div class="section">
-<h3><a name="Error:_No_handler_was_ready_to_authenticateCheck_your_credentials"></a>Error: &#x2018;No handler was ready to authenticate&#x2026;Check your credentials&#x2019;</h3>
+<h3><a name="Error:_.E2.80.98No_handler_was_ready_to_authenticate.E2.80.A6Check_your_credentials.E2.80.99"></a>Error: &#x2018;No handler was ready to authenticate&#x2026;Check your credentials&#x2019;</h3>
 
-<div class="source">
-<div class="source">
-<pre>TASK [Define keypair] **********************************************************
+<div>
+<div>
+<pre class="source">TASK [Define keypair] **********************************************************
 failed: [localhost] =&gt; (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXbcb1AlWsEPP
   r9jEFrn0yun3PYNidJ/...david@hasselhoff.com) =&gt; {&quot;failed&quot;: true, &quot;item&quot;: &quot;ssh-r
   sa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXbcb1AlWsEPPr9jEFr... david@hasselhoff.com&quot;,
   &quot;msg&quot;: &quot;No handler was ready to authenticate. 1 handlers were checked.
   ['HmacAuthV4Handler'] Check your credentials&quot;}
 </pre></div></div>
+
 <div class="section">
 <h4><a name="Solution_1"></a>Solution 1</h4>
-<p>This occurs when Ansible does not have the correct AWS access keys. The following commands must return a valid access key that is defined within Amazon&#x2019;s <a class="externalLink" href="https://console.aws.amazon.com/iam/">Identity and Access Management</a> console. </p>
+<p>This occurs when Ansible does not have the correct AWS access keys.  The following commands must return a valid access key that is defined within Amazon&#x2019;s <a class="externalLink" href="https://console.aws.amazon.com/iam/">Identity and Access Management</a> console.</p>
 
-<div class="source">
-<div class="source">
-<pre>$ echo $AWS_ACCESS_KEY_ID
+<div>
+<div>
+<pre class="source">$ echo $AWS_ACCESS_KEY_ID
 AKIAI6NRFEO27E5FFELQ
 
 $ echo $AWS_SECRET_ACCESS_KEY
 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
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Solution_2"></a>Solution 2</h4>
-<p>This error can occur if you have exported the correct AWS access key, but you are using <tt>sudo</tt> to run the Ansible playbook. Do not use the <tt>sudo</tt> command when running the Ansible playbook.</p></div></div>
+<p>This error can occur if you have exported the correct AWS access key, but you are using <tt>sudo</tt> to run the Ansible playbook.  Do not use the <tt>sudo</tt> command when running the Ansible playbook.</p></div></div>
 <div class="section">
-<h3><a name="Error:_OptInRequired:__you_need_to_accept_terms_and_subscribe"></a>Error: &#x2018;OptInRequired: &#x2026; you need to accept terms and subscribe&#x2019;</h3>
+<h3><a name="Error:_.E2.80.98OptInRequired:_.E2.80.A6_you_need_to_accept_terms_and_subscribe.E2.80.99"></a>Error: &#x2018;OptInRequired: &#x2026; you need to accept terms and subscribe&#x2019;</h3>
 
-<div class="source">
-<div class="source">
-<pre>TASK [metron-test: Instantiate 1 host(s) as sensors,ambari_master,metron,ec2] **
+<div>
+<div>
+<pre class="source">TASK [metron-test: Instantiate 1 host(s) as sensors,ambari_master,metron,ec2] **
 fatal: [localhost]: FAILED! =&gt; {&quot;changed&quot;: false, &quot;failed&quot;: true, &quot;msg&quot;:
 &quot;Instance creation failed =&gt; OptInRequired: In order to use this AWS Marketplace
 product you need to accept terms and subscribe. To do so please visit
 http://aws.amazon.com/marketplace/pp?sku=6x5jmcajty9edm3f211pqjfn2&quot;}
 to retry, use: --limit @playbook.retry
 </pre></div></div>
+
 <div class="section">
 <h4><a name="Solution"></a>Solution</h4>
-<p>Apache Metron uses the <a class="externalLink" href="https://aws.amazon.com/marketplace/pp?sku=6x5jmcajty9edm3f211pqjfn2">official CentOS 6 Amazon Machine Image</a> when provisioning hosts. Amazon requires that you accept certain terms and conditions when using any Amazon Machine Image (AMI). Follow the link provided in the error message to accept the terms and conditions then re-run the playbook. </p></div></div>
+<p>Apache Metron uses the <a class="externalLink" href="https://aws.amazon.com/marketplace/pp?sku=6x5jmcajty9edm3f211pqjfn2">official CentOS 6 Amazon Machine Image</a> when provisioning hosts. Amazon requires that you accept certain terms and conditions when using any Amazon Machine Image (AMI).  Follow the link provided in the error message to accept the terms and conditions then re-run the playbook.</p></div></div>
 <div class="section">
-<h3><a name="Error:_PendingVerification:_Your_account_is_currently_being_verified"></a>Error: &#x2018;PendingVerification: Your account is currently being verified&#x2019;</h3>
+<h3><a name="Error:_.E2.80.98PendingVerification:_Your_account_is_currently_being_verified.E2.80.99"></a>Error: &#x2018;PendingVerification: Your account is currently being verified&#x2019;</h3>
 
-<div class="source">
-<div class="source">
-<pre>TASK [metron-test: Instantiate 1 host(s) as sensors,ambari_master,metron,ec2] **
+<div>
+<div>
+<pre class="source">TASK [metron-test: Instantiate 1 host(s) as sensors,ambari_master,metron,ec2] **
 fatal: [localhost]: FAILED! =&gt; {&quot;changed&quot;: false, &quot;failed&quot;: true, &quot;msg&quot;:
 &quot;Instance creation failed =&gt; PendingVerification: Your account is currently
 being verified. Verification normally takes less than 2 hours. Until your
@@ -540,19 +432,20 @@ than 2 hours, please let us know by writing to aws-verification@amazon.com. We
 appreciate your patience.&quot;}
 to retry, use: --limit @playbook.retry
 </pre></div></div>
+
 <div class="section">
 <h4><a name="Solution"></a>Solution</h4>
-<p>This will occur if you are attempting to deploy Apache Metron using a newly created Amazon Web Services account. Follow the advice of the message and wait until Amazon&#x2019;s verification process is complete. Amazon has some additional <a class="externalLink" href="http://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html">advice for dealing with this error and more</a>.</p>
-
+<p>This will occur if you are attempting to deploy Apache Metron using a newly created Amazon Web Services account.  Follow the advice of the message and wait until Amazon&#x2019;s verification process is complete.  Amazon has some additional <a class="externalLink" href="http://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html">advice for dealing with this error and more</a>.</p>
 <blockquote>
+
 <p>Your account is pending verification. Until the verification process is complete, you may not be able to carry out requests with this account. If you have questions, contact <a class="externalLink" href="http://console.aws.amazon.com/support/home#/">AWS Support</a>.</p>
 </blockquote></div></div>
 <div class="section">
-<h3><a name="Error:_Instance_creation_failed__InstanceLimitExceeded"></a>Error: &#x2018;Instance creation failed =&gt; InstanceLimitExceeded&#x2019;</h3>
+<h3><a name="Error:_.E2.80.98Instance_creation_failed_.3D.3E_InstanceLimitExceeded.E2.80.99"></a>Error: &#x2018;Instance creation failed =&gt; InstanceLimitExceeded&#x2019;</h3>
 
-<div class="source">
-<div class="source">
-<pre>TASK [metron-test: Instantiate 3 host(s) as search,metron,ec2] *****************
+<div>
+<div>
+<pre class="source">TASK [metron-test: Instantiate 3 host(s) as search,metron,ec2] *****************
 fatal: [localhost]: FAILED! =&gt; {&quot;changed&quot;: false, &quot;failed&quot;: true, &quot;msg&quot;:
 &quot;Instance creation failed =&gt; InstanceLimitExceeded: You have requested more
 instances (11) than your current instance limit of 10 allows for the specified
@@ -560,44 +453,40 @@ instance type. Please visit http://aws.amazon.com/contact-us/ec2-request to
 request an adjustment to this limit.&quot;}
 to retry, use: --limit @playbook.retry
 </pre></div></div>
+
 <div class="section">
 <h4><a name="Solution"></a>Solution</h4>
-<p>This will occur if Apache Metron attempts to deploy more host instances than allowed by your account. The total number of instances required for Apache Metron can be reduced by editing <tt>deployment/amazon-ec/playbook.yml</tt>. Perhaps a better alternative is to request of Amazon that this limit be increased. Amazon has some additional <a class="externalLink" href="http://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html">advice for dealing with this error and more</a>.</p>
-
+<p>This will occur if Apache Metron attempts to deploy more host instances than allowed by your account.  The total number of instances required for Apache Metron can be reduced by editing <tt>deployment/amazon-ec/playbook.yml</tt>.  Perhaps a better alternative is to request of Amazon that this limit be increased.  Amazon has some additional <a class="externalLink" href="http://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html">advice for dealing with this error and more</a>.</p>
 <blockquote>
+
 <p>You&#x2019;ve reached the limit on the number of instances you can run concurrently. The limit depends on the instance type. For more information, see <a class="externalLink" href="http://aws.amazon.com/ec2/faqs/#How_many_instances_can_I_run_in_Amazon_EC2">How many instances can I run in Amazon EC2</a>. If you need additional instances, complete the <a class="externalLink" href="https://console.aws.amazon.com/support/home#/case/create?issueType=service-limit-increase&amp;limitType=service-code-ec2-instances">Amazon EC2 Instance Request Form</a>.</p>
 </blockquote></div></div>
 <div class="section">
-<h3><a name="Error:_SSH_encountered_an_unknown_error_during_the_connection"></a>Error: &#x2018;SSH encountered an unknown error during the connection&#x2019;</h3>
+<h3><a name="Error:_.E2.80.98SSH_encountered_an_unknown_error_during_the_connection.E2.80.99"></a>Error: &#x2018;SSH encountered an unknown error during the connection&#x2019;</h3>
 
-<div class="source">
-<div class="source">
-<pre>TASK [setup] *******************************************************************
+<div>
+<div>
+<pre class="source">TASK [setup] *******************************************************************
 fatal: [ec2-52-26-113-221.us-west-2.compute.amazonaws.com]: UNREACHABLE! =&gt; {
   &quot;changed&quot;: false, &quot;msg&quot;: &quot;SSH encountered an unknown error during the
   connection. We recommend you re-run the command using -vvvv, which will enable
   SSH debugging output to help diagnose the issue&quot;, &quot;unreachable&quot;: true}
 </pre></div></div>
+
 <div class="section">
 <h4><a name="Solution"></a>Solution</h4>
-<p>This most often indicates that Ansible cannot connect to the host with the SSH key that it has access to. This could occur if hosts are provisioned with one SSH key, but the playbook is executed subsequently with a different SSH key. The issue can be addressed by either altering the <tt>key_file</tt> variable to point to the key that was used to provision the hosts or by simply terminating all hosts and re-running the playbook.</p></div></div></div>
-                  </div>
-            </div>
-          </div>
-
+<p>This most often indicates that Ansible cannot connect to the host with the SSH key that it has access to.  This could occur if hosts are provisioned with one SSH key, but the playbook is executed subsequently with a different SSH key.  The issue can be addressed by either altering the <tt>key_file</tt> variable to point to the key that was used to provision the hosts or by simply terminating all hosts and re-running the playbook.</p></div></div></div>
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/ansible/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/ansible/index.html b/site/current-book/metron-deployment/ansible/index.html
new file mode 100644
index 0000000..383898a
--- /dev/null
+++ b/site/current-book/metron-deployment/ansible/index.html
@@ -0,0 +1,128 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/ansible/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Ansible Assets</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Ansible Assets</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Ansible</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-deployment/ansible/roles/index.html" title="Roles"><span class="icon-chevron-right"></span>Roles</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Ansible Assets</h1>
+<p><a name="Ansible_Assets"></a></p>
+<p>This directory contains all of the shared Ansible assets used to deploy Metron in a <a href="../development/index.html">development environment</a>.  The scope of what Ansible is used for is intentionally limited.  The recommended means of deploying Metron is to use the <a href="../packaging/ambari/metron-mpack/index.html">Metron MPack</a> for <a class="externalLink" href="https://ambari.apache.org/">Apache Ambari</a>.</p>
+<p>Ansible is only used primarily to prepare the development environment for Ambari and for deploying a suite of test sensors to drive telemetry through Metron.  The Ansible assets contained here are of limited use outside of this scenario.</p>
+<p><b>Warning</b> It is not recommended that you use these assets for deploying Metron in your environment for either production or testing.  Support for this use case cannot be provided.</p>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/ansible/roles/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/ansible/roles/index.html b/site/current-book/metron-deployment/ansible/roles/index.html
new file mode 100644
index 0000000..62cc911
--- /dev/null
+++ b/site/current-book/metron-deployment/ansible/roles/index.html
@@ -0,0 +1,140 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/ansible/roles/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Ansible Roles</title>
+    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../css/site.css" />
+    <link rel="stylesheet" href="../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Ansible Roles</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-down"></span>Ansible</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Roles</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/ansible/roles/opentaxii/index.html" title="Opentaxii"><span class="none"></span>Opentaxii</a></li>
+    <li><a href="../../../metron-deployment/ansible/roles/pcap_replay/index.html" title="Pcap_replay"><span class="none"></span>Pcap_replay</a></li>
+    <li><a href="../../../metron-deployment/ansible/roles/sensor-stubs/index.html" title="Sensor-stubs"><span class="none"></span>Sensor-stubs</a></li>
+    <li><a href="../../../metron-deployment/ansible/roles/sensor-test-mode/index.html" title="Sensor-test-mode"><span class="none"></span>Sensor-test-mode</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+    <li><a href="../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Ansible Roles</h1>
+<p><a name="Ansible_Roles"></a></p>
+<ul>
+
+<li>Monit</li>
+<li>OpenTaxii</li>
+<li>Pcap Replay</li>
+<li>Sensor Stubs</li>
+<li>Sensor Test Mode</li>
+</ul>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>


[26/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html b/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html
index 83d98f6..8145c8f 100644
--- a/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html
+++ b/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html
@@ -1,324 +1,174 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/docker/ansible-docker/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Overview</title>
-    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.3.0.min.css" />
+    <title>Metron &#x2013; </title>
+    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../../../css/site.css" />
     <link rel="stylesheet" href="../../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Overview</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active "></li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Ansible-docker</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Overview</h1>
-<p><a name="Overview"></a></p>
-<p>The Metron ansible-docker container is provided in an effort reduce the installation burden of building Metron. It may also be used to deploy Metron in a live environment. It is provisioned with software required to sucessfully build metron and run the deployment scripts.</p>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<p>This provides a Docker Container containing all of the prerequisites required to build Metron.  This allows you to easily build Metron without installing all of the build dependencies manually.</p>
+<div class="section">
 <div class="section">
-<h2><a name="Building_the_Container"></a>Building the Container</h2>
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<ul>
 
+<li>Docker: Please ensure that Docker is installed and that the daemon is running.</li>
+</ul></div>
+<div class="section">
+<h3><a name="Quick_Start"></a>Quick Start</h3>
 <ol style="list-style-type: decimal">
-  
-<li>Install Docker ( <a class="externalLink" href="https://www.docker.com/products/overview">https://www.docker.com/products/overview</a> )</li>
-  
-<li>Navigate to &lt;project-directory&gt;/metron-deployment/packaging/docker/ansible-docker</li>
-  
-<li>Build the container <tt>docker build -t ansible-docker:2.0.0.2 .</tt></li>
+
+<li>
+
+<p>Build the Docker container.</p>
+
+<div>
+<div>
+<pre class="source">cd metron-deployment/packaging/docker/ansible-docker
+docker build -t ansible-docker:latest .
+</pre></div></div>
+</li>
+<li>
+
+<p>Launch the container.</p>
+
+<div>
+<div>
+<pre class="source">docker run -it \
+  -v `pwd`/../../../..:/root/metron \
+  -v ~/.m2:/root/.m2 \
+  ansible-docker:latest bash
+</pre></div></div>
+
+<p>This maps the Metron source code along with your local Maven repository to the container.  This will prevent you from having to re-download all of these dependencies each time you build Metron.</p>
+</li>
+<li>
+
+<p>Execute the following inside the Docker container to build Metron.</p>
+
+<div>
+<div>
+<pre class="source">cd /root/metron
+mvn clean package -DskipTests
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
-<h2><a name="Using_the_Container_to_build_metron"></a>Using the Container to build metron</h2>
-<p>anytime after building the container you can run it with the following command</p>
-<p><tt>docker run -it -v \&lt;project-directory\&gt;:/root/metron ansible-docker:2.0.0.2 bash</tt></p>
-<p>If you are going to build metron multiple times, you may want to map the /root/.m2 maven repo from outside of the container so that you don&#x2019;t start with an empty repo every build and have to download the world.</p>
-<p><tt>docker run -it -v \&lt;project-directory\&gt;:/root/metron -v \&lt;your .m2 directory\&gt;:/root/.m2 ansible-docker:2.0.0.2 bash</tt></p>
-<p>After running the container:</p>
-
-<ol style="list-style-type: decimal">
-  
-<li>cd /root/metron</li>
-  
-<li>run build commands, for example:
-  
-<ul>
-    
-<li>build metron without tests : <tt>mvn clean package -DskipTests</tt></li>
-    
-<li>build metron and build the rpms as well : <tt>mvn clean install &amp;&amp; cd metron-deployment &amp;&amp; mvn package -P build-rpms</tt></li>
-  </ul></li>
-</ol>
+<h3><a name="Notes"></a>Notes</h3>
 <p>If you wish to use this build with a vagrant instance, then after building with rpms as above, modify your usual vagrant up command to skip the build role, as so:</p>
-<p><tt>vagrant --ansible-skip-tags=&quot;build&quot; up</tt></p></div>
-<div class="section">
-<h2><a name="Using_the_Container_for_deployment"></a>Using the Container for deployment</h2>
-
-<blockquote>
-<p>Note these instructions are outdated</p>
-</blockquote>
-<p>Full instructions are found on the wiki at <a class="externalLink" href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144361">https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144361</a></p></div>
-                  </div>
-            </div>
-          </div>
 
+<div>
+<div>
+<pre class="source">  vagrant --ansible-skip-tags=&quot;build&quot; up
+</pre></div></div></div></div>
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/packaging/docker/deb-docker/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/packaging/docker/deb-docker/index.html b/site/current-book/metron-deployment/packaging/docker/deb-docker/index.html
new file mode 100644
index 0000000..a83a363
--- /dev/null
+++ b/site/current-book/metron-deployment/packaging/docker/deb-docker/index.html
@@ -0,0 +1,194 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/docker/deb-docker/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; </title>
+    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../../../css/site.css" />
+    <link rel="stylesheet" href="../../../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active "></li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<p>This project builds packages that allow you to install Apache Metron on an APT-based operating system like Ubuntu.</p>
+<p>If you are installing Metron using Ambari, these packages are necessary prerequisites when installing on an APT-based platform like Ubuntu.  Installing Metron using <b>only</b> these packages still leaves a considerable amount of configuration necessary to get Metron running.  Installing with Ambari automates these additional steps.</p>
+<p><b>WARNING</b>: These packages are a recent addition to Metron.  These have not undergone the same level of testing as the RPM packages.  Improvements and more rigorous testing of these packages is underway and will improve in future releases.  Until then, use these at your own risk.</p>
+<div class="section">
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<ul>
+
+<li>Docker: Please ensure that Docker is installed and that the daemon is running.</li>
+</ul></div>
+<div class="section">
+<h3><a name="Quick_Start"></a>Quick Start</h3>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Execute the following command from the project&#x2019;s root directory.</p>
+
+<div>
+<div>
+<pre class="source">mvn clean package -DskipTests -Pbuild-debs
+</pre></div></div>
+</li>
+<li>
+
+<p>The packages will be accessible from the following location once the build process completes.</p>
+
+<div>
+<div>
+<pre class="source">metron-deployment/packaging/docker/deb-docker/target
+</pre></div></div>
+</li>
+</ol></div>
+<div class="section">
+<h3><a name="Build_Packages"></a>Build Packages</h3>
+<p>If Metron has already been built, just the DEB packages can be built by executing the following commands.</p>
+
+<div>
+<div>
+<pre class="source">    cd metron-deployment
+    mvn clean package -Pbuild-debs
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="How_does_this_work.3F"></a>How does this work?</h3>
+<p>Using the <tt>build-debs</tt> profile as shown above effectively automates the following steps.</p>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Copy the tarball for each Metron sub-project to the <tt>target</tt> working directory.</p>
+</li>
+<li>
+
+<p>Build a Docker image of a Ubuntu Trusty host called <tt>docker-deb</tt> that contains all of the tools needed to build the packages.</p>
+
+<div>
+<div>
+<pre class="source">docker build -t deb-docker .
+</pre></div></div>
+</li>
+<li>
+
+<p>Execute the <tt>build.sh</tt> script within a Docker container.  The argument passed to the build script is the current version of Metron.</p>
+
+<div>
+<div>
+<pre class="source">docker run -v `pwd`:/root deb-docker:latest /bin/bash -c ./build.sh &lt;metron-version&gt;
+</pre></div></div>
+</li>
+<li>
+
+<p>This results in the DEBs being generated within the <tt>target</tt> directory.</p>
+</li>
+</ol></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html b/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html
index 53b2885..d684c0d 100644
--- a/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html
+++ b/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html
@@ -1,294 +1,198 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/docker/rpm-docker/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Overview</title>
-    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.3.0.min.css" />
+    <title>Metron &#x2013; </title>
+    <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../../../css/site.css" />
     <link rel="stylesheet" href="../../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Overview</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active "></li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Rpm-docker</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Overview</h1>
-<p><a name="Overview"></a></p>
-<p>The Metron ansible-docker container is provided in an effort reduce the installation burden of deploying Metron in a live envirionment. It is provisioned with software required to sucessfully run the deployment scripts.</p>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<p>This project provides RPM packages that allow you to install Metron on an RPM-based operating system like CentOS.</p>
+<p>If you are installing Metron using Ambari, these packages are necessary prerequisites when installing on an RPM-based platform like CentOS.  Installing Metron using <b>only</b> these packages still leaves a considerable amount of configuration necessary to get Metron running.  Installing with Ambari automates these additional steps.</p>
 <div class="section">
-<h2><a name="Building_the_Container"></a>Building the Container</h2>
+<div class="section">
+<h3><a name="Prerequisites"></a>Prerequisites</h3>
+<ul>
 
+<li>Docker: Please ensure that Docker is installed and that the daemon is running.</li>
+</ul></div>
+<div class="section">
+<h3><a name="Quick_Start"></a>Quick Start</h3>
 <ol style="list-style-type: decimal">
-  
-<li>Install Docker ( <a class="externalLink" href="https://www.docker.com/products/overview">https://www.docker.com/products/overview</a> )</li>
-  
-<li>Navigate to &lt;project-directory&gt;/metron-deployment/packaging/rpm-docker</li>
-  
-<li>Build the container <tt>docker build -t rpm-docker .</tt></li>
+
+<li>
+
+<p>Execute the following command from the project&#x2019;s root directory.</p>
+
+<div>
+<div>
+<pre class="source">mvn clean package -DskipTests -Pbuild-rpms
+</pre></div></div>
+</li>
+<li>
+
+<p>The packages will be accessible from the following location once the build process completes.</p>
+
+<div>
+<div>
+<pre class="source">metron-deployment/packaging/docker/rpm-docker/RPMS/noarch
+</pre></div></div>
+</li>
 </ol></div>
-                  </div>
-            </div>
-          </div>
+<div class="section">
+<h3><a name="Build_Packages"></a>Build Packages</h3>
+<p>If Metron has already been built, just the RPM packages can be built by executing the following commands.</p>
 
-    <hr/>
+<div>
+<div>
+<pre class="source">  cd metron-deployment
+  mvn clean package -Pbuild-debs
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="How_does_this_work.3F"></a>How does this work?</h3>
+<p>Using the <tt>build-rpms</tt> profile as shown above, effectively automates the following steps.</p>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p>Copy the tarball for each Metron sub-project to the <tt>target</tt> working directory.</p>
+</li>
+<li>
+
+<p>Build a Docker image of a CentOS host called <tt>rpm-docker</tt> that contains all of the tools needed to build the packages.</p>
+
+<div>
+<div>
+<pre class="source">docker build -t rpm-docker .
+</pre></div></div>
+</li>
+<li>
 
+<p>Execute the <tt>build.sh</tt> script within the Docker container.  The argument passed to the build script is the current version of Metron.</p>
+
+<div>
+<div>
+<pre class="source">docker run -v `pwd`:/root rpm-docker:latest /bin/bash -c ./build.sh &lt;metron-version&gt;
+</pre></div></div>
+</li>
+<li>
+
+<p>This results in the RPMs being generated within the following directory.</p>
+
+<div>
+<div>
+<pre class="source">metron-deployment/packaging/docker/rpm-docker/RPMS/noarch
+</pre></div></div>
+</li>
+</ol></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/packaging/packer-build/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/packaging/packer-build/index.html b/site/current-book/metron-deployment/packaging/packer-build/index.html
index 066073c..fb4b71b 100644
--- a/site/current-book/metron-deployment/packaging/packer-build/index.html
+++ b/site/current-book/metron-deployment/packaging/packer-build/index.html
@@ -1,334 +1,172 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/packer-build/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Build Metron Images</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../../css/site.css" />
     <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Build Metron Images</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Build Metron Images</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Packer-build</a>
-          </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Build Metron Images</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Build Metron Images</h1>
 <p>Based on the fantastic <a class="externalLink" href="https://github.com/chef/bento">Bento</a> project developed by Chef.</p>
 <div class="section">
 <h2><a name="Images_Provided"></a>Images Provided</h2>
-
 <ul>
-  
+
 <li>base-centos-6.7: Centos 6.7 + HDP. Used in the full-dev-platform Vagrant image</li>
 </ul></div>
 <div class="section">
 <h2><a name="Prerequisites"></a>Prerequisites</h2>
-
 <ul>
-  
+
 <li><a class="externalLink" href="https://www.packer.io/">Packer</a> 0.12.2</li>
-  
 <li><a class="externalLink" href="https://www.virtualbox.org/">Virtualbox</a> 5.0.16+ (Tested with 5.0.20)</li>
 </ul></div>
 <div class="section">
 <h2><a name="Build_Both_Images"></a>Build Both Images</h2>
-<p>Navigate to &lt;your-project-directory&gt;/metron-deployment/packer-build  Execute bin/bento build</p>
+<p>Navigate to &lt;your-project-directory&gt;/metron-deployment/packer-build Execute bin/bento build</p>
 <p>Packer will build both images and export .box files to the ./builds directory.</p></div>
 <div class="section">
 <h2><a name="Build_Single_Images"></a>Build Single Images</h2>
 <p>Navigate to <i>your-project-directory</i>/metron-deployment/packer-build</p>
-
 <ul>
-  
+
 <li>Base Centos (full-dev)</li>
 </ul>
 
-<div class="source">
-<div class="source">
-<pre>bin/bento build base-centos-6.7.json
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">bin/bento build base-centos-6.7.json
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Using_Your_New_Box_File"></a>Using Your New Box File</h2>
 <p>Modify the relevant Vagrantfile (full-dev-platform) replacing the lines:</p>
 
-<div class="source">
-<div class="source">
-<pre>&lt;pre&gt;&lt;code&gt;config.vm.box = &quot;&lt;i&gt;box_name&lt;/i&gt;&quot;
+<div>
+<div>
+<pre class="source">&lt;pre&gt;&lt;code&gt;config.vm.box = &quot;&lt;i&gt;box_name&lt;/i&gt;&quot;
 config.ssh.insert_key = true&lt;/code&gt;&lt;/pre&gt;
 </pre></div></div>
+
 <p>with</p>
 
-<div class="source">
-<div class="source">
-<pre>&lt;pre&gt;&lt;/code&gt;config.vm.box = &quot;&lt;i&gt;test_box_name&lt;/i&gt;&quot;
+<div>
+<div>
+<pre class="source">&lt;pre&gt;&lt;/code&gt;config.vm.box = &quot;&lt;i&gt;test_box_name&lt;/i&gt;&quot;
 config.vm.box = &quot;&lt;i&gt;PathToBoxfile/Boxfilename&lt;/i&gt;&quot;
 config.ssh.insert_key = true&lt;/code&gt;&lt;/pre&gt;
 </pre></div></div>
+
 <p>Launch the image as usual.</p>
 <p>Node: Vagrant will cache boxes, you can force Vagrant to reload your box by running <tt>vagrant box remove <i>test_box_name</i></tt> before launching your new image.</p></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[12/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-sensors/bro-plugin-kafka/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-sensors/bro-plugin-kafka/index.html b/site/current-book/metron-sensors/bro-plugin-kafka/index.html
deleted file mode 100644
index 638fdcd..0000000
--- a/site/current-book/metron-sensors/bro-plugin-kafka/index.html
+++ /dev/null
@@ -1,519 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Logging Bro Output to Kafka</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../css/site.css" />
-    <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Logging Bro Output to Kafka</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                                                                                
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-down"></i>
-        Sensors</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Bro-plugin-kafka</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-sensors/fastcapa/index.html" title="Fastcapa">
-          <i class="none"></i>
-        Fastcapa</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-sensors/pycapa/index.html" title="Pycapa">
-          <i class="none"></i>
-        Pycapa</a>
-            </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Logging Bro Output to Kafka</h1>
-<p>A Bro log writer that sends logging output to Kafka. This provides a convenient means for tools in the Hadoop ecosystem, such as Storm, Spark, and others, to process the data generated by Bro.</p>
-<div class="section">
-<h2><a name="Installation"></a>Installation</h2>
-
-<ol style="list-style-type: decimal">
-  
-<li>
-<p>Install <a class="externalLink" href="https://github.com/edenhill/librdkafka">librdkafka</a>, a native client library for Kafka. This plugin has been tested against the latest release of librdkafka, which at the time of this writing is v0.9.4.</p>
-<p>In order to use this plugin within a kerberized Kafka environment, you will also need <tt>libsasl2</tt> installed and will need to pass <tt>--enable-sasl</tt> to the <tt>configure</tt> script.</p>
-  
-<div class="source">
-<div class="source">
-<pre>curl -L https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz | tar xvz
-cd librdkafka-0.9.4/
-./configure --enable-sasl
-make
-sudo make install
-</pre></div></div></li>
-  
-<li>
-<p>Build the plugin using the following commands.</p>
-  
-<div class="source">
-<div class="source">
-<pre>./configure --bro-dist=$BRO_SRC
-make
-sudo make install
-</pre></div></div></li>
-  
-<li>
-<p>Run the following command to ensure that the plugin was installed successfully.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ bro -N Bro::Kafka
-Bro::Kafka - Writes logs to Kafka (dynamic, version 0.1)
-</pre></div></div></li>
-</ol></div>
-<div class="section">
-<h2><a name="Activation"></a>Activation</h2>
-<p>The following examples highlight different ways that the plugin can be used. Simply add the Bro script language to your <tt>local.bro</tt> file (for example, <tt>/usr/share/bro/site/local.bro</tt>) as shown to demonstrate the example.</p>
-<div class="section">
-<h3><a name="Example_1"></a>Example 1</h3>
-<p>The goal in this example is to send all HTTP and DNS records to a Kafka topic named <tt>bro</tt>. </p>
-
-<ul>
-  
-<li>Any configuration value accepted by librdkafka can be added to the <tt>kafka_conf</tt> configuration table.</li>
-  
-<li>By defining <tt>topic_name</tt> all records will be sent to the same Kafka topic.</li>
-  
-<li>Defining <tt>logs_to_send</tt> will ensure that only HTTP and DNS records are sent.</li>
-</ul>
-
-<div class="source">
-<div class="source">
-<pre>@load Bro/Kafka/logs-to-kafka.bro
-redef Kafka::logs_to_send = set(HTTP::LOG, DNS::LOG);
-redef Kafka::topic_name = &quot;bro&quot;;
-redef Kafka::kafka_conf = table(
-    [&quot;metadata.broker.list&quot;] = &quot;localhost:9092&quot;
-);
-</pre></div></div></div>
-<div class="section">
-<h3><a name="Example_2"></a>Example 2</h3>
-<p>It is also possible to send each log stream to a uniquely named topic. The goal in this example is to send all HTTP records to a Kafka topic named <tt>http</tt> and all DNS records to a separate Kafka topic named <tt>dns</tt>.</p>
-
-<ul>
-  
-<li>The <tt>topic_name</tt> value must be set to an empty string.</li>
-  
-<li>The <tt>$path</tt> value of Bro&#x2019;s Log Writer mechanism is used to define the topic name.</li>
-  
-<li>Any configuration value accepted by librdkafka can be added to the <tt>$config</tt> configuration table.</li>
-  
-<li>Each log writer accepts a separate configuration table.</li>
-</ul>
-
-<div class="source">
-<div class="source">
-<pre>@load Bro/Kafka/logs-to-kafka.bro
-redef Kafka::topic_name = &quot;&quot;;
-redef Kafka::tag_json = T;
-
-event bro_init()
-{
-    # handles HTTP
-    local http_filter: Log::Filter = [
-        $name = &quot;kafka-http&quot;,
-        $writer = Log::WRITER_KAFKAWRITER,
-        $config = table(
-                [&quot;metadata.broker.list&quot;] = &quot;localhost:9092&quot;
-        ),
-        $path = &quot;http&quot;
-    ];
-    Log::add_filter(HTTP::LOG, http_filter);
-
-    # handles DNS
-    local dns_filter: Log::Filter = [
-        $name = &quot;kafka-dns&quot;,
-        $writer = Log::WRITER_KAFKAWRITER,
-        $config = table(
-                [&quot;metadata.broker.list&quot;] = &quot;localhost:9092&quot;
-        ),
-        $path = &quot;dns&quot;
-    ];
-    Log::add_filter(DNS::LOG, dns_filter);
-}
-</pre></div></div></div>
-<div class="section">
-<h3><a name="Example_3"></a>Example 3</h3>
-<p>You may want to configure bro to filter log messages with certain characteristics from being sent to your kafka topics. For instance, Metron currently doesn&#x2019;t support IPv6 source or destination IPs in the default enrichments, so it may be helpful to filter those log messages from being sent to kafka (although there are <a href="#notes">multiple ways</a> to approach this). In this example we will do that that, and are assuming a somewhat standard bro kafka plugin configuration, such that:</p>
-
-<ul>
-  
-<li>All bro logs are sent to the <tt>bro</tt> topic, by configuring <tt>Kafka::topic_name</tt>.</li>
-  
-<li>Each JSON message is tagged with the appropriate log type (such as <tt>http</tt>, <tt>dns</tt>, or <tt>conn</tt>), by setting <tt>tag_json</tt> to true.</li>
-  
-<li>If the log message contains a 128 byte long source or destination IP address, the log is not sent to kafka.</li>
-</ul>
-
-<div class="source">
-<div class="source">
-<pre>@load Bro/Kafka/logs-to-kafka.bro
-redef Kafka::topic_name = &quot;bro&quot;;
-redef Kafka::tag_json = T;
-
-event bro_init() &amp;priority=-5
-{
-    # handles HTTP
-    Log::add_filter(HTTP::LOG, [
-        $name = &quot;kafka-http&quot;,
-        $writer = Log::WRITER_KAFKAWRITER,
-        $pred(rec: HTTP::Info) = { return ! (( |rec$id$orig_h| == 128 || |rec$id$resp_h| == 128 )); },
-        $config = table(
-            [&quot;metadata.broker.list&quot;] = &quot;localhost:9092&quot;
-        )
-    ]);
-
-    # handles DNS
-    Log::add_filter(DNS::LOG, [
-        $name = &quot;kafka-dns&quot;,
-        $writer = Log::WRITER_KAFKAWRITER,
-        $pred(rec: DNS::Info) = { return ! (( |rec$id$orig_h| == 128 || |rec$id$resp_h| == 128 )); },
-        $config = table(
-            [&quot;metadata.broker.list&quot;] = &quot;localhost:9092&quot;
-        )
-    ]);
-
-    # handles Conn
-    Log::add_filter(Conn::LOG, [
-        $name = &quot;kafka-conn&quot;,
-        $writer = Log::WRITER_KAFKAWRITER,
-        $pred(rec: Conn::Info) = { return ! (( |rec$id$orig_h| == 128 || |rec$id$resp_h| == 128 )); },
-        $config = table(
-            [&quot;metadata.broker.list&quot;] = &quot;localhost:9092&quot;
-        )
-    ]);
-}
-</pre></div></div>
-<div class="section">
-<h4><a name="Notes"></a>Notes</h4>
-
-<ul>
-  
-<li><tt>logs_to_send</tt> is mutually exclusive with <tt>$pred</tt>, thus for each log you want to set <tt>$pred</tt> on, you must individually setup a <tt>Log::add_filter</tt> and refrain from including that log in <tt>logs_to_send</tt>.</li>
-  
-<li>You can also filter IPv6 logs from within your Metron cluster <a href="../../metron-stellar/stellar-common/index.html#IS_IP">using Stellar</a>. In that case, you wouldn&#x2019;t apply a predicate in your bro configuration, and instead Stellar would filter the logs out before they were processed by the enrichment layer of Metron.</li>
-  
-<li>It is also possible to use the <tt>is_v6_subnet()</tt> bro function in your predicate, as of their <a class="externalLink" href="https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-5">2.5 release</a>, however the above example should work on <a class="externalLink" href="https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-4">bro 2.4</a> and newer, which has been the focus of the kafka plugin.</li>
-</ul></div></div></div>
-<div class="section">
-<h2><a name="Settings"></a>Settings</h2>
-<div class="section">
-<h3><a name="kafka_conf"></a><tt>kafka_conf</tt></h3>
-<p>The global configuration settings for Kafka. These values are passed through directly to librdkafka. Any valid librdkafka settings can be defined in this table. The full set of valid librdkafka settings are available <a class="externalLink" href="https://github.com/edenhill/librdkafka/blob/v0.9.4/CONFIGURATION.md">here</a>.</p>
-
-<div class="source">
-<div class="source">
-<pre>redef Kafka::kafka_conf = table(
-    [&quot;metadata.broker.list&quot;] = &quot;localhost:9092&quot;,
-    [&quot;client.id&quot;] = &quot;bro&quot;
-);
-</pre></div></div></div>
-<div class="section">
-<h3><a name="topic_name"></a><tt>topic_name</tt></h3>
-<p>The name of the topic in Kafka where all Bro logs will be sent to.</p>
-
-<div class="source">
-<div class="source">
-<pre>redef Kafka::topic_name = &quot;bro&quot;;
-</pre></div></div></div>
-<div class="section">
-<h3><a name="max_wait_on_shutdown"></a><tt>max_wait_on_shutdown</tt></h3>
-<p>The maximum number of milliseconds that the plugin will wait for any backlog of queued messages to be sent to Kafka before forced shutdown.</p>
-
-<div class="source">
-<div class="source">
-<pre>redef Kafka::max_wait_on_shutdown = 3000;
-</pre></div></div></div>
-<div class="section">
-<h3><a name="tag_json"></a><tt>tag_json</tt></h3>
-<p>If true, a log stream identifier is appended to each JSON-formatted message. For example, a Conn::LOG message will look like <tt>{ 'conn' : { ... }}</tt>.</p>
-
-<div class="source">
-<div class="source">
-<pre>redef Kafka::tag_json = T;
-</pre></div></div></div>
-<div class="section">
-<h3><a name="debug"></a><tt>debug</tt></h3>
-<p>A comma separated list of debug contexts in librdkafka which you want to enable. The available contexts are:</p>
-
-<ul>
-  
-<li>generic</li>
-  
-<li>broker</li>
-  
-<li>topic</li>
-  
-<li>metadata</li>
-  
-<li>queue</li>
-  
-<li>msg</li>
-  
-<li>protocol</li>
-  
-<li>cgrp</li>
-  
-<li>security</li>
-  
-<li>fetch</li>
-  
-<li>feature</li>
-  
-<li>all</li>
-</ul></div></div>
-<div class="section">
-<h2><a name="Kerberos"></a>Kerberos</h2>
-<p>This plugin supports producing messages from a kerberized kafka. There are a couple of prerequisites and a couple of settings to set. </p>
-<div class="section">
-<h3><a name="SASL"></a>SASL</h3>
-<p>If you are using SASL as a security protocol for kafka, then you must have libsasl or libsasl2 installed. You can tell if sasl is enabled by running the following from the directory in which you have build librdkafka:</p>
-
-<div class="source">
-<div class="source">
-<pre>examples/rdkafka_example -X builtin.features
-builtin.features = gzip,snappy,ssl,sasl,regex
-</pre></div></div></div>
-<div class="section">
-<h3><a name="Producer_Config"></a>Producer Config</h3>
-<p>As stated above, you can configure the producer kafka configs in <tt>${BRO_HOME}/share/bro/site/local.bro</tt>. There are a few configs necessary to set, which are described <a class="externalLink" href="https://github.com/edenhill/librdkafka/wiki/Using-SASL-with-librdkafka">here</a>. For an environment where the following is true:</p>
-
-<ul>
-  
-<li>The broker is <tt>node1:6667</tt></li>
-  
-<li>This kafka is using <tt>SASL_PLAINTEXT</tt> as the security protocol</li>
-  
-<li>The keytab used is the <tt>metron</tt> keytab</li>
-  
-<li>The service principal for <tt>metron</tt> is <tt>metron@EXAMPLE.COM</tt></li>
-</ul>
-<p>The kafka topic <tt>bro</tt> has been given permission for the <tt>metron</tt> user to write:</p>
-
-<div class="source">
-<div class="source">
-<pre># login using the metron user
-kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM
-${KAFKA_HOME}/kafka-broker/bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=node1:2181 --add --allow-principal User:metron --topic bro
-</pre></div></div>
-<p>The following is how the <tt>${BRO_HOME}/share/bro/site/local.bro</tt> looks:</p>
-
-<div class="source">
-<div class="source">
-<pre>@load Bro/Kafka/logs-to-kafka.bro
-redef Kafka::logs_to_send = set(HTTP::LOG, DNS::LOG);
-redef Kafka::topic_name = &quot;bro&quot;;
-redef Kafka::tag_json = T;
-redef Kafka::kafka_conf = table( [&quot;metadata.broker.list&quot;] = &quot;node1:6667&quot;
-                               , [&quot;security.protocol&quot;] = &quot;SASL_PLAINTEXT&quot;
-                               , [&quot;sasl.kerberos.keytab&quot;] = &quot;/etc/security/keytabs/metron.headless.keytab&quot;
-                               , [&quot;sasl.kerberos.principal&quot;] = &quot;metron@EXAMPLE.COM&quot;
-                               );
-</pre></div></div></div></div>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>


[15/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-management/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-management/index.html b/site/current-book/metron-platform/metron-management/index.html
index cca14f1..f9ea1ce 100644
--- a/site/current-book/metron-platform/metron-management/index.html
+++ b/site/current-book/metron-platform/metron-management/index.html
@@ -1,1040 +1,644 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-management/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Stellar REPL Management Utilities</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Stellar REPL Management Utilities</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Stellar REPL Management Utilities</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Management</a>
-          </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Stellar REPL Management Utilities</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Stellar REPL Management Utilities</h1>
 <p><a name="Stellar_REPL_Management_Utilities"></a></p>
 <p>In order to augment the functionality of the Stellar REPL, a few management functions surrounding the management of the configurations and the management of Stellar transformations in the following areas have been added:</p>
-
 <ul>
-  
+
 <li>Stellar field transformations in the Parsers</li>
-  
 <li>Stellar enrichments in the Enrichment topology</li>
-  
 <li>Stellar threat triage rules</li>
 </ul>
-<p>Additionally, some shell functions have been added to </p>
-
+<p>Additionally, some shell functions have been added to</p>
 <ul>
-  
+
 <li>provide the ability to refer to the Stellar expression used to create a variable</li>
-  
 <li>print structured data in a way that is easier to view (i.e. tabular)</li>
 </ul>
 <p>This functionality is exposed as a pack of Stellar functions in this project.</p>
-
 <ul>
-  
+
 <li><a href="#Functions">Functions</a>
-  
 <ul>
-    
+
 <li><a href="#Grok_Functions">Grok Functions</a></li>
-    
 <li><a href="#File_Functions">File Functions</a></li>
-    
-<li><a href="#Shell_Functions">Shell Functions</a></li>
-    
 <li><a href="#Configuration_Functions">Configuration Functions</a></li>
-    
 <li><a href="#Parser_Functions">Parser Functions</a></li>
-    
 <li><a href="#Indexing_Functions">Indexing Functions</a></li>
-    
 <li><a href="#Enrichment_Functions">Enrichment Functions</a></li>
-    
 <li><a href="#Threat_Triage_Functions">Threat Triage Functions</a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><a href="#Examples">Examples</a>
-  
 <ul>
-    
+
 <li><a href="#Iterate_to_Find_a_Valid_Grok_Pattern">Iterate to Find a Valid Grok Pattern</a></li>
-    
 <li><a href="#Manage_Stellar_Field_Transformations">Manage Stellar Field Transformations</a></li>
-    
 <li><a href="#Manage_Stellar_Enrichments">Manage Stellar Enrichments</a></li>
-    
 <li><a href="#Manage_Threat_Triage_Rules">Manage Threat Triage Rules</a></li>
-    
 <li><a href="#Simulate_Threat_Triage_Rules">Simulate Threat Triage Rules</a></li>
-  </ul></li>
+</ul>
+</li>
 </ul>
 <div class="section">
 <h2><a name="Functions"></a>Functions</h2>
 <p>The functions are split roughly into a few sections:</p>
-
 <ul>
-  
+
 <li>Shell functions - Functions surrounding interacting with the shell in either a nicer way or a more functional way.</li>
-  
 <li>Grok Functions - Functions that allow you to evaluate grok expressions.</li>
-  
 <li>File functions - Functions around interacting with local or HDFS files</li>
-  
 <li>Configuration functions - Functions surrounding pulling and pushing configs from zookeeper</li>
-  
 <li>Parser functions - Functions surrounding adding, viewing, and removing Parser functions.</li>
-  
 <li>Enrichment functions - Functions surrounding adding, viewing and removing Stellar enrichments as well as managing batch size, batch timeout, and index names for the enrichment topology configuration</li>
-  
 <li>Threat Triage functions - Functions surrounding adding, viewing and removing threat triage functions.</li>
 </ul>
 <div class="section">
 <h3><a name="Grok_Functions"></a>Grok Functions</h3>
-
 <ul>
-  
+
 <li><tt>GROK_EVAL</tt>
-  
 <ul>
-    
+
 <li>Description: Evaluate a grok expression for a statement.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>grokExpression - The grok expression to evaluate</li>
-      
 <li>data - Either a data message or a list of data messages to evaluate using the grokExpression</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The Map associated with the grok expression being evaluated on the list of messages.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>GROK_PREDICT</tt>
-  
 <ul>
-    
+
 <li>Description: Discover a grok statement for an input doc</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>data - The data to discover a grok expression from</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: A grok expression that should match the data.</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h3><a name="File_Functions"></a>File Functions</h3>
-
 <ul>
-  
+
 <li>Local Files
-  
 <ul>
-    
+
 <li><tt>LOCAL_LS</tt>
-    
 <ul>
-      
+
 <li>Description: Lists the contents of a directory.</li>
-      
 <li>Input:
-      
 <ul>
-        
+
 <li>path - The path of the file</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li>Returns: The contents of the directory in tabular form sorted by last modification date.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li><tt>LOCAL_RM</tt>
-    
 <ul>
-      
+
 <li>Description: Removes the path</li>
-      
 <li>Input:
-      
 <ul>
-        
+
 <li>path - The path of the file or directory.</li>
-        
 <li>recursive - Recursively remove or not (optional and defaulted to false)</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li>Returns: boolean - true if successful, false otherwise</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li><tt>LOCAL_READ</tt>
-    
 <ul>
-      
+
 <li>Description: Retrieves the contents as a string of a file.</li>
-      
 <li>Input:
-      
 <ul>
-        
+
 <li>path - The path of the file</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li>Returns: The contents of the file and null otherwise.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li><tt>LOCAL_READ_LINES</tt>
-    
 <ul>
-      
+
 <li>Description: Retrieves the contents of a file as a list of strings.</li>
-      
 <li>Input:
-      
 <ul>
-        
+
 <li>path - The path of the file</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li>Returns: A list of lines</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li><tt>LOCAL_WRITE</tt>
-    
 <ul>
-      
+
 <li>Description: Writes the contents of a string to a local file</li>
-      
 <li>Input:
-      
 <ul>
-        
+
 <li>content - The content to write out</li>
-        
 <li>path - The path of the file</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li>Returns: true if the file was written and false otherwise.</li>
-    </ul></li>
-  </ul></li>
-  
+</ul>
+</li>
+</ul>
+</li>
 <li>HDFS Files
-  
 <ul>
-    
+
 <li><tt>HDFS_LS</tt>
-    
 <ul>
-      
+
 <li>Description: Lists the contents of a directory in HDFS.</li>
-      
 <li>Input:
-      
 <ul>
-        
+
 <li>path - The path of the file</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li>Returns: The contents of the directory in tabular form sorted by last modification date.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li><tt>HDFS_RM</tt>
-    
 <ul>
-      
+
 <li>Description: Removes the path in HDFS.</li>
-      
 <li>Input:
-      
 <ul>
-        
+
 <li>path - The path of the file or directory.</li>
-        
 <li>recursive - Recursively remove or not (optional and defaulted to false)</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li>Returns: boolean - true if successful, false otherwise</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li><tt>HDFS_READ</tt>
-    
 <ul>
-      
+
 <li>Description: Retrieves the contents as a string of a file in HDFS.</li>
-      
 <li>Input:
-      
 <ul>
-        
+
 <li>path - The path of the file</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li>Returns: The contents of the file and null otherwise.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li><tt>HDFS_READ_LINES</tt>
-    
 <ul>
-      
+
 <li>Description: Retrieves the contents of a HDFS file as a list of strings.</li>
-      
 <li>Input:
-      
 <ul>
-        
+
 <li>path - The path of the file</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li>Returns: A list of lines</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li><tt>HDFS_WRITE</tt>
-    
 <ul>
-      
+
 <li>Description: Writes the contents of a string to a HDFS file</li>
-      
 <li>Input:
-      
 <ul>
-        
+
 <li>content - The content to write out</li>
-        
 <li>path - The path of the file</li>
-      </ul></li>
-      
+</ul>
+</li>
 <li>Returns: true if the file was written and false otherwise.</li>
-    </ul></li>
-  </ul></li>
-</ul></div>
-<div class="section">
-<h3><a name="Shell_Functions"></a>Shell Functions</h3>
-
-<ul>
-  
-<li><tt>SHELL_EDIT</tt>
-  
-<ul>
-    
-<li>Description: Open an editor (optionally initialized with text) and return whatever is saved from the editor. The editor to use is pulled from <tt>EDITOR</tt> or <tt>VISUAL</tt> environment variable.</li>
-    
-<li>Input:
-    
-<ul>
-      
-<li>string - (Optional) A string whose content is used to initialize the editor.</li>
-    </ul></li>
-    
-<li>Returns: The content that the editor saved after editor exit.</li>
-  </ul></li>
-  
-<li><tt>SHELL_GET_EXPRESSION</tt>
-  
-<ul>
-    
-<li>Description: Get a stellar expression from a variable</li>
-    
-<li>Input:
-    
-<ul>
-      
-<li>variable - variable name</li>
-    </ul></li>
-    
-<li>Returns: The stellar expression associated with the variable.</li>
-  </ul></li>
-  
-<li><tt>SHELL_LIST_VARS</tt>
-  
-<ul>
-    
-<li>Description: Return the variables in a tabular form</li>
-    
-<li>Input:
-    
-<ul>
-      
-<li>wrap : Length of string to wrap the columns</li>
-    </ul></li>
-    
-<li>Returns: A tabular representation of the variables.</li>
-  </ul></li>
-  
-<li><tt>SHELL_MAP2TABLE</tt>
-  
-<ul>
-    
-<li>Description: Take a map and return a table</li>
-    
-<li>Input:
-    
-<ul>
-      
-<li>map - Map</li>
-    </ul></li>
-    
-<li>Returns: The map in table form</li>
-  </ul></li>
-  
-<li><tt>SHELL_VARS2MAP</tt>
-  
-<ul>
-    
-<li>Description: Take a set of variables and return a map</li>
-    
-<li>Input:
-    
-<ul>
-      
-<li>variables* - variable names to use to create map</li>
-    </ul></li>
-    
-<li>Returns: A map associating the variable name with the stellar expression.</li>
-  </ul></li>
+</ul>
+</li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h3><a name="Configuration_Functions"></a>Configuration Functions</h3>
-
 <ul>
-  
+
 <li><tt>CONFIG_GET</tt>
-  
 <ul>
-    
+
 <li>Description: Retrieve a Metron configuration from zookeeper.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>type - One of ENRICHMENT, INDEXING, PARSER, GLOBAL, PROFILER</li>
-      
 <li>sensor - Sensor to retrieve (required for enrichment and parser, not used for profiler and global)</li>
-      
 <li>emptyIfNotPresent - If true, then return an empty, minimally viable config</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the config in zookeeper</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>CONFIG_PUT</tt>
-  
 <ul>
-    
+
 <li>Description: Updates a Metron config to Zookeeper.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>type - One of ENRICHMENT, INDEXING, PARSER, GLOBAL, PROFILER</li>
-      
 <li>config - The config (a string in JSON form) to update</li>
-      
 <li>sensor - Sensor to retrieve (required for enrichment and parser, not used for profiler and global)</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the config in zookeeper</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h3><a name="Parser_Functions"></a>Parser Functions</h3>
-
 <ul>
-  
+
 <li><tt>PARSER_STELLAR_TRANSFORM_ADD</tt>
-  
 <ul>
-    
+
 <li>Description: Add stellar field transformation.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
 <li>stellarTransforms - A Map associating fields to stellar expressions</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the config in zookeeper</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>PARSER-STELLAR_TRANSFORM_PRINT</tt>
-  
 <ul>
-    
+
 <li>Description: Retrieve stellar field transformations.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the transformations</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>PARSER_STELLAR_TRANSFORM_REMOVE</tt>
-  
 <ul>
-    
+
 <li>Description: Remove stellar field transformation.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
 <li>stellarTransforms - A list of stellar transforms to remove</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the config in zookeeper</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h3><a name="Indexing_Functions"></a>Indexing Functions</h3>
-
 <ul>
-  
+
 <li><tt>INDEXING_SET_BATCH</tt>
-  
 <ul>
-    
+
 <li>Description: Set batch size and timeout</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
 <li>writer - The writer to update (e.g. elasticsearch, solr or hdfs)</li>
-      
 <li>size - batch size (integer), defaults to 1, meaning batching disabled</li>
-      
 <li>timeout - (optional) batch timeout in seconds (integer), defaults to 0, meaning system default</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the config in zookeeper</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>INDEXING_SET_ENABLED</tt>
-  
 <ul>
-    
+
 <li>Description: Enable or disable an indexing writer for a sensor.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
 <li>writer - The writer to update (e.g. elasticsearch, solr or hdfs)</li>
-      
-<li>enabled? - boolean indicating whether the writer is enabled. If omitted, then it will set enabled.</li>
-    </ul></li>
-    
+<li>enabled? - boolean indicating whether the writer is enabled.  If omitted, then it will set enabled.</li>
+</ul>
+</li>
 <li>Returns: The String representation of the config in zookeeper</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>INDEXING_SET_INDEX</tt>
-  
 <ul>
-    
+
 <li>Description: Set the index for the sensor</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
 <li>writer - The writer to update (e.g. elasticsearch, solr or hdfs)</li>
-      
 <li>sensor - sensor name</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the config in zookeeper</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h3><a name="Enrichment_Functions"></a>Enrichment Functions</h3>
-
 <ul>
-  
+
 <li><tt>ENRICHMENT_STELLAR_TRANSFORM_ADD</tt>
-  
 <ul>
-    
+
 <li>Description: Add stellar field transformation.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
 <li>type - ENRICHMENT or THREAT_INTEL</li>
-      
 <li>stellarTransforms - A Map associating fields to stellar expressions</li>
-      
 <li>group - Group to add to (optional)</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the config in zookeeper</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>ENRICHMENT_STELLAR_TRANSFORM_PRINT</tt>
-  
 <ul>
-    
+
 <li>Description: Retrieve stellar enrichment transformations.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
 <li>type - ENRICHMENT or THREAT_INTEL</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the transformations</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>ENRICHMENT_STELLAR_TRANSFORM_REMOVE</tt>
-  
 <ul>
-    
+
 <li>Description: Remove one or more stellar field transformations.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
 <li>type - ENRICHMENT or THREAT_INTEL</li>
-      
 <li>stellarTransforms - A list of removals</li>
-      
 <li>group - Group to remove from (optional)</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the config in zookeeper</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h3><a name="Threat_Triage_Functions"></a>Threat Triage Functions</h3>
-
 <ul>
-  
+
 <li><tt>THREAT_TRIAGE_INIT</tt>
-  
 <ul>
-    
+
 <li>Description: Create a threat triage engine.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>config - the threat triage configuration (optional)</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: A threat triage engine.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>THREAT_TRIAGE_CONFIG</tt>
-  
 <ul>
-    
+
 <li>Description: Export the configuration used by a threat triage engine.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>engine - threat triage engine returned by THREAT_TRIAGE_INIT.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The configuration used by the threat triage engine.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>THREAT_TRIAGE_SCORE</tt>
-  
 <ul>
-    
+
 <li>Description: Scores a message using a set of triage rules.</li>
-    
 <li>Inputs:
-    
 <ul>
-      
+
 <li>message - a string containing the message to score.</li>
-      
 <li>engine - threat triage engine returned by THREAT_TRIAGE_INIT.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: A threat triage engine.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>THREAT_TRIAGE_ADD</tt>
-  
 <ul>
-    
+
 <li>Description: Add a threat triage rule.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
 <li>stellarTransforms - A Map associating stellar rules to scores</li>
-      
-<li>triageRules - Map (or list of Maps) representing a triage rule. It must contain &#x2018;rule&#x2019; and &#x2018;score&#x2019; keys, the stellar expression for the rule and triage score respectively. It may contain &#x2018;name&#x2019; and &#x2018;comment&#x2019;, the name of the rule and comment associated with the rule respectively.&quot;</li>
-    </ul></li>
-    
+<li>triageRules - Map (or list of Maps) representing a triage rule.  It must contain &#x2018;rule&#x2019; and &#x2018;score&#x2019; keys, the stellar expression for the rule and triage score respectively.  It may contain &#x2018;name&#x2019; and &#x2018;comment&#x2019;, the name of the rule and comment associated with the rule respectively.&quot;</li>
+</ul>
+</li>
 <li>Returns: The String representation of the threat triage rules</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>THREAT_TRIAGE_REMOVE</tt>
-  
 <ul>
-    
+
 <li>Description: Remove stellar threat triage rule(s).</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
 <li>rules - A list of stellar rules or rule names to remove</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the enrichment config</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>THREAT_TRIAGE_PRINT</tt>
-  
 <ul>
-    
+
 <li>Description: Retrieve stellar enrichment transformations.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the threat triage rules</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>THREAT_TRIAGE_SET_AGGREGATOR</tt>
-  
 <ul>
-    
+
 <li>Description: Set the threat triage aggregator.</li>
-    
 <li>Input:
-    
 <ul>
-      
+
 <li>sensorConfig - Sensor config to add transformation to.</li>
-      
-<li>aggregator - Aggregator to use. One of MIN, MAX, MEAN, SUM, POSITIVE_MEAN</li>
-      
+<li>aggregator - Aggregator to use.  One of MIN, MAX, MEAN, SUM, POSITIVE_MEAN</li>
 <li>aggregatorConfig - Optional config for aggregator</li>
-    </ul></li>
-    
+</ul>
+</li>
 <li>Returns: The String representation of the enrichment config</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div></div>
 <div class="section">
 <h2><a name="Deployment_Instructions"></a>Deployment Instructions</h2>
-
 <ul>
-  
+
 <li>Clusters installed via Ambari Management Pack (default)
-  
 <ul>
-    
+
 <li>Automatically deployed</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Manual installation
-  
 <ul>
-    
+
 <li>Deployment is as simple as dropping the jar created by this project into <tt>$METRON_HOME/lib</tt> and starting the Stellar shell via <tt>$METRON_HOME/bin/stellar</tt></li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h2><a name="Examples"></a>Examples</h2>
@@ -1042,9 +646,9 @@
 <div class="section">
 <h3><a name="Iterate_to_Find_a_Valid_Grok_pattern"></a>Iterate to Find a Valid Grok pattern</h3>
 
-<div class="source">
-<div class="source">
-<pre>Stellar, Go!
+<div>
+<div>
+<pre class="source">Stellar, Go!
 Please note that functions are loading lazily in the background and will be unavailable until loaded fully.
 [Stellar]&gt;&gt;&gt; # We are going to debug a squid grok statement with a bug in it
 [Stellar]&gt;&gt;&gt; squid_grok_orig := '%{NUMBER:timestamp} %{SPACE:UNWANTED}  %{INT:elapsed} %{IP:ip_src_addr} %{WORD:action}/%{NUMBER:code} %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} 
@@ -1117,13 +721,14 @@ WORD:UNWANTED}/(%{IP:ip_dst_addr})?'
 [Stellar]&gt;&gt;&gt; # Ahh, that is much better.
 [Stellar]&gt;&gt;&gt; 
 [Stellar]&gt;&gt;&gt; 
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Manage_Stellar_Field_Transformations"></a>Manage Stellar Field Transformations</h3>
 
-<div class="source">
-<div class="source">
-<pre>964  [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
+<div>
+<div>
+<pre class="source">964  [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
 1025 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
 Stellar, Go!
 Please note that functions are loading lazily in the background and will be unavailable until loaded fully.
@@ -1304,13 +909,14 @@ Returns: The String representation of the config in zookeeper
 }
 [Stellar]&gt;&gt;&gt; #And quit the REPL
 [Stellar]&gt;&gt;&gt; quit
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Manage_Stellar_Enrichments"></a>Manage Stellar Enrichments</h3>
 
-<div class="source">
-<div class="source">
-<pre>1010 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
+<div>
+<div>
+<pre class="source">1010 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
 1077 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
 Stellar, Go!
 Please note that functions are loading lazily in the background and will be unavailable until loaded fully.
@@ -1540,13 +1146,14 @@ Returns: The String representation of the config in zookeeper
   &quot;configuration&quot; : { }
 }
 [Stellar]&gt;&gt;&gt; 
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Manage_Threat_Triage_Rules"></a>Manage Threat Triage Rules</h3>
 
-<div class="source">
-<div class="source">
-<pre>987  [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
+<div>
+<div>
+<pre class="source">987  [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
 1047 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
 Stellar, Go!
 Please note that functions are loading lazily in the background and will be unavailable until loaded fully.
@@ -1640,7 +1247,7 @@ Returns: A Map associated with the indicator and enrichment type.  Empty otherwi
 [Stellar]&gt;&gt;&gt; non_us := whois_info.home_country != 'US'
 [Stellar]&gt;&gt;&gt; is_local := IN_SUBNET( if IS_IP(ip_src_addr) then ip_src_addr else NULL, '192.168.0.0/21')
 [Stellar]&gt;&gt;&gt; is_both := whois_info.home_country != 'US' &amp;&amp; IN_SUBNET( if IS_IP(ip_src_addr) then ip_src_addr else NULL, '192.168.0.0/21')
-[Stellar]&gt;&gt;&gt; rules := [ { 'name' : 'is non-us', 'rule' : SHELL_GET_EXPRESSION('non_us'), 'score' : 10 } , { 'name' : 'is local', 'rule' : SHELL_GET_EXPRESSION('is_local '), 'score' : 20 } , { 'name' : 'both non-us and local', 'comment' : 'union of both rules.',  'rule' : SHELL_GET_EXPRESSION('is_both'), 'score' : 50 } ]  
+[Stellar]&gt;&gt;&gt; rules := [ { 'name' : 'is non-us', 'rule' : SHELL_GET_EXPRESSION('non_us'), 'score' : 10 } , { 'name' : 'is local', 'rule' : SHELL_GET_EXPRESSION('is_local'), 'score' : 20 } , { 'name' : 'both non-us and local', 'comment' : 'union of both rules.',  'rule' : SHELL_GET_EXPRESSION('is_both'), 'score' : 50 } ]
 [Stellar]&gt;&gt;&gt; # Now that we have our rules staged, we can add them to our config.
 [Stellar]&gt;&gt;&gt; squid_enrichment_config_new := THREAT_TRIAGE_ADD( squid_enrichment_config_new, rules )
 [Stellar]&gt;&gt;&gt; THREAT_TRIAGE_PRINT(squid_enrichment_config_new)
@@ -1747,46 +1354,49 @@ SION('is_both') ] )
   &quot;configuration&quot; : { }
 }
 [Stellar]&gt;&gt;&gt; 
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Simulate_Threat_Triage_Rules"></a>Simulate Threat Triage Rules</h3>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Create a threat triage engine.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; t := THREAT_TRIAGE_INIT()
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; t := THREAT_TRIAGE_INIT()
 [Stellar]&gt;&gt;&gt; t
 ThreatTriage{0 rule(s)}
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Add a few triage rules.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_ADD(t, {&quot;name&quot;:&quot;rule1&quot;, &quot;rule&quot;:&quot;value&gt;10&quot;, 
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_ADD(t, {&quot;name&quot;:&quot;rule1&quot;, &quot;rule&quot;:&quot;value&gt;10&quot;, &quot;score&quot;:10})
 </pre></div></div>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_ADD(t, {&quot;name&quot;:&quot;rule2&quot;, &quot;rule&quot;:&quot;value&gt;20&quot;, &quot;score&quot;:20})
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_ADD(t, {&quot;name&quot;:&quot;rule2&quot;, &quot;rule&quot;:&quot;value&gt;20&quot;, &quot;score&quot;:20})
+</pre></div></div>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_ADD(t, {&quot;name&quot;:&quot;rule3&quot;, &quot;rule&quot;:&quot;value&gt;30&quot;, &quot;score&quot;:30})
 </pre></div></div>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_ADD(t, {&quot;name&quot;:&quot;rule3&quot;, &quot;rule&quot;:&quot;value&gt;30&quot;, &quot;score&quot;:30})
-</pre></div></div></li>
-  
+</li>
 <li>
+
 <p>Review the rules that you have created.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_PRINT(t)
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_PRINT(t)
 &#x2554;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2564;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2564;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2564;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2564;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2557;
 &#x2551; Name  &#x2502; Comment &#x2502; Triage Rule &#x2502; Score &#x2502; Reason &#x2551;
 &#x2560;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x256a;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x256a;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x256a;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x256a;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2563;
@@ -1796,46 +1406,49 @@ ThreatTriage{0 rule(s)}
 &#x255f;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x253c;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x253c;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x253c;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x253c;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2500;&#x2562;
 &#x2551; rule3 &#x2502;         &#x2502; value&gt;30    &#x2502; 30    &#x2502;        &#x2551;
 &#x255a;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2567;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2567;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2567;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2567;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x2550;&#x255d;
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Create a few test messages to simulate your telemetry.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; msg1 := &quot;{ \&quot;value\&quot;:22 }&quot;
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; msg1 := &quot;{ \&quot;value\&quot;:22 }&quot;
 [Stellar]&gt;&gt;&gt; msg1
 { &quot;value&quot;:22 }
 </pre></div></div>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; msg2 := &quot;{ \&quot;value\&quot;:44 }&quot;
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; msg2 := &quot;{ \&quot;value\&quot;:44 }&quot;
 [Stellar]&gt;&gt;&gt; msg2
 { &quot;value&quot;:44 }
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Score a message based on the rules that have been defined. The result allows you to see the total score, the aggregator, along with details about each rule that fired.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_SCORE( msg1, t)
+
+<p>Score a message based on the rules that have been defined.  The result allows you to see the total score, the aggregator, along with details about each rule that fired.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_SCORE( msg1, t)
 {score=20.0, aggregator=MAX, rules=[{score=10.0, name=rule1, rule=value&gt;10}, {score=20.0, name=rule2, rule=value&gt;20}]}
 </pre></div></div>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_SCORE( msg2, t)
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; THREAT_TRIAGE_SCORE( msg2, t)
 {score=30.0, aggregator=MAX, rules=[{score=10.0, name=rule1, rule=value&gt;10}, {score=20.0, name=rule2, rule=value&gt;20}, {score=30.0, name=rule3, rule=value&gt;30}]}
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>From here you can iterate on your rule set until it does exactly what you need it to do. Once you have a working rule set, extract the configuration and push it into your live, Metron cluster.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; conf := THREAT_TRIAGE_CONFIG( t)
+
+<p>From here you can iterate on your rule set until it does exactly what you need it to do.  Once you have a working rule set, extract the configuration and push it into your live, Metron cluster.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; conf := THREAT_TRIAGE_CONFIG( t)
 [Stellar]&gt;&gt;&gt; conf
 {
   &quot;enrichment&quot; : {
@@ -1868,29 +1481,24 @@ ThreatTriage{0 rule(s)}
   &quot;configuration&quot; : { }
 }
 </pre></div></div>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; CONFIG_PUT(&quot;ENRICHMENT&quot;, conf, &quot;bro&quot;)
-</pre></div></div></li>
-</ol></div></div>
-                  </div>
-            </div>
-          </div>
 
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; CONFIG_PUT(&quot;ENRICHMENT&quot;, conf, &quot;bro&quot;)
+</pre></div></div>
+</li>
+</ol></div></div>
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-parsers/3rdPartyParser.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-parsers/3rdPartyParser.html b/site/current-book/metron-platform/metron-parsers/3rdPartyParser.html
new file mode 100644
index 0000000..988580b
--- /dev/null
+++ b/site/current-book/metron-platform/metron-parsers/3rdPartyParser.html
@@ -0,0 +1,467 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-parsers/3rdPartyParser.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Custom Metron Parsers</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Custom Metron Parsers</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-down"></span>Parsers</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>3rdPartyParser</a></li>
+    <li><a href="../../metron-platform/metron-parsers/parser-testing.html" title="parser-testing"><span class="none"></span>parser-testing</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Custom Metron Parsers</h1>
+<p><a name="Custom_Metron_Parsers"></a></p>
+<p>We have many stock parsers for normal operations.  Some of these are networking and cybersecurity focused (e.g. the ASA Parser), some of these are general purpose (e.g. the CSVParser), but inevitably users will want to extend the system to process their own data formats.  To enable this, this is a walkthrough of how to create and use a custom parser within Metron.</p>
+<p><a name="Writing_A_Custom_Parser"></a></p>
+<h1>Writing A Custom Parser</h1>
+<p>Before we can use a parser, we will need to create a custom parser.  The parser is the workhorse of Metron ingest.  It provides the mapping between the raw data coming in via the Kafka value and a <tt>JSONObject</tt>, the internal data structure provided.</p>
+<div class="section">
+<h2><a name="Implementation"></a>Implementation</h2>
+<p>In order to do create a custom parser, we need to do one of the following:</p>
+<ul>
+
+<li>Write a class which conforms to the <tt>org.apache.metron.parsers.interfaces.MessageParser&lt;JSONObject&gt;</tt> and <tt>java.util.Serializable</tt> interfaces
+<ul>
+
+<li>Implement <tt>init()</tt>, <tt>validate(JSONObject message)</tt>, and <tt>List&lt;JSONObject&gt; parse(byte[] rawMessage)</tt></li>
+</ul>
+</li>
+<li>Write a class which extends <tt>org.apache.metron.parsers.BasicParser</tt>
+<ul>
+
+<li>Provides convenience implementations to <tt>validate</tt> which ensures <tt>timestamp</tt> and <tt>original_string</tt> fields exist.</li>
+</ul>
+</li>
+</ul></div>
+<div class="section">
+<h2><a name="Example"></a>Example</h2>
+<p>In order to illustrate how this might be done, let&#x2019;s create a very simple parser that takes a comma separated pair and creates a couple of fields:</p>
+<ul>
+
+<li><tt>original_string</tt> &#x2013; the raw data</li>
+<li><tt>timestamp</tt> &#x2013; the current time</li>
+<li><tt>first</tt> &#x2013; the first field of the comma separated pair</li>
+<li><tt>last</tt> &#x2013; the last field of the comma separated pair</li>
+</ul>
+<p>For this demonstration, let&#x2019;s create a maven project to compile our project.  We&#x2019;ll call it <tt>extra_parsers</tt>, so in your workspace, let&#x2019;s set up the maven project:</p>
+<ul>
+
+<li>Create the maven infrastructure for <tt>extra_parsers</tt> via</li>
+</ul>
+
+<div>
+<div>
+<pre class="source">mkdir -p extra_parsers/src/{main,test}/java
+</pre></div></div>
+
+<ul>
+
+<li>Create a pom file indicating how we should build our parsers by editing <tt>extra_parsers/pom.xml</tt> with the following content:</li>
+</ul>
+
+<div>
+<div>
+<pre class="source">&lt;project xmlns=&quot;http://maven.apache.org/POM/4.0.0&quot; xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot;
+  xsi:schemaLocation=&quot;http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd&quot;&gt;
+  &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
+  &lt;groupId&gt;com.3rdparty&lt;/groupId&gt;
+  &lt;artifactId&gt;extra-parsers&lt;/artifactId&gt;
+  &lt;packaging&gt;jar&lt;/packaging&gt;
+  &lt;version&gt;1.0-SNAPSHOT&lt;/version&gt;
+  &lt;name&gt;extra-parsers&lt;/name&gt;
+  &lt;url&gt;http://thirdpartysoftware.org&lt;/url&gt;
+  &lt;properties&gt;
+    &lt;!-- The java version to conform to.  Metron works all the way to 1.8 --&gt;
+    &lt;java_version&gt;1.8&lt;/java_version&gt;
+    &lt;!-- The version of Metron that we'll be targetting. --&gt;
+    &lt;metron_version&gt;0.4.1&lt;/metron_version&gt;
+    &lt;!-- To complete the simulation, we'll depend on a common dependency --&gt;
+    &lt;guava_version&gt;19.0&lt;/guava_version&gt;
+    &lt;!-- We will shade our dependencies to create a single jar at the end --&gt;
+    &lt;shade_version&gt;2.4.3&lt;/shade_version&gt;
+  &lt;/properties&gt;
+  &lt;dependencies&gt;
+    &lt;!--
+    We want to depend on Metron, but ensure that the scope is &quot;provided&quot;
+    as we do not want to include it in our bundle.
+    --&gt;
+    &lt;dependency&gt;
+      &lt;groupId&gt;org.apache.metron&lt;/groupId&gt;
+      &lt;artifactId&gt;metron-parsers&lt;/artifactId&gt;
+      &lt;version&gt;${metron_version}&lt;/version&gt;
+      &lt;scope&gt;provided&lt;/scope&gt;
+    &lt;/dependency&gt;
+    &lt;dependency&gt;
+      &lt;groupId&gt;com.google.guava&lt;/groupId&gt;
+      &lt;artifactId&gt;guava&lt;/artifactId&gt;
+      &lt;version&gt;${guava_version}&lt;/version&gt;
+    &lt;/dependency&gt;
+    &lt;dependency&gt;
+      &lt;groupId&gt;junit&lt;/groupId&gt;
+      &lt;artifactId&gt;junit&lt;/artifactId&gt;
+      &lt;version&gt;3.8.1&lt;/version&gt;
+      &lt;scope&gt;test&lt;/scope&gt;
+    &lt;/dependency&gt;
+  &lt;/dependencies&gt;
+  &lt;build&gt;
+    &lt;plugins&gt;
+     &lt;!-- We will set up the shade plugin to create a single jar at the
+           end of the build lifecycle.  We will exclude some things and
+           relocate others to simulate a real situation.
+           
+           One thing to note is that it's a good practice to shade and
+           relocate common libraries that may be dependencies in Metron.
+           Your jar will be merged with the parsers jar, so the metron
+           version will be included for all overlapping classes.
+           So, shade and relocate to ensure that YOUR version of the library is used.
+      --&gt;
+
+      &lt;plugin&gt;
+        &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
+        &lt;artifactId&gt;maven-shade-plugin&lt;/artifactId&gt;
+        &lt;version&gt;${shade_version}&lt;/version&gt;
+        &lt;configuration&gt;
+          &lt;createDependencyReducedPom&gt;true&lt;/createDependencyReducedPom&gt;
+          &lt;artifactSet&gt;
+            &lt;excludes&gt;
+              &lt;!-- Exclude slf4j for no reason other than to illustrate how to exclude dependencies.
+                   The metron team has nothing against slf4j. :-)
+               --&gt;
+              &lt;exclude&gt;*slf4j*&lt;/exclude&gt;
+            &lt;/excludes&gt;
+          &lt;/artifactSet&gt;
+        &lt;/configuration&gt;
+        &lt;executions&gt;
+          &lt;execution&gt;
+            &lt;phase&gt;package&lt;/phase&gt;
+            &lt;goals&gt;
+              &lt;goal&gt;shade&lt;/goal&gt;
+            &lt;/goals&gt;
+            &lt;configuration&gt;
+              &lt;shadedArtifactAttached&gt;true&lt;/shadedArtifactAttached&gt;
+              &lt;shadedClassifierName&gt;uber&lt;/shadedClassifierName&gt;
+              &lt;filters&gt;
+                &lt;filter&gt;
+                  &lt;!-- Sometimes these get added and confuse the uber jar out of shade --&gt;
+                  &lt;artifact&gt;*:*&lt;/artifact&gt;
+                  &lt;excludes&gt;
+                    &lt;exclude&gt;META-INF/*.SF&lt;/exclude&gt;
+                    &lt;exclude&gt;META-INF/*.DSA&lt;/exclude&gt;
+                    &lt;exclude&gt;META-INF/*.RSA&lt;/exclude&gt;
+                  &lt;/excludes&gt;
+                &lt;/filter&gt;
+              &lt;/filters&gt;
+              &lt;relocations&gt;
+                &lt;!-- Relocate guava as it's used in Metron and I really want 0.19 --&gt;
+                &lt;relocation&gt;
+                  &lt;pattern&gt;com.google&lt;/pattern&gt;
+                  &lt;shadedPattern&gt;com.thirdparty.guava&lt;/shadedPattern&gt;
+                &lt;/relocation&gt;
+              &lt;/relocations&gt;
+              &lt;artifactSet&gt;
+                &lt;excludes&gt;
+                  &lt;!-- We can also exclude by artifactId and groupId --&gt;
+                  &lt;exclude&gt;storm:storm-core:*&lt;/exclude&gt;
+                  &lt;exclude&gt;storm:storm-lib:*&lt;/exclude&gt;
+                  &lt;exclude&gt;org.slf4j.impl*&lt;/exclude&gt;
+                  &lt;exclude&gt;org.slf4j:slf4j-log4j*&lt;/exclude&gt;
+                &lt;/excludes&gt;
+              &lt;/artifactSet&gt;
+            &lt;/configuration&gt;
+          &lt;/execution&gt;
+        &lt;/executions&gt;
+      &lt;/plugin&gt;
+      &lt;!--
+      We want to make sure we compile using java 1.8.
+      --&gt;
+      &lt;plugin&gt;
+        &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
+        &lt;artifactId&gt;maven-compiler-plugin&lt;/artifactId&gt;
+        &lt;version&gt;3.5.1&lt;/version&gt;
+        &lt;configuration&gt;
+          &lt;forceJavacCompilerUse&gt;true&lt;/forceJavacCompilerUse&gt;
+          &lt;source&gt;${java_version}&lt;/source&gt;
+          &lt;compilerArgument&gt;-Xlint:unchecked&lt;/compilerArgument&gt;
+          &lt;target&gt;${java_version}&lt;/target&gt;
+          &lt;showWarnings&gt;true&lt;/showWarnings&gt;
+        &lt;/configuration&gt;
+      &lt;/plugin&gt;
+    &lt;/plugins&gt;
+  &lt;/build&gt;
+&lt;/project&gt;
+</pre></div></div>
+
+<ul>
+
+<li>Now let&#x2019;s create our parser  <tt>com.thirdparty.SimpleParser</tt> by creating the file <tt>extra-parsers/src/main/java/com/thirdparty/SimpleParser.java</tt> with the following content:</li>
+</ul>
+
+<div>
+<div>
+<pre class="source">package com.thirdparty;
+
+import com.google.common.base.Splitter;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Iterables;
+import org.apache.metron.parsers.BasicParser;
+import org.json.simple.JSONObject;
+
+import java.util.List;
+import java.util.Map;
+
+public class SimpleParser extends BasicParser {
+  @Override
+  public void init() {
+
+  }
+
+  @Override
+  public List&lt;JSONObject&gt; parse(byte[] bytes) {
+    String input = new String(bytes);
+    Iterable&lt;String&gt; it = Splitter.on(&quot;,&quot;).split(input);
+    JSONObject ret = new JSONObject();
+    ret.put(&quot;original_string&quot;, input);
+    ret.put(&quot;timestamp&quot;, System.currentTimeMillis());
+    ret.put(&quot;first&quot;, Iterables.getFirst(it, &quot;missing&quot;));
+    ret.put(&quot;last&quot;, Iterables.getLast(it, &quot;missing&quot;));
+    return ImmutableList.of(ret);
+  }
+
+  @Override
+  public void configure(Map&lt;String, Object&gt; map) {
+
+  }
+}
+</pre></div></div>
+
+<ul>
+
+<li>Compile the parser via <tt>mvn clean package</tt> in <tt>extra_parsers</tt></li>
+</ul>
+<p>This will create a jar containing your parser and its dependencies (sans Metron dependencies) in <tt>extra-parsers/target/extra-parsers-1.0-SNAPSHOT-uber.jar</tt></p>
+<p><a name="Deploying_Your_Custom_Parser"></a></p>
+<h1>Deploying Your Custom Parser</h1>
+<p>In order to deploy your newly built custom parser, you would place the jar file above in the <tt>$METRON_HOME/parser_contrib</tt> directory on the Metron host (i.e. any host you would start parsers from or, alternatively, where the Metron REST is hosted).</p></div>
+<div class="section">
+<h2><a name="Example"></a>Example</h2>
+<p>Let&#x2019;s work through deploying the example above.</p>
+<div class="section">
+<h3><a name="Preliminaries"></a>Preliminaries</h3>
+<p>We assume that the following environment variables are set:</p>
+<ul>
+
+<li><tt>METRON_HOME</tt> - the home directory for metron</li>
+<li><tt>ZOOKEEPER</tt> - The zookeeper quorum (comma separated with port specified: e.g. <tt>node1:2181</tt> for full-dev)</li>
+<li><tt>BROKERLIST</tt> - The Kafka broker list (comma separated with port specified: e.g. <tt>node1:6667</tt> for full-dev)</li>
+<li><tt>ES_HOST</tt> - The elasticsearch master (and port) e.g. <tt>node1:9200</tt> for full-dev.</li>
+</ul>
+<p>Also, this does not assume that you are using a kerberized cluster.  If you are, then the parser start command will adjust slightly to include the security protocol.</p></div>
+<div class="section">
+<h3><a name="Copy_the_jar_file_up"></a>Copy the jar file up</h3>
+<p>Copy the jar file located in <tt>extra-parsers/target/extra-parsers-1.0-SNAPSHOT-uber.jar</tt> to <tt>$METRON_HOME/parser_contrib</tt> and ensure the permissions are such that the <tt>metron</tt> user can read and execute.</p></div>
+<div class="section">
+<h3><a name="Restart_the_REST_service_in_Ambari"></a>Restart the REST service in Ambari</h3>
+<p>In order for new parsers to be picked up, the REST service must be restarted.  You can do that from within Ambari by restarting the <tt>Metron REST</tt> service.</p></div>
+<div class="section">
+<h3><a name="Push_the_Zookeeper_Configs"></a>Push the Zookeeper Configs</h3>
+<p>Now push the config to Zookeeper with the following command: <tt>$METRON_HOME/bin/zk_load_configs.sh -m PUSH -i $METRON_HOME/config/zookeeper/ -z $ZOOKEEPER</tt></p></div>
+<div class="section">
+<h3><a name="Create_a_Kafka_Topic"></a>Create a Kafka Topic</h3>
+<p>Create a kafka topic, let&#x2019;s call it <tt>test</tt> via: <tt>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic test --partitions 1 --replication-factor 1</tt></p>
+<p>Note, in a real deployment, that topic would be named something more descriptive and would have replication factor and partitions set to something less trivial.</p></div>
+<div class="section">
+<h3><a name="Configure_Test_Parser"></a>Configure Test Parser</h3>
+<p>Create the a file called <tt>$METRON_HOME/config/zookeeper/parsers/test.json</tt> with the following content:</p>
+
+<div>
+<div>
+<pre class="source">{
+  &quot;parserClassName&quot;:&quot;com.thirdparty.SimpleParser&quot;,
+  &quot;sensorTopic&quot;:&quot;test&quot;
+}
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="Start_Parser"></a>Start Parser</h3>
+<p>Now we can start the parser and send some data through:</p>
+<ul>
+
+<li>Start the parser</li>
+</ul>
+
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/start_parser_topology.sh -k $BROKERLIST -z $ZOOKEEPER -s test
+</pre></div></div>
+
+<ul>
+
+<li>Send example data through:</li>
+</ul>
+
+<div>
+<div>
+<pre class="source">echo &quot;apache,metron&quot; | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list $BROKERLIST --topic test
+</pre></div></div>
+
+<ul>
+
+<li>Validate data was written in ES:</li>
+</ul>
+
+<div>
+<div>
+<pre class="source">curl -XPOST &quot;http://$ES_HOST/test*/_search?pretty&quot; -d '
+{
+  &quot;_source&quot; : [ &quot;original_string&quot;, &quot;timestamp&quot;, &quot;first&quot;, &quot;last&quot;]
+}
+'
+</pre></div></div>
+
+<p>This should yield something like:</p>
+
+<div>
+<div>
+<pre class="source">{
+  &quot;took&quot; : 23,
+  &quot;timed_out&quot; : false,
+  &quot;_shards&quot; : {
+    &quot;total&quot; : 1,
+    &quot;successful&quot; : 1,
+    &quot;failed&quot; : 0
+  },
+  &quot;hits&quot; : {
+    &quot;total&quot; : 1,
+    &quot;max_score&quot; : 1.0,
+    &quot;hits&quot; : [ {
+      &quot;_index&quot; : &quot;test_index_2017.10.04.17&quot;,
+      &quot;_type&quot; : &quot;test_doc&quot;,
+      &quot;_id&quot; : &quot;3ae4dd4d-8c09-4f2a-93c0-26ec5508baaa&quot;,
+      &quot;_score&quot; : 1.0,
+      &quot;_source&quot; : {
+        &quot;original_string&quot; : &quot;apache,metron&quot;,
+        &quot;last&quot; : &quot;metron&quot;,
+        &quot;first&quot; : &quot;apache&quot;,
+        &quot;timestamp&quot; : 1507138373223
+      }
+    } ]
+  }
+}
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="Via_the_Management_UI"></a>Via the Management UI</h3>
+<p>As long as the REST service is restarted after new parsers are added to <tt>$METRON_HOME/parser_contrib</tt>, they are available in the UI to creating and deploying parsers.</p></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>


[32/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/Kerberos-manual-setup.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/Kerberos-manual-setup.html b/site/current-book/metron-deployment/Kerberos-manual-setup.html
index 1c1ed0a..229b90a 100644
--- a/site/current-book/metron-deployment/Kerberos-manual-setup.html
+++ b/site/current-book/metron-deployment/Kerberos-manual-setup.html
@@ -1,302 +1,139 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/Kerberos-manual-setup.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Kerberos Setup</title>
-    <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../css/site.css" />
     <link rel="stylesheet" href="../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Kerberos Setup</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Kerberos Setup</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Kerberos-manual-setup</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li><a href="../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Kerberos Setup</h1>
-<p>This document provides instructions for kerberizing Metron&#x2019;s Vagrant-based development environments. These instructions do not cover the Ambari MPack or sensors. General Kerberization notes can be found in the metron-deployment <a href="../index.html">README.md</a>.</p>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
 
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Kerberos Setup</h1>
+<p>This document provides instructions for kerberizing Metron&#x2019;s Vagrant-based development environments.  These instructions do not cover the Ambari MPack or sensors.  General Kerberization notes can be found in the metron-deployment <a href="../index.html">README.md</a>.</p>
 <ul>
-  
+
 <li><a href="#Setup">Setup</a></li>
-  
 <li><a href="#Setup_a_KDC">Setup a KDC</a></li>
-  
 <li><a href="#Verify_KDC">Verify KDC</a></li>
-  
 <li><a href="#Enable_Kerberos">Enable Kerberos</a></li>
-  
 <li><a href="#Kafka_Authorization">Kafka Authorization</a></li>
-  
 <li><a href="#HBase_Authorization">HBase Authorization</a></li>
-  
 <li><a href="#Storm_Authorization">Storm Authorization</a></li>
-  
 <li><a href="#Start_Metron">Start Metron</a></li>
-  
 <li><a href="#Push_Data">Push Data</a></li>
-  
 <li><a href="#More_Information">More Information</a></li>
+<li><a href="#x-pack">Elasticseach X-Pack</a></li>
 </ul>
 <div class="section">
 <h2><a name="Setup"></a>Setup</h2>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Deploy the <a href="vagrant/full-dev-platform/index.html">development environment.</a>.</p></li>
-  
+
+<p>Deploy the <a href="development/centos6/index.html">development environment.</a>.</p>
+</li>
 <li>
-<p>Export the following environment variables. These need to be set for the remainder of the instructions. Replace <tt>node1</tt> with the appropriate hosts, if you are running Metron anywhere other than Vagrant.</p>
-  
-<div class="source">
-<div class="source">
-<pre># execute as root
+
+<p>Export the following environment variables.  These need to be set for the remainder of the instructions. Replace <tt>node1</tt> with the appropriate hosts, if you are running Metron anywhere other than Vagrant.</p>
+
+<div>
+<div>
+<pre class="source"># execute as root
 sudo su -
 export KAFKA_HOME=&quot;/usr/hdp/current/kafka-broker&quot;
 export ZOOKEEPER=node1:2181
@@ -304,224 +141,244 @@ export ELASTICSEARCH=node1:9200
 export BROKERLIST=node1:6667
 export HDP_HOME=&quot;/usr/hdp/current&quot;
 export KAFKA_HOME=&quot;${HDP_HOME}/kafka-broker&quot;
-export METRON_VERSION=&quot;0.4.2&quot;
+export METRON_VERSION=&quot;${METRON_VERSION}&quot;
 export METRON_HOME=&quot;/usr/metron/${METRON_VERSION}&quot;
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Execute the following commands as root.</p>
-  
-<div class="source">
-<div class="source">
-<pre>sudo su -
-</pre></div></div></li>
-  
-<li>
-<p>Stop all Metron topologies. They will be restarted again once Kerberos has been enabled.</p>
-  
-<div class="source">
-<div class="source">
-<pre>for topology in bro snort enrichment indexing; do
+
+<div>
+<div>
+<pre class="source">sudo su -
+</pre></div></div>
+</li>
+<li>
+
+<p>Stop all Metron topologies.  They will be restarted again once Kerberos has been enabled.</p>
+
+<div>
+<div>
+<pre class="source">for topology in bro snort enrichment indexing; do
 	storm kill $topology;
 done
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Create the <tt>metron</tt> user&#x2019;s home directory in HDFS.</p>
-  
-<div class="source">
-<div class="source">
-<pre>sudo -u hdfs hdfs dfs -mkdir /user/metron
+
+<div>
+<div>
+<pre class="source">sudo -u hdfs hdfs dfs -mkdir /user/metron
 sudo -u hdfs hdfs dfs -chown metron:hdfs /user/metron
 sudo -u hdfs hdfs dfs -chmod 770 /user/metron
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h2><a name="Setup_a_KDC"></a>Setup a KDC</h2>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Install dependencies.</p>
-  
-<div class="source">
-<div class="source">
-<pre>yum -y install krb5-server krb5-libs krb5-workstation
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">yum -y install krb5-server krb5-libs krb5-workstation
+</pre></div></div>
+</li>
 <li>
+
 <p>Define the current host as the KDC.</p>
-  
-<div class="source">
-<div class="source">
-<pre>KDC=`hostname`
+
+<div>
+<div>
+<pre class="source">KDC=`hostname`
 sed -i.orig 's/kerberos.example.com/'&quot;$KDC&quot;'/g' /etc/krb5.conf
 cp -f /etc/krb5.conf /var/lib/ambari-server/resources/scripts
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Ensure that the KDC can issue renewable tickets. This may be necessary on a real cluster, but should not be on <a href="vagrant/full-dev-platform/index.html">Full Dev</a>.</p>
+
+<p>Ensure that the KDC can issue renewable tickets. This may be necessary on a real cluster, but should not be on a <a href="development/centos6/index.html">single VM</a>.</p>
 <p>Edit <tt>/var/kerberos/krb5kdc/kdc.conf</tt> and ensure the following is added to the <tt>realm</tt> section</p>
-  
-<div class="source">
-<div class="source">
-<pre>max_renewable_life = 7d
-</pre></div></div></li>
-  
-<li>
-<p>Create the KDC principal database. You will be prompted for a password. This step takes a moment.</p>
-  
-<div class="source">
-<div class="source">
-<pre>kdb5_util create -s
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">max_renewable_life = 7d
+</pre></div></div>
+</li>
+<li>
+
+<p>Create the KDC principal database.  You will be prompted for a password.  This step takes a moment.</p>
+
+<div>
+<div>
+<pre class="source">kdb5_util create -s
+</pre></div></div>
+</li>
 <li>
+
 <p>Start the KDC and ensure that it starts on boot.</p>
-  
-<div class="source">
-<div class="source">
-<pre>/etc/rc.d/init.d/krb5kdc start
+
+<div>
+<div>
+<pre class="source">/etc/rc.d/init.d/krb5kdc start
 chkconfig krb5kdc on    
-</pre></div></div></li>
-  
-<li>
-<p>Start the Kerberos Admin service and ensure that it starts on boot. </p>
-  
-<div class="source">
-<div class="source">
-<pre>/etc/rc.d/init.d/kadmin start
+</pre></div></div>
+</li>
+<li>
+
+<p>Start the Kerberos Admin service and ensure that it starts on boot.</p>
+
+<div>
+<div>
+<pre class="source">/etc/rc.d/init.d/kadmin start
 chkconfig kadmin on
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Setup the <tt>admin</tt> principal. You will be prompted for a password; do not forget it.</p>
-  
-<div class="source">
-<div class="source">
-<pre>kadmin.local -q &quot;addprinc admin/admin&quot;
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">kadmin.local -q &quot;addprinc admin/admin&quot;
+</pre></div></div>
+</li>
 <li>
+
 <p>Setup the <tt>metron</tt> principal. You will <tt>kinit</tt> as the <tt>metron</tt> principal when running topologies. You will be prompted for a password; do not forget it.</p>
-  
-<div class="source">
-<div class="source">
-<pre>kadmin.local -q &quot;addprinc metron&quot;
-</pre></div></div></li>
+
+<div>
+<div>
+<pre class="source">kadmin.local -q &quot;addprinc metron&quot;
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h2><a name="Verify_KDC"></a>Verify KDC</h2>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Ticket renewal is disallowed by default in many Linux distributions. If the KDC cannot issue renewable tickets, an error will be thrown when starting Metron&#x2019;s Storm topologies:</p>
-  
-<div class="source">
-<div class="source">
-<pre>Exception in thread &quot;main&quot; java.lang.RuntimeException:
+
+<div>
+<div>
+<pre class="source">Exception in thread &quot;main&quot; java.lang.RuntimeException:
 java.lang.RuntimeException: The TGT found is not renewable
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Ensure the Metron keytab is renewable. Look for the &#x2018;R&#x2019; flag in the output of the following command.</p>
-  
-<div class="source">
-<div class="source">
-<pre>klist -f
+
+<p>Ensure the Metron keytab is renewable.  Look for the &#x2018;R&#x2019; flag in the output of the following command.</p>
+
+<div>
+<div>
+<pre class="source">klist -f
 </pre></div></div>
-  
+
 <ul>
-    
+
 <li>If the &#x2018;R&#x2019; flags are present, you may skip to next section.</li>
-    
 <li>If the &#x2018;R&#x2019; flags are absent, you will need to follow the below steps:</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>
+
 <p>If the KDC is already setup, then editing <tt>max_life</tt> and <tt>max_renewable_life</tt> in <tt>/var/kerberos/krb5kdc/kdc.conf</tt>, then restarting <tt>kadmin</tt> and <tt>krb5kdc</tt> services will not change the policies for existing users.</p>
 <p>You need to set the renew lifetime for existing users and the <tt>krbtgt</tt> realm. Modify the appropriate principals to allow renewable tickets using the following commands. Adjust the parameters to match your desired KDC parameters:</p>
-  
-<div class="source">
-<div class="source">
-<pre>kadmin.local -q &quot;modprinc -maxlife 1days -maxrenewlife 7days +allow_renewable krbtgt/EXAMPLE.COM@EXAMPLE.COM&quot;
+
+<div>
+<div>
+<pre class="source">kadmin.local -q &quot;modprinc -maxlife 1days -maxrenewlife 7days +allow_renewable krbtgt/EXAMPLE.COM@EXAMPLE.COM&quot;
 kadmin.local -q &quot;modprinc -maxlife 1days -maxrenewlife 7days +allow_renewable metron@EXAMPLE.COM&quot;
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h2><a name="Enable_Kerberos"></a>Enable Kerberos</h2>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>In <a class="externalLink" href="http://node1:8080">Ambari</a>, setup Storm to use Kerberos and run worker jobs as the submitting user.</p>
 <p>a. Add the following properties to the custom storm-site:</p>
-  
-<div class="source">
-<div class="source">
-<pre>topology.auto-credentials=['org.apache.storm.security.auth.kerberos.AutoTGT']
+
+<div>
+<div>
+<pre class="source">topology.auto-credentials=['org.apache.storm.security.auth.kerberos.AutoTGT']
 nimbus.credential.renewers.classes=['org.apache.storm.security.auth.kerberos.AutoTGT']
 supervisor.run.worker.as.user=true
 </pre></div></div>
+
 <p>b. In the Storm config section in Ambari, choose &#x201c;Add Property&#x201d; under custom storm-site:</p>
 <p><img src="../images/ambari-storm-site.png" alt="custom storm-site" /></p>
 <p>c. In the dialog window, choose the &#x201c;bulk property add mode&#x201d; toggle button and add the below values:</p>
-<p><img src="../images/ambari-storm-site-properties.png" alt="custom storm-site properties" /></p></li>
-  
+<p><img src="../images/ambari-storm-site-properties.png" alt="custom storm-site properties" /></p>
+</li>
 <li>
+
 <p>Kerberize the cluster via Ambari. More detailed documentation can be found <a class="externalLink" href="http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_security/content/_enabling_kerberos_security_in_ambari.html">here</a>.</p>
 <p>a. For this exercise, choose existing MIT KDC (this is what we setup and installed in the previous steps.)</p>
 <p><img src="../images/enable-kerberos.png" alt="enable keberos" /></p>
 <p><img src="../images/enable-kerberos-started.png" alt="enable keberos get started" /></p>
-<p>b. Setup Kerberos configuration. Realm is EXAMPLE.COM. The admin principal will end up as admin/admin@EXAMPLE.COM when testing the KDC. Use the password you entered during the step for adding the admin principal.</p>
+<p>b. Setup Kerberos configuration. Realm is EXAMPLE.COM. The admin principal will end up as <a class="externalLink" href="mailto:admin/admin@EXAMPLE.COM">admin/admin@EXAMPLE.COM</a> when testing the KDC. Use the password you entered during the step for adding the admin principal.</p>
 <p><img src="../images/enable-kerberos-configure-kerberos.png" alt="enable keberos configure" /></p>
 <p>c. Click through to &#x201c;Start and Test Services.&#x201d; Let the cluster spin up, but don&#x2019;t worry about starting up Metron via Ambari - we&#x2019;re going to run the parsers manually against the rest of the Hadoop cluster Kerberized. The wizard will fail at starting Metron, but this is OK. Click &#x201c;continue.&#x201d; When you&#x2019;re finished, the custom storm-site should look similar to the following:</p>
-<p><img src="../images/custom-storm-site-final.png" alt="enable keberos configure" /></p></li>
-  
+<p><img src="../images/custom-storm-site-final.png" alt="enable keberos configure" /></p>
+</li>
 <li>
+
 <p>Create a Metron keytab</p>
-  
-<div class="source">
-<div class="source">
-<pre>kadmin.local -q &quot;ktadd -k metron.headless.keytab metron@EXAMPLE.COM&quot;
+
+<div>
+<div>
+<pre class="source">kadmin.local -q &quot;ktadd -k metron.headless.keytab metron@EXAMPLE.COM&quot;
 cp metron.headless.keytab /etc/security/keytabs
 chown metron:hadoop /etc/security/keytabs/metron.headless.keytab
 chmod 440 /etc/security/keytabs/metron.headless.keytab
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h2><a name="Kafka_Authorization"></a>Kafka Authorization</h2>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Acquire a Kerberos ticket using the <tt>metron</tt> principal.</p>
-  
-<div class="source">
-<div class="source">
-<pre>kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM
-</pre></div></div></li>
-  
-<li>
-<p>Create any additional Kafka topics that you will need. We need to create the topics before adding the required ACLs. The current full dev installation will deploy bro, snort, enrichments, and indexing only. For example, you may want to add a topic for &#x2018;yaf&#x2019; telemetry.</p>
-  
-<div class="source">
-<div class="source">
-<pre>${KAFKA_HOME}/bin/kafka-topics.sh \
+
+<div>
+<div>
+<pre class="source">kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM
+</pre></div></div>
+</li>
+<li>
+
+<p>Create any additional Kafka topics that you will need. We need to create the topics before adding the required ACLs. The current full dev installation will deploy bro, snort, enrichments, and indexing only.  For example, you may want to add a topic for &#x2018;yaf&#x2019; telemetry.</p>
+
+<div>
+<div>
+<pre class="source">${KAFKA_HOME}/bin/kafka-topics.sh \
   --zookeeper ${ZOOKEEPER} \
   --create \
   --topic yaf \
   --partitions 1 \
   --replication-factor 1
-</pre></div></div></li>
-  
-<li>
-<p>Setup Kafka ACLs for the <tt>bro</tt>, <tt>snort</tt>, <tt>enrichments</tt>, and <tt>indexing</tt> topics. Run the same command against any additional topics that you might be using; for example <tt>yaf</tt>.</p>
-  
-<div class="source">
-<div class="source">
-<pre>export KERB_USER=metron
+</pre></div></div>
+</li>
+<li>
+
+<p>Setup Kafka ACLs for the <tt>bro</tt>, <tt>snort</tt>, <tt>enrichments</tt>, and <tt>indexing</tt> topics.  Run the same command against any additional topics that you might be using; for example <tt>yaf</tt>.</p>
+
+<div>
+<div>
+<pre class="source">export KERB_USER=metron
 for topic in bro snort enrichments indexing; do
 	${KAFKA_HOME}/bin/kafka-acls.sh \
       --authorizer kafka.security.auth.SimpleAclAuthorizer \
@@ -530,15 +387,16 @@ for topic in bro snort enrichments indexing; do
       --allow-principal User:${KERB_USER} \
       --topic ${topic}
 done
-</pre></div></div></li>
-  
-<li>
-<p>Setup Kafka ACLs for the consumer groups. This command sets the ACLs for Bro, Snort, YAF, Enrichments, Indexing, and the Profiler. Execute the same command for any additional Parsers that you may be running.</p>
-  
-<div class="source">
-<div class="source">
-<pre>export KERB_USER=metron
-for group in bro_parser snort_parser yaf_parser enrichments indexing profiler; do
+</pre></div></div>
+</li>
+<li>
+
+<p>Setup Kafka ACLs for the consumer groups.  This command sets the ACLs for Bro, Snort, YAF, Enrichments, Indexing, and the Profiler.  Execute the same command for any additional Parsers that you may be running.</p>
+
+<div>
+<div>
+<pre class="source">export KERB_USER=metron
+for group in bro_parser snort_parser yaf_parser enrichments indexing-ra indexing-batch profiler; do
 	${KAFKA_HOME}/bin/kafka-acls.sh \
       --authorizer kafka.security.auth.SimpleAclAuthorizer \
       --authorizer-properties zookeeper.connect=${ZOOKEEPER} \
@@ -546,84 +404,92 @@ for group in bro_parser snort_parser yaf_parser enrichments indexing profiler; d
       --allow-principal User:${KERB_USER} \
       --group ${group}
 done
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Add the <tt>metron</tt> principal to the <tt>kafka-cluster</tt> ACL.</p>
-  
-<div class="source">
-<div class="source">
-<pre>${KAFKA_HOME}/bin/kafka-acls.sh \
+
+<div>
+<div>
+<pre class="source">${KAFKA_HOME}/bin/kafka-acls.sh \
     --authorizer kafka.security.auth.SimpleAclAuthorizer \
     --authorizer-properties zookeeper.connect=${ZOOKEEPER} \
     --add \
     --allow-principal User:${KERB_USER} \
     --cluster kafka-cluster
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h2><a name="HBase_Authorization"></a>HBase Authorization</h2>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Acquire a Kerberos ticket using the <tt>hbase</tt> principal</p>
-  
-<div class="source">
-<div class="source">
-<pre>kinit -kt /etc/security/keytabs/hbase.headless.keytab hbase-metron_cluster@EXAMPLE.COM
-</pre></div></div></li>
-  
+
+<div>
+<div>
+<pre class="source">kinit -kt /etc/security/keytabs/hbase.headless.keytab hbase-metron_cluster@EXAMPLE.COM
+</pre></div></div>
+</li>
 <li>
+
 <p>Grant permissions for the HBase tables used in Metron.</p>
-  
-<div class="source">
-<div class="source">
-<pre>echo &quot;grant 'metron', 'RW', 'threatintel'&quot; | hbase shell
+
+<div>
+<div>
+<pre class="source">echo &quot;grant 'metron', 'RW', 'threatintel'&quot; | hbase shell
 echo &quot;grant 'metron', 'RW', 'enrichment'&quot; | hbase shell
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>If you are using the Profiler, do the same for its HBase table.</p>
-  
-<div class="source">
-<div class="source">
-<pre>echo &quot;create 'profiler', 'P'&quot; | hbase shell
+
+<div>
+<div>
+<pre class="source">echo &quot;create 'profiler', 'P'&quot; | hbase shell
 echo &quot;grant 'metron', 'RW', 'profiler', 'P'&quot; | hbase shell
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h2><a name="Storm_Authorization"></a>Storm Authorization</h2>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Switch to the <tt>metron</tt> user and acquire a Kerberos ticket for the <tt>metron</tt> principal.</p>
-  
-<div class="source">
-<div class="source">
-<pre>su metron
+
+<div>
+<div>
+<pre class="source">su metron
 kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Create the directory <tt>/home/metron/.storm</tt> and switch to that directory.</p>
-  
-<div class="source">
-<div class="source">
-<pre>mkdir /home/metron/.storm
+
+<div>
+<div>
+<pre class="source">mkdir /home/metron/.storm
 cd /home/metron/.storm
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Ensure the Metron keytab is renewable. See <a href="#Verify_KDC">Verify KDC</a> above.</p></li>
-  
+
+<p>Ensure the Metron keytab is renewable. See <a href="#Verify_KDC">Verify KDC</a> above.</p>
+</li>
 <li>
-<p>Create a client JAAS file at <tt>/home/metron/.storm/client_jaas.conf</tt>. This should look identical to the Storm client JAAS file located at <tt>/etc/storm/conf/client_jaas.conf</tt> except for the addition of a <tt>Client</tt> stanza. The <tt>Client</tt> stanza is used for Zookeeper. All quotes and semicolons are necessary.</p>
-  
-<div class="source">
-<div class="source">
-<pre>cat &lt;&lt; EOF &gt; client_jaas.conf
+
+<p>Create a client JAAS file at <tt>/home/metron/.storm/client_jaas.conf</tt>.  This should look identical to the Storm client JAAS file located at <tt>/etc/storm/conf/client_jaas.conf</tt> except for the addition of a <tt>Client</tt> stanza. The <tt>Client</tt> stanza is used for Zookeeper. All quotes and semicolons are necessary.</p>
+
+<div>
+<div>
+<pre class="source">cat &lt;&lt; EOF &gt; client_jaas.conf
 StormClient {
     com.sun.security.auth.module.Krb5LoginModule required
     useTicketCache=true
@@ -649,127 +515,132 @@ KafkaClient {
     principal=&quot;metron@EXAMPLE.COM&quot;;
 };
 EOF
-</pre></div></div></li>
-  
-<li>
-<p>Create a YAML file at <tt>/home/metron/.storm/storm.yaml</tt>. This should point to the client JAAS file. Set the array of nimbus hosts accordingly.</p>
-  
-<div class="source">
-<div class="source">
-<pre>cat &lt;&lt; EOF &gt; /home/metron/.storm/storm.yaml
+</pre></div></div>
+</li>
+<li>
+
+<p>Create a YAML file at <tt>/home/metron/.storm/storm.yaml</tt>.  This should point to the client JAAS file.  Set the array of nimbus hosts accordingly.</p>
+
+<div>
+<div>
+<pre class="source">cat &lt;&lt; EOF &gt; /home/metron/.storm/storm.yaml
 nimbus.seeds : ['node1']
 java.security.auth.login.config : '/home/metron/.storm/client_jaas.conf'
 storm.thrift.transport : 'org.apache.storm.security.auth.kerberos.KerberosSaslTransportPlugin'
 EOF
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Create an auxiliary storm configuration file at <tt>/home/metron/storm-config.json</tt>. Note the login config option in the file points to the client JAAS file.</p>
-  
-<div class="source">
-<div class="source">
-<pre>cat &lt;&lt; EOF &gt; /home/metron/storm-config.json
+
+<div>
+<div>
+<pre class="source">cat &lt;&lt; EOF &gt; /home/metron/storm-config.json
 {
     &quot;topology.worker.childopts&quot; : &quot;-Djava.security.auth.login.config=/home/metron/.storm/client_jaas.conf&quot;
 }
 EOF
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Configure the Enrichment, Indexing and Profiler topologies to use the client JAAS file. To do this, the following key-value pairs:</p>
-  
+
+<p>Configure the Enrichment, Indexing and Profiler topologies to use the client JAAS file.  To do this, the following key-value pairs:</p>
 <ul>
-    
+
 <li><tt>kafka.security.protocol=PLAINTEXTSASL</tt></li>
-    
 <li><tt>topology.worker.childopts=-Djava.security.auth.login.config=/home/metron/.storm/client_jaas.conf</tt></li>
-  </ul>
+</ul>
 <p>must be added to each of the topology properties files:</p>
-  
 <ul>
-    
+
 <li><tt>${METRON_HOME}/config/enrichment.properties</tt></li>
-    
 <li><tt>${METRON_HOME}/config/elasticsearch.properties</tt></li>
-    
 <li><tt>${METRON_HOME}/config/profiler.properties</tt></li>
-  </ul>
+</ul>
 <p>You may use the following command to automate this step:</p>
-  
-<div class="source">
-<div class="source">
-<pre>for file in enrichment.properties elasticsearch.properties profiler.properties; do
+
+<div>
+<div>
+<pre class="source">for file in enrichment.properties elasticsearch.properties profiler.properties; do
   echo ${file}
   sed -i &quot;s/^kafka.security.protocol=.*/kafka.security.protocol=PLAINTEXTSASL/&quot; &quot;${METRON_HOME}/config/${file}&quot;
   sed -i &quot;s/^topology.worker.childopts=.*/topology.worker.childopts=-Djava.security.auth.login.config=\/home\/metron\/.storm\/client_jaas.conf/&quot; &quot;${METRON_HOME}/config/${file}&quot;
 done
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h2><a name="Start_Metron"></a>Start Metron</h2>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Switch to the <tt>metron</tt> user and acquire a Kerberos ticket for the <tt>metron</tt> principal.</p>
-  
-<div class="source">
-<div class="source">
-<pre>su metron
+
+<div>
+<div>
+<pre class="source">su metron
 kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM
-</pre></div></div></li>
-  
-<li>
-<p>Restart the parser topologies. Be sure to pass in the new parameter, <tt>-ksp</tt> or <tt>--kafka_security_protocol</tt>. The following command will start only the Bro and Snort topologies. Execute the same command for any other Parsers that you may need, for example <tt>yaf</tt>.</p>
-  
-<div class="source">
-<div class="source">
-<pre>for parser in bro snort; do
+</pre></div></div>
+</li>
+<li>
+
+<p>Restart the parser topologies. Be sure to pass in the new parameter, <tt>-ksp</tt> or <tt>--kafka_security_protocol</tt>.  The following command will start only the Bro and Snort topologies.  Execute the same command for any other Parsers that you may need, for example <tt>yaf</tt>.</p>
+
+<div>
+<div>
+<pre class="source">for parser in bro snort; do
    ${METRON_HOME}/bin/start_parser_topology.sh \
            -z ${ZOOKEEPER} \
            -s ${parser} \
            -ksp SASL_PLAINTEXT \
            -e /home/metron/storm-config.json;
 done
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Restart the Enrichment and Indexing topologies.</p>
-  
-<div class="source">
-<div class="source">
-<pre>${METRON_HOME}/bin/start_enrichment_topology.sh
+
+<div>
+<div>
+<pre class="source">${METRON_HOME}/bin/start_enrichment_topology.sh
 ${METRON_HOME}/bin/start_elasticsearch_topology.sh
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol>
 <p>Metron should be ready to receive data.</p></div>
 <div class="section">
 <h2><a name="Push_Data"></a>Push Data</h2>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Push some sample data to one of the parser topics. E.g for Bro we took raw data from <a href="../metron-platform/metron-integration-test/src/main/sample/data/bro/raw/BroExampleOutput/index.html">metron/metron-platform/metron-integration-test/src/main/sample/data/bro/raw/BroExampleOutput</a></p>
-  
-<div class="source">
-<div class="source">
-<pre>cat sample-bro.txt | ${KAFKA_HOME}/kafka-broker/bin/kafka-console-producer.sh \
+
+<div>
+<div>
+<pre class="source">cat sample-bro.txt | ${KAFKA_HOME}/kafka-broker/bin/kafka-console-producer.sh \
         --broker-list ${BROKERLIST} \
         --security-protocol SASL_PLAINTEXT \
         --topic bro
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Wait a few moments for data to flow through the system and then check for data in the Elasticsearch indices. Replace yaf with whichever parser type you&#x2019;ve chosen.</p>
-  
-<div class="source">
-<div class="source">
-<pre>curl -XGET &quot;${ELASTICSEARCH}/bro*/_search&quot;
+
+<div>
+<div>
+<pre class="source">curl -XGET &quot;${ELASTICSEARCH}/bro*/_search&quot;
 curl -XGET &quot;${ELASTICSEARCH}/bro*/_count&quot;
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>You should have data flowing from the parsers all the way through to the indexes. This completes the Kerberization instructions</p></li>
+
+<p>You should have data flowing from the parsers all the way through to the indexes. This completes the Kerberization instructions</p>
+</li>
 </ol></div>
 <div class="section">
 <h2><a name="More_Information"></a>More Information</h2>
@@ -777,15 +648,16 @@ curl -XGET &quot;${ELASTICSEARCH}/bro*/_count&quot;
 <h3><a name="Kerberos"></a>Kerberos</h3>
 <p>Unsure of your Kerberos principal associated with a keytab? There are a couple ways to get this. One is via the list of principals that Ambari provides via downloadable csv. If you didn&#x2019;t download this list, you can also check the principal manually by running the following against the keytab.</p>
 
-<div class="source">
-<div class="source">
-<pre>klist -kt /etc/security/keytabs/&lt;keytab-file-name&gt;
+<div>
+<div>
+<pre class="source">klist -kt /etc/security/keytabs/&lt;keytab-file-name&gt;
 </pre></div></div>
+
 <p>E.g.</p>
 
-<div class="source">
-<div class="source">
-<pre>klist -kt /etc/security/keytabs/hbase.headless.keytab
+<div>
+<div>
+<pre class="source">klist -kt /etc/security/keytabs/hbase.headless.keytab
 Keytab name: FILE:/etc/security/keytabs/hbase.headless.keytab
 KVNO Timestamp         Principal
 ---- ----------------- --------------------------------------------------------
@@ -794,16 +666,17 @@ KVNO Timestamp         Principal
    1 03/28/17 19:29:36 hbase-metron_cluster@EXAMPLE.COM
    1 03/28/17 19:29:36 hbase-metron_cluster@EXAMPLE.COM
    1 03/28/17 19:29:36 hbase-metron_cluster@EXAMPLE.COM
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Kafka_with_Kerberos_enabled"></a>Kafka with Kerberos enabled</h3>
 <div class="section">
 <h4><a name="Running_Sensors"></a>Running Sensors</h4>
 <p>A couple steps are required to produce data to a Kerberized Kafka topic. On the host you&#x2019;ll be setting up your sensor(s), switch to the metron user and create a client_jaas.conf file in the metron home directory if one doesn&#x2019;t already exist. It should be owned by metron:metron and contain at least the following stanza that tells the Kafka client how to interact with Kerberos:</p>
 
-<div class="source">
-<div class="source">
-<pre>su - metron
+<div>
+<div>
+<pre class="source">su - metron
 cat ${METRON_HOME}/client_jaas.conf
 ...
 KafkaClient {
@@ -816,91 +689,296 @@ KafkaClient {
    principal=&quot;metron@EXAMPLE.COM&quot;;
 };
 </pre></div></div>
+
 <p>You&#x2019;ll also need to set KAFKA_OPTS to tell the Kafka client how to interact with Kerberos.</p>
 
-<div class="source">
-<div class="source">
-<pre>export KAFKA_OPTS=&quot;-Djava.security.auth.login.config=${METRON_HOME}/client_jaas.conf&quot;
+<div>
+<div>
+<pre class="source">export KAFKA_OPTS=&quot;-Djava.security.auth.login.config=${METRON_HOME}/client_jaas.conf&quot;
 </pre></div></div>
+
 <p>For sensors that leverage the Kafka console producer to pipe data into Metron, e.g. Snort and Yaf, you will need to modify the corresponding sensor shell scripts or config to append the SASL security protocol property. <tt>--security-protocol SASL_PLAINTEXT</tt>. Be sure to kinit with the metron user&#x2019;s keytab before executing the script that starts the sensor.</p>
 <p>More notes can be found in <a href="../metron-sensors/index.html">metron/metron-sensors/README.md</a></p></div>
 <div class="section">
 <h4><a name="Write_data_to_a_topic_with_SASL"></a>Write data to a topic with SASL</h4>
 
-<div class="source">
-<div class="source">
-<pre>cat sample-yaf.txt | ${KAFKA_HOME}/bin/kafka-console-producer.sh \
+<div>
+<div>
+<pre class="source">cat sample-yaf.txt | ${KAFKA_HOME}/bin/kafka-console-producer.sh \
         --broker-list ${BROKERLIST} \
         --security-protocol PLAINTEXTSASL \
         --topic yaf
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="View_topic_data_from_latest_offset_with_SASL"></a>View topic data from latest offset with SASL</h4>
 
-<div class="source">
-<div class="source">
-<pre>${KAFKA_HOME}/bin/kafka-console-consumer.sh \
+<div>
+<div>
+<pre class="source">${KAFKA_HOME}/bin/kafka-console-consumer.sh \
         --zookeeper ${ZOOKEEPER} \
         --security-protocol PLAINTEXTSASL \
         --topic yaf
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Modify_the_sensor-stubs_to_send_logs_via_SASL"></a>Modify the sensor-stubs to send logs via SASL</h4>
 
-<div class="source">
-<div class="source">
-<pre>sed -i 's/node1:6667 --topic/node1:6667 --security-protocol PLAINTEXTSASL --topic/' /opt/sensor-stubs/bin/start-*-stub
+<div>
+<div>
+<pre class="source">sed -i 's/node1:6667 --topic/node1:6667 --security-protocol PLAINTEXTSASL --topic/' /opt/sensor-stubs/bin/start-*-stub
 for sensorstub in bro snort; do
     service sensor-stubs stop ${sensorstub};
     service sensor-stubs start ${sensorstub};
 done
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Model_as_a_Service_on_Kerberos"></a>Model as a Service on Kerberos</h4>
-<p>MaaS works with kerberos, you have to remember to kinit with the metron user. There is one small issue out of the box (particularly on vagrant), you get an error like so when running <tt>$METRON_HOME/bin/maas_service.sh</tt>:</p>
+<p>MaaS works with kerberos, you have to remember to kinit with the metron user.  There is one small issue out of the box (particularly on vagrant), you get an error like so when running <tt>$METRON_HOME/bin/maas_service.sh</tt>:</p>
 
-<div class="source">
-<div class="source">
-<pre>Requested user metron is not whitelisted and has id 501,which is below the minimum allowed 1000.
+<div>
+<div>
+<pre class="source">Requested user metron is not whitelisted and has id 501,which is below the minimum allowed 1000.
 </pre></div></div>
-<p>In order to correct this, you should:</p>
 
+<p>In order to correct this, you should:</p>
 <ul>
-  
+
 <li>Navigate to the Yarn configuration in Ambari</li>
-  
 <li>Click on &#x201c;Advanced&#x201d;</li>
-  
 <li>Scroll to &#x201c;Advanced yarn-env&#x201d;</li>
-  
 <li>Adjust the &#x201c;Minimum user ID for submitting job&#x201d; config to 500 from 1000</li>
-  
 <li>You should then restart Yarn to have the change take effect.</li>
 </ul></div></div>
 <div class="section">
 <h3><a name="References"></a>References</h3>
-
 <ul>
-  
+
 <li><a class="externalLink" href="https://github.com/apache/storm/blob/master/SECURITY.md">https://github.com/apache/storm/blob/master/SECURITY.md</a></li>
 </ul></div></div>
-                  </div>
-            </div>
-          </div>
+<div class="section">
+<h2><a name="X-Pack"></a>X-Pack</h2>
+<p>First, stop the random_access_indexing topology through the Storm UI or from the CLI, e.g.</p>
 
-    <hr/>
+<div>
+<div>
+<pre class="source">storm kill random_access_indexing
+</pre></div></div>
+
+<p>Here are instructions for enabling X-Pack with Elasticsearch and Kibana: <a class="externalLink" href="https://www.elastic.co/guide/en/x-pack/5.6/installing-xpack.html">https://www.elastic.co/guide/en/x-pack/5.6/installing-xpack.html</a></p>
+<p>You need to be sure to add the appropriate username and password for Elasticsearch and Kibana to enable external connections from Metron components. e.g. the following will create a user &#x201c;transport_client_user&#x201d; with password &#x201c;changeme&#x201d; and &#x201c;superuser&#x201d; credentials.</p>
+
+<div>
+<div>
+<pre class="source">sudo /usr/share/elasticsearch/bin/x-pack/users useradd transport_client_user -p changeme -r superuser
+</pre></div></div>
 
+<p>Once you&#x2019;ve picked a password to connect to ES, you need to upload a 1-line file to HDFS with that password in it. Metron will use this file to securely read the password in order to connect to ES securely.</p>
+<p>Here is an example using &#x201c;changeme&#x201d; as the password</p>
+
+<div>
+<div>
+<pre class="source">echo changeme &gt; /tmp/xpack-password
+sudo -u hdfs hdfs dfs -mkdir /apps/metron/elasticsearch/
+sudo -u hdfs hdfs dfs -put /tmp/xpack-password /apps/metron/elasticsearch/
+sudo -u hdfs hdfs dfs -chown metron:metron /apps/metron/elasticsearch/xpack-password
+</pre></div></div>
+
+<p>New settings have been added to configure the Elasticsearch client. By default the client will run as the normal ES prebuilt transport client. If you enable X-Pack you should set the es.client.class as shown below.</p>
+<p>Add the es settings to global.json</p>
+
+<div>
+<div>
+<pre class="source">/usr/metron/0.5.0/config/zookeeper/global.json -&gt;
+
+  &quot;es.client.settings&quot; : {
+      &quot;es.client.class&quot; : &quot;org.elasticsearch.xpack.client.PreBuiltXPackTransportClient&quot;,
+      &quot;es.xpack.username&quot; : &quot;transport_client_user&quot;,
+      &quot;es.xpack.password.file&quot; : &quot;/apps/metron/elasticsearch/xpack-password&quot;
+  }
+</pre></div></div>
+
+<p>Submit the update to Zookeeper</p>
+
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/zk_load_configs.sh -m PUSH -i METRON_HOME/config/zookeeper/ -z $ZOOKEEPER
+</pre></div></div>
+
+<p>The last step before restarting the topology is to create a custom X-Pack shaded and relocated jar. This is up to you because of licensing restrictions, but here is a sample Maven pom file that should help.</p>
+
+<div>
+<div>
+<pre class="source">&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;
+&lt;!--
+  Licensed to the Apache Software
+	Foundation (ASF) under one or more contributor license agreements. See the
+	NOTICE file distributed with this work for additional information regarding
+	copyright ownership. The ASF licenses this file to You under the Apache License,
+	Version 2.0 (the &quot;License&quot;); you may not use this file except in compliance
+	with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+	Unless required by applicable law or agreed to in writing, software distributed
+	under the License is distributed on an &quot;AS IS&quot; BASIS, WITHOUT WARRANTIES
+	OR CONDITIONS OF ANY KIND, either express or implied. See the License for
+  the specific language governing permissions and limitations under the License.
+  --&gt;
+&lt;project xmlns=&quot;http://maven.apache.org/POM/4.0.0&quot;
+         xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot;
+         xsi:schemaLocation=&quot;http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd&quot;&gt;
+    &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
+    &lt;groupId&gt;org.elasticsearch&lt;/groupId&gt;
+    &lt;artifactId&gt;elasticsearch-xpack-shaded&lt;/artifactId&gt;
+    &lt;name&gt;elasticsearch-xpack-shaded&lt;/name&gt;
+    &lt;packaging&gt;jar&lt;/packaging&gt;
+    &lt;version&gt;5.6.2&lt;/version&gt;
+    &lt;repositories&gt;
+        &lt;repository&gt;
+            &lt;id&gt;elasticsearch-releases&lt;/id&gt;
+            &lt;url&gt;https://artifacts.elastic.co/maven&lt;/url&gt;
+            &lt;releases&gt;
+                &lt;enabled&gt;true&lt;/enabled&gt;
+            &lt;/releases&gt;
+            &lt;snapshots&gt;
+                &lt;enabled&gt;false&lt;/enabled&gt;
+            &lt;/snapshots&gt;
+        &lt;/repository&gt;
+    &lt;/repositories&gt;
+    &lt;dependencies&gt;
+        &lt;dependency&gt;
+            &lt;groupId&gt;org.elasticsearch.client&lt;/groupId&gt;
+            &lt;artifactId&gt;x-pack-transport&lt;/artifactId&gt;
+            &lt;version&gt;5.6.2&lt;/version&gt;
+            &lt;exclusions&gt;
+              &lt;exclusion&gt;
+                &lt;groupId&gt;com.fasterxml.jackson.dataformat&lt;/groupId&gt;
+                &lt;artifactId&gt;jackson-dataformat-smile&lt;/artifactId&gt;
+              &lt;/exclusion&gt;
+              &lt;exclusion&gt;
+                &lt;groupId&gt;com.fasterxml.jackson.dataformat&lt;/groupId&gt;
+                &lt;artifactId&gt;jackson-dataformat-yaml&lt;/artifactId&gt;
+              &lt;/exclusion&gt;
+              &lt;exclusion&gt;
+                &lt;groupId&gt;com.fasterxml.jackson.dataformat&lt;/groupId&gt;
+                &lt;artifactId&gt;jackson-dataformat-cbor&lt;/artifactId&gt;
+              &lt;/exclusion&gt;
+              &lt;exclusion&gt;
+                &lt;groupId&gt;com.fasterxml.jackson.core&lt;/groupId&gt;
+                &lt;artifactId&gt;jackson-core&lt;/artifactId&gt;
+              &lt;/exclusion&gt;
+              &lt;exclusion&gt;
+                &lt;groupId&gt;org.slf4j&lt;/groupId&gt;
+                &lt;artifactId&gt;slf4j-api&lt;/artifactId&gt;
+              &lt;/exclusion&gt;
+              &lt;exclusion&gt;
+                &lt;groupId&gt;org.slf4j&lt;/groupId&gt;
+                &lt;artifactId&gt;slf4j-log4j12&lt;/artifactId&gt;
+              &lt;/exclusion&gt;
+              &lt;exclusion&gt;
+                &lt;groupId&gt;log4j&lt;/groupId&gt;
+                &lt;artifactId&gt;log4j&lt;/artifactId&gt;
+              &lt;/exclusion&gt;
+              &lt;exclusion&gt; &lt;!-- this is causing a weird build error if not excluded - Error creating shaded jar: null: IllegalArgumentException --&gt;
+                    &lt;groupId&gt;org.apache.logging.log4j&lt;/groupId&gt;
+                    &lt;artifactId&gt;log4j-api&lt;/artifactId&gt;
+                &lt;/exclusion&gt;
+            &lt;/exclusions&gt;
+          &lt;/dependency&gt;
+    &lt;/dependencies&gt;
+    &lt;build&gt;
+        &lt;plugins&gt;
+            &lt;plugin&gt;
+                &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
+                &lt;artifactId&gt;maven-shade-plugin&lt;/artifactId&gt;
+                &lt;version&gt;2.4.3&lt;/version&gt;
+                &lt;configuration&gt;
+                    &lt;createDependencyReducedPom&gt;true&lt;/createDependencyReducedPom&gt;
+                &lt;/configuration&gt;
+                &lt;executions&gt;
+                    &lt;execution&gt;
+                        &lt;phase&gt;package&lt;/phase&gt;
+                        &lt;goals&gt;
+                            &lt;goal&gt;shade&lt;/goal&gt;
+                        &lt;/goals&gt;
+                        &lt;configuration&gt;
+                          &lt;filters&gt;
+                            &lt;filter&gt;
+                              &lt;artifact&gt;*:*&lt;/artifact&gt;
+                              &lt;excludes&gt;
+                                &lt;exclude&gt;META-INF/*.SF&lt;/exclude&gt;
+                                &lt;exclude&gt;META-INF/*.DSA&lt;/exclude&gt;
+                                &lt;exclude&gt;META-INF/*.RSA&lt;/exclude&gt;
+                              &lt;/excludes&gt;
+                            &lt;/filter&gt;
+                          &lt;/filters&gt;
+                          &lt;relocations&gt;
+				&lt;relocation&gt;
+                                    &lt;pattern&gt;io.netty&lt;/pattern&gt;
+                                    &lt;shadedPattern&gt;org.apache.metron.io.netty&lt;/shadedPattern&gt;
+                                &lt;/relocation&gt;
+                                &lt;relocation&gt;
+                                    &lt;pattern&gt;org.apache.logging.log4j&lt;/pattern&gt;
+                                    &lt;shadedPattern&gt;org.apache.metron.logging.log4j&lt;/shadedPattern&gt;
+                                &lt;/relocation&gt;
+                            &lt;/relocations&gt;
+                            &lt;artifactSet&gt;
+                                &lt;excludes&gt;
+                                    &lt;exclude&gt;org.slf4j.impl*&lt;/exclude&gt;
+                                    &lt;exclude&gt;org.slf4j:slf4j-log4j*&lt;/exclude&gt;
+                                &lt;/excludes&gt;
+                            &lt;/artifactSet&gt;
+                            &lt;transformers&gt;
+                                &lt;transformer
+                                  implementation=&quot;org.apache.maven.plugins.shade.resource.DontIncludeResourceTransformer&quot;&gt;
+                                     &lt;resources&gt;
+                                        &lt;resource&gt;.yaml&lt;/resource&gt;
+                                        &lt;resource&gt;LICENSE.txt&lt;/resource&gt;
+                                        &lt;resource&gt;ASL2.0&lt;/resource&gt;
+                                        &lt;resource&gt;NOTICE.txt&lt;/resource&gt;
+                                      &lt;/resources&gt;
+                                &lt;/transformer&gt;
+                                &lt;transformer
+                                        implementation=&quot;org.apache.maven.plugins.shade.resource.ServicesResourceTransformer&quot;/&gt;
+                                &lt;transformer
+                                        implementation=&quot;org.apache.maven.plugins.shade.resource.ManifestResourceTransformer&quot;&gt;
+                                    &lt;mainClass&gt;&lt;/mainClass&gt;
+                                &lt;/transformer&gt;
+                            &lt;/transformers&gt;
+                        &lt;/configuration&gt;
+                    &lt;/execution&gt;
+                &lt;/executions&gt;
+            &lt;/plugin&gt;
+        &lt;/plugins&gt;
+    &lt;/build&gt;
+&lt;/project&gt;
+</pre></div></div>
+
+<p>Once you&#x2019;ve built the elasticsearch-xpack-shaded-5.6.2.jar, it needs to be made available to Storm when you submit the topology. Create a contrib directory for indexing and put the jar file in this directory.</p>
+
+<div>
+<div>
+<pre class="source">/usr/metron/0.5.0/indexing_contrib/elasticsearch-xpack-shaded-5.6.2.jar
+</pre></div></div>
+
+<p>Now you can restart the Elasticsearch topology. Note, you should perform this step manually, as follows.</p>
+
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/start_elasticsearch_topology.sh
+</pre></div></div>
+
+<p>Once you&#x2019;ve performed these steps, you shoud be able to start seeing data in your ES indexes.</p></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[20/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-api/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-api/index.html b/site/current-book/metron-platform/metron-api/index.html
index 690ab94..ddcaec5 100644
--- a/site/current-book/metron-platform/metron-api/index.html
+++ b/site/current-book/metron-platform/metron-api/index.html
@@ -1,346 +1,161 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-api/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Metron PCAP Service</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Metron PCAP Service</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron PCAP Service</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Api</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-common/index.html" title="Common">
-          <i class="none"></i>
-        Common</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Api</a></li>
+    <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Metron PCAP Service</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron PCAP Service</h1>
 <p><a name="Metron_PCAP_Service"></a></p>
-<p>The purpose of the Metron PCAP service is to provide a middle tier to negotiate retrieving packet capture data which flows into Metron. This packet data is of a form which <tt>libpcap</tt> based tools can read.</p>
+<p>The purpose of the Metron PCAP service is to provide a middle tier to negotiate retrieving packet capture data which flows into Metron.  This packet data is of a form which <tt>libpcap</tt> based tools can read.</p>
 <div class="section">
 <h2><a name="Starting_the_Service"></a>Starting the Service</h2>
 <p>You can start the service either via the init.d script installed, <tt>/etc/init.d/pcapservice</tt> or directly via the <tt>yarn jar</tt> command: <tt>yarn jar $METRON_HOME/lib/metron-api-$METRON_VERSION.jar org.apache.metron.pcapservice.rest.PcapService -port $SERVICE_PORT -query_hdfs_path $QUERY_PATH -pcap_hdfs_path $PCAP_PATH</tt></p>
 <p>where</p>
-
 <ul>
-  
+
 <li><tt>METRON_HOME</tt> is the location of the metron installation</li>
-  
 <li><tt>METRON_VERSION</tt> is the version of the metron installation</li>
-  
 <li><tt>SERVICE_PORT</tt> is the port to bind the REST service to.</li>
-  
-<li><tt>QUERY_PATH</tt> is the temporary location to store query results. They are deleted after the service reads them.</li>
-  
+<li><tt>QUERY_PATH</tt> is the temporary location to store query results.  They are deleted after the service reads them.</li>
 <li><tt>PCAP_PATH</tt> is the path to the packet data on HDFS</li>
 </ul></div>
 <div class="section">
-<h2><a name="The_pcapGettergetPcapsByIdentifiers_endpoint"></a>The <tt>/pcapGetter/getPcapsByIdentifiers</tt> endpoint</h2>
+<h2><a name="The_.2FpcapGetter.2FgetPcapsByIdentifiers_endpoint"></a>The <tt>/pcapGetter/getPcapsByIdentifiers</tt> endpoint</h2>
 <p>This endpoint takes the following query parameters and returns the subset of packets matching this query:</p>
-
 <ul>
-  
+
 <li><tt>srcIp</tt> : The source IP to match on</li>
-  
 <li><tt>srcPort</tt> : The source port to match on</li>
-  
 <li><tt>dstIp</tt> : The destination IP to match on</li>
-  
 <li><tt>dstPort</tt> : The destination port to match on</li>
-  
 <li><tt>startTime</tt> : The start time in milliseconds</li>
-  
 <li><tt>endTime</tt> : The end time in milliseconds</li>
-  
 <li><tt>numReducers</tt> : Specify the number of reducers to use when executing the mapreduce job</li>
-  
 <li><tt>includeReverseTraffic</tt> : Indicates if filter should check swapped src/dest addresses and IPs</li>
 </ul></div>
 <div class="section">
-<h2><a name="The_pcapGettergetPcapsByQuery_endpoint"></a>The <tt>/pcapGetter/getPcapsByQuery</tt> endpoint</h2>
+<h2><a name="The_.2FpcapGetter.2FgetPcapsByQuery_endpoint"></a>The <tt>/pcapGetter/getPcapsByQuery</tt> endpoint</h2>
 <p>This endpoint takes the following query parameters and returns the subset of packets matching this query. This endpoint exposes Stellar querying capabilities:</p>
-
 <ul>
-  
+
 <li><tt>query</tt> : The Stellar query to execute</li>
-  
 <li><tt>startTime</tt> : The start time in milliseconds</li>
-  
 <li><tt>endTime</tt> : The end time in milliseconds</li>
-  
 <li><tt>numReducers</tt> : Specify the number of reducers to use when executing the mapreduce job</li>
 </ul>
 <p>Example: <tt>curl -XGET &quot;http://node1:8081/pcapGetter/getPcapsByQuery?query=ip_src_addr+==+'192.168.66.121'+and+ip_src_port+==+'60500'&amp;startTime=1476936000000&quot;</tt></p>
-<p>All of these parameters are optional. In the case of a missing parameter, it is treated as a wildcard.</p>
+<p>All of these parameters are optional.  In the case of a missing parameter, it is treated as a wildcard.</p>
 <p>Unlike the CLI tool, there is no paging mechanism. The REST API will stream back data as a single file.</p></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-common/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-platform/metron-common/index.html b/site/current-book/metron-platform/metron-common/index.html
index 7111111..8d0dc0e 100644
--- a/site/current-book/metron-platform/metron-common/index.html
+++ b/site/current-book/metron-platform/metron-common/index.html
@@ -1,330 +1,152 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-common/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Contents</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Contents</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Contents</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                              
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-down"></i>
-        Platform</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide">
-          <i class="none"></i>
-        Performance-tuning-guide</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-api/index.html" title="Api">
-          <i class="none"></i>
-        Api</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Common</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-data-management/index.html" title="Data-management">
-          <i class="none"></i>
-        Data-management</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch">
-          <i class="none"></i>
-        Elasticsearch</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment">
-          <i class="none"></i>
-        Enrichment</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-indexing/index.html" title="Indexing">
-          <i class="none"></i>
-        Indexing</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-management/index.html" title="Management">
-          <i class="none"></i>
-        Management</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-platform/metron-parsers/index.html" title="Parsers">
-          <i class="icon-chevron-right"></i>
-        Parsers</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend">
-          <i class="none"></i>
-        Pcap-backend</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-platform/metron-writer/index.html" title="Writer">
-          <i class="none"></i>
-        Writer</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li>
+    <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Common</a></li>
+    <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li>
+    <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li>
+    <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li>
+    <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li>
+    <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li>
+    <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li>
+    <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li>
+    <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Contents</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Contents</h1>
 <p><a name="Contents"></a></p>
-
 <ul>
-  
+
 <li><a href="#Stellar_Language">Stellar Language</a></li>
-  
 <li><a href="#Global_Configuration">Global Configuration</a></li>
-  
 <li><a href="#Validation_Framework">Validation Framework</a></li>
-  
 <li><a href="#Management_Utility">Management Utility</a></li>
-  
 <li><a href="topology-errors/index.html">Topology Errors</a></li>
-  
 <li><a href="#Performance_Logging">Performance Logging</a></li>
+<li><a href="#Metron_Debugging">Metron Debugging</a></li>
 </ul>
 <p><a name="Stellar_Language"></a></p>
 <h1>Stellar Language</h1>
-<p>For a variety of components (threat intelligence triage and field transformations) we have the need to do simple computation and transformation using the data from messages as variables.<br />For those purposes, there exists a simple, scaled down DSL created to do simple computation and transformation.</p>
+<p>For a variety of components (threat intelligence triage and field transformations) we have the need to do simple computation and transformation using the data from messages as variables. For those purposes, there exists a simple, scaled down DSL created to do simple computation and transformation.</p>
 <p>The query language supports the following:</p>
-
 <ul>
-  
+
 <li>Referencing fields in the enriched JSON</li>
-  
 <li>String literals are quoted with either <tt>'</tt> or <tt>&quot;</tt>, and support escaping for <tt>'</tt>, <tt>&quot;</tt>, <tt>\t</tt>, <tt>\r</tt>, <tt>\n</tt>, and backslash</li>
-  
 <li>Simple boolean operations: <tt>and</tt>, <tt>not</tt>, <tt>or</tt>
-  
 <ul>
-    
+
 <li>Boolean expressions are short-circuited (e.g. <tt>true or FUNC()</tt> would never execute <tt>FUNC</tt>)</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Simple arithmetic operations: <tt>*</tt>, <tt>/</tt>, <tt>+</tt>, <tt>-</tt> on real numbers or integers</li>
-  
 <li>Simple comparison operations <tt>&lt;</tt>, <tt>&gt;</tt>, <tt>&lt;=</tt>, <tt>&gt;=</tt></li>
-  
 <li>Simple equality comparison operations <tt>==</tt>, <tt>!=</tt></li>
-  
 <li>if/then/else comparisons (i.e. <tt>if var1 &lt; 10 then 'less than 10' else '10 or more'</tt>)</li>
-  
 <li>Determining whether a field exists (via <tt>exists</tt>)</li>
-  
 <li>An <tt>in</tt> operator that works like the <tt>in</tt> in Python</li>
-  
 <li>The ability to have parenthesis to make order of operations explicit</li>
-  
 <li>User defined functions, including Lambda expressions</li>
 </ul>
 <p>For documentation of Stellar, please see the <a href="../../metron-stellar/stellar-common/index.html">Stellar README</a>.</p>
 <p><a name="Global_Configuration"></a></p>
 <h1>Global Configuration</h1>
-<p>The format of the global enrichment is a JSON String to Object map. This is intended for configuration which is non sensor specific configuration.</p>
+<p>The format of the global enrichment is a JSON String to Object map.  This is intended for configuration which is non sensor specific configuration.</p>
 <p>This configuration is stored in zookeeper, but looks something like</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;es.clustername&quot;: &quot;metron&quot;,
   &quot;es.ip&quot;: &quot;node1&quot;,
   &quot;es.port&quot;: &quot;9300&quot;,
@@ -337,237 +159,140 @@
                 &quot;config&quot; : {
                     &quot;type&quot; : &quot;IPV4&quot;
                            }
-              } 
+              }
                        ]
 }
 </pre></div></div>
-<p>Various parts of our stack uses the global config are documented throughout the Metron documentation, but a convenient index is provided here:</p>
 
+<p>Various parts of our stack uses the global config are documented throughout the Metron documentation, but a convenient index is provided here:</p>
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Property Name </th>
-      
-<th>Subsystem </th>
-      
-<th>Type </th>
-      
-<th>Ambari Property </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Property Name                                                                                                       </th>
+<th> Subsystem     </th>
+<th> Type       </th>
+<th> Ambari Property            </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td><a href="../metron-elasticsearch/index.html#es.clustername"><tt>es.clustername</tt></a> </td>
-      
-<td>Indexing </td>
-      
-<td>String </td>
-      
-<td><tt>es_cluster_name</tt> </td>
-    </tr>
-    
+<td> <a href="../metron-elasticsearch/index.html#es.clustername"><tt>es.clustername</tt></a>                                                           </td>
+<td> Indexing      </td>
+<td> String     </td>
+<td> <tt>es_cluster_name</tt>          </td></tr>
 <tr class="a">
-      
-<td><a href="../metron-elasticsearch/index.html#es.ip"><tt>es.ip</tt></a> </td>
-      
-<td>Indexing </td>
-      
-<td>String </td>
-      
-<td><tt>es_hosts</tt> </td>
-    </tr>
-    
+<td> <a href="../metron-elasticsearch/index.html#es.ip"><tt>es.ip</tt></a>                                                                             </td>
+<td> Indexing      </td>
+<td> String     </td>
+<td> <tt>es_hosts</tt>                 </td></tr>
 <tr class="b">
-      
-<td><a href="../metron-elasticsearch/index.html#es.port"><tt>es.port</tt></a> </td>
-      
-<td>Indexing </td>
-      
-<td>String </td>
-      
-<td><tt>es_port</tt> </td>
-    </tr>
-    
+<td> <a href="../metron-elasticsearch/index.html#es.port"><tt>es.port</tt></a>                                                                         </td>
+<td> Indexing      </td>
+<td> String     </td>
+<td> <tt>es_port</tt>                  </td></tr>
 <tr class="a">
-      
-<td><a href="../metron-elasticsearch/index.html#es.date.format"><tt>es.date.format</tt></a> </td>
-      
-<td>Indexing </td>
-      
-<td>String </td>
-      
-<td><tt>es_date_format</tt> </td>
-    </tr>
-    
+<td> <a href="../metron-elasticsearch/index.html#es.date.format"><tt>es.date.format</tt></a>                                                            </td>
+<td> Indexing      </td>
+<td> String     </td>
+<td> <tt>es_date_format</tt>           </td></tr>
 <tr class="b">
-      
-<td><a href="#validation-framework"><tt>fieldValidations</tt></a> </td>
-      
-<td>Parsing </td>
-      
-<td>Object </td>
-      
-<td>N/A </td>
-    </tr>
-    
+<td> <a href="#validation-framework"><tt>fieldValidations</tt></a>                                                                         </td>
+<td> Parsing       </td>
+<td> Object     </td>
+<td> N/A                        </td></tr>
 <tr class="a">
-      
-<td><a href="../metron-parsers/index.html#parser.error.topic"><tt>parser.error.topic</tt></a> </td>
-      
-<td>Parsing </td>
-      
-<td>String </td>
-      
-<td>N/A </td>
-    </tr>
-    
+<td> <a href="../metron-parsers/index.html#parser.error.topic"><tt>parser.error.topic</tt></a>                                                          </td>
+<td> Parsing       </td>
+<td> String     </td>
+<td> N/A                        </td></tr>
 <tr class="b">
-      
-<td><a href="../../metron-stellar/stellar-common/index.html#stellar.function.paths"><tt>stellar.function.paths</tt></a> </td>
-      
-<td>Stellar </td>
-      
-<td>CSV String </td>
-      
-<td>N/A </td>
-    </tr>
-    
+<td> <a href="../../metron-stellar/stellar-common/index.html#stellar.function.paths"><tt>stellar.function.paths</tt></a>                                </td>
+<td> Stellar       </td>
+<td> CSV String </td>
+<td> N/A                        </td></tr>
 <tr class="a">
-      
-<td><a href="../../metron-stellar/stellar-common/index.html#stellarfunctionresolverincludesexcludes"><tt>stellar.function.resolver.includes</tt></a> </td>
-      
-<td>Stellar </td>
-      
-<td>CSV String </td>
-      
-<td>N/A </td>
-    </tr>
-    
+<td> <a href="../../metron-stellar/stellar-common/index.html#stellarfunctionresolverincludesexcludes"><tt>stellar.function.resolver.includes</tt></a> </td>
+<td> Stellar       </td>
+<td> CSV String </td>
+<td> N/A                        </td></tr>
 <tr class="b">
-      
-<td><a href="../../metron-stellar/stellar-common/index.html#stellarfunctionresolverincludesexcludes"><tt>stellar.function.resolver.excludes</tt></a> </td>
-      
-<td>Stellar </td>
-      
-<td>CSV String </td>
-      
-<td>N/A </td>
-    </tr>
-    
+<td> <a href="../../metron-stellar/stellar-common/index.html#stellarfunctionresolverincludesexcludes"><tt>stellar.function.resolver.excludes</tt></a> </td>
+<td> Stellar       </td>
+<td> CSV String </td>
+<td> N/A                        </td></tr>
 <tr class="a">
-      
-<td><a href="../../metron-analytics/metron-profiler/index.html#profiler.period.duration"><tt>profiler.period.duration</tt></a> </td>
-      
-<td>Profiler </td>
-      
-<td>Integer </td>
-      
-<td><tt>profiler_period_duration</tt> </td>
-    </tr>
-    
+<td> <a href="../../metron-analytics/metron-profiler/index.html#profiler.period.duration"><tt>profiler.period.duration</tt></a>                         </td>
+<td> Profiler      </td>
+<td> Integer    </td>
+<td> <tt>profiler_period_duration</tt> </td></tr>
 <tr class="b">
-      
-<td><a href="../../metron-analytics/metron-profiler/index.html#profiler.period.duration.units"><tt>profiler.period.duration.units</tt></a> </td>
-      
-<td>Profiler </td>
-      
-<td>String </td>
-      
-<td><tt>profiler_period_units</tt> </td>
-    </tr>
-    
+<td> <a href="../../metron-analytics/metron-profiler/index.html#profiler.period.duration.units"><tt>profiler.period.duration.units</tt></a>              </td>
+<td> Profiler      </td>
+<td> String     </td>
+<td> <tt>profiler_period_units</tt>    </td></tr>
 <tr class="a">
-      
-<td><a href="../metron-indexing/index.html#update.hbase.table"><tt>update.hbase.table</tt></a> </td>
-      
-<td>REST/Indexing </td>
-      
-<td>String </td>
-      
-<td><tt>update_hbase_table</tt> </td>
-    </tr>
-    
+<td> <a href="../metron-indexing/index.html#update.hbase.table"><tt>update.hbase.table</tt></a>                                                         </td>
+<td> REST/Indexing </td>
+<td> String     </td>
+<td> <tt>update_hbase_table</tt>       </td></tr>
 <tr class="b">
-      
-<td><a href="../metron-indexing/index.html#update.hbase.cf"><tt>update.hbase.cf</tt></a> </td>
-      
-<td>REST/Indexing </td>
-      
-<td>String </td>
-      
-<td><tt>update_hbase_cf</tt> </td>
-    </tr>
-    
+<td> <a href="../metron-indexing/index.html#update.hbase.cf"><tt>update.hbase.cf</tt></a>                                                               </td>
+<td> REST/Indexing </td>
+<td> String     </td>
+<td> <tt>update_hbase_cf</tt>          </td></tr>
 <tr class="a">
-      
-<td><a href="../metron-enrichment/index.html#geo.hdfs.file"><tt>geo.hdfs.file</tt></a> </td>
-      
-<td>Enrichment </td>
-      
-<td>String </td>
-      
-<td><tt>geo_hdfs_file</tt> </td>
-    </tr>
-  </tbody>
+<td> <a href="../metron-enrichment/index.html#geo.hdfs.file"><tt>geo.hdfs.file</tt></a>                                                                 </td>
+<td> Enrichment    </td>
+<td> String     </td>
+<td> <tt>geo_hdfs_file</tt>            </td></tr>
+<tr class="b">
+<td> <a href="../../metron-interface/metron-alerts/index.html#source.type.field"><tt>source.type.field</tt></a>                                         </td>
+<td> UI            </td>
+<td> String     </td>
+<td>  N/A                       </td></tr>
+</tbody>
 </table>
 <div class="section">
 <h2><a name="Note_Configs_in_Ambari"></a>Note Configs in Ambari</h2>
-<p>If a field is managed via ambari, you should change the field via ambari. Otherwise, upon service restarts, you may find your update overwritten.</p>
+<p>If a field is managed via ambari, you should change the field via ambari.  Otherwise, upon service restarts, you may find your update overwritten.</p>
 <p><a name="Validation_Framework"></a></p>
 <h1>Validation Framework</h1>
-<p>Inside of the global configuration, there is a validation framework in place that enables the validation that messages coming from all parsers are valid. This is done in the form of validation plugins where assertions about fields or whole messages can be made. </p>
-<p>The format for this is a <tt>fieldValidations</tt> field inside of global config. This is associated with an array of field validation objects structured like so:</p>
-
+<p>Inside of the global configuration, there is a validation framework in place that enables the validation that messages coming from all parsers are valid.  This is done in the form of validation plugins where assertions about fields or whole messages can be made.</p>
+<p>The format for this is a <tt>fieldValidations</tt> field inside of global config.  This is associated with an array of field validation objects structured like so:</p>
 <ul>
-  
-<li><tt>input</tt> : An array of input fields or a single field. If this is omitted, then the whole messages is passed to the validator.</li>
-  
-<li><tt>config</tt> : A String to Object map for validation configuration. This is optional if the validation function requires no configuration.</li>
-  
-<li><tt>validation</tt> : The validation function to be used. This is one of
-  
+
+<li><tt>input</tt> : An array of input fields or a single field.  If this is omitted, then the whole messages is passed to the validator.</li>
+<li><tt>config</tt> : A String to Object map for validation configuration.  This is optional if the validation function requires no configuration.</li>
+<li><tt>validation</tt> : The validation function to be used.  This is one of
 <ul>
-    
-<li><tt>STELLAR</tt> : Execute a Stellar Language statement. Expects the query string in the <tt>condition</tt> field of the config.</li>
-    
-<li><tt>IP</tt> : Validates that the input fields are an IP address. By default, if no configuration is set, it assumes <tt>IPV4</tt>, but you can specify the type by passing in the config by passing in <tt>type</tt> with either <tt>IPV6</tt> or <tt>IPV4</tt> or by passing in a list [<tt>IPV4</tt>,<tt>IPV6</tt>] in which case the input(s) will be validated against both.</li>
-    
+
+<li><tt>STELLAR</tt> : Execute a Stellar Language statement.  Expects the query string in the <tt>condition</tt> field of the config.</li>
+<li><tt>IP</tt> : Validates that the input fields are an IP address.  By default, if no configuration is set, it assumes <tt>IPV4</tt>, but you can specify the type by passing in the config by passing in <tt>type</tt> with either <tt>IPV6</tt> or <tt>IPV4</tt> or by passing in a list [<tt>IPV4</tt>,<tt>IPV6</tt>] in which case the input(s) will be validated against both.</li>
 <li><tt>DOMAIN</tt> : Validates that the fields are all domains.</li>
-    
 <li><tt>EMAIL</tt> : Validates that the fields are all email addresses</li>
-    
 <li><tt>URL</tt> : Validates that the fields are all URLs</li>
-    
-<li><tt>DATE</tt> : Validates that the fields are a date. Expects <tt>format</tt> in the config.</li>
-    
-<li><tt>INTEGER</tt> : Validates that the fields are an integer. String representation of an integer is allowed.</li>
-    
-<li><tt>REGEX_MATCH</tt> : Validates that the fields match a regex. Expects <tt>pattern</tt> in the config.</li>
-    
+<li><tt>DATE</tt> : Validates that the fields are a date.  Expects <tt>format</tt> in the config.</li>
+<li><tt>INTEGER</tt> : Validates that the fields are an integer.  String representation of an integer is allowed.</li>
+<li><tt>REGEX_MATCH</tt> : Validates that the fields match a regex.  Expects <tt>pattern</tt> in the config.</li>
 <li><tt>NOT_EMPTY</tt> : Validates that the fields exist and are not empty (after trimming.)</li>
-  </ul></li>
+</ul>
+</li>
 </ul>
 <p><a name="Management_Utility"></a></p>
 <h1>Management Utility</h1>
 <p>Configurations should be stored on disk in the following structure starting at <tt>$BASE_DIR</tt>:</p>
-
 <ul>
-  
+
 <li>global.json : The global config</li>
-  
 <li><tt>sensors</tt> : The subdirectory containing sensor enrichment configuration JSON (e.g. <tt>snort.json</tt>, <tt>bro.json</tt>)</li>
 </ul>
 <p>By default, this directory as deployed by the ansible infrastructure is at <tt>$METRON_HOME/config/zookeeper</tt></p>
-<p>While the configs are stored on disk, they must be loaded into Zookeeper to be used. To this end, there is a utility program to assist in this called <tt>$METRON_HOME/bin/zk_load_config.sh</tt></p>
+<p>While the configs are stored on disk, they must be loaded into Zookeeper to be used.  To this end, there is a utility program to assist in this called <tt>$METRON_HOME/bin/zk_load_config.sh</tt></p>
 <p>This has the following options:</p>
 
-<div class="source">
-<div class="source">
-<pre> -c,--config_type &lt;CONFIG_TYPE&gt;            The configuration type: GLOBAL,
+<div>
+<div>
+<pre class="source"> -c,--config_type &lt;CONFIG_TYPE&gt;            The configuration type: GLOBAL,
                                            PARSER, ENRICHMENT, INDEXING,
                                            PROFILER
  -f,--force                                Force operation
@@ -592,22 +317,16 @@
  -z,--zk_quorum &lt;host:port,[host:port]*&gt;   Zookeeper Quorum URL
                                            (zk1:port,zk2:port,...)
 </pre></div></div>
-<p>Usage examples:</p>
 
+<p>Usage examples:</p>
 <ul>
-  
+
 <li>To dump the existing configs from zookeeper on the singlenode vagrant machine: <tt>$METRON_HOME/bin/zk_load_configs.sh -z node1:2181 -m DUMP</tt></li>
-  
 <li>To dump the existing GLOBAL configs from zookeeper on the singlenode vagrant machine: <tt>$METRON_HOME/bin/zk_load_configs.sh -z node1:2181 -m DUMP -c GLOBAL</tt></li>
-  
 <li>To push the configs into zookeeper on the singlenode vagrant machine: <tt>$METRON_HOME/bin/zk_load_configs.sh -z node1:2181 -m PUSH -i $METRON_HOME/config/zookeeper</tt></li>
-  
 <li>To push only the GLOBAL configs into zookeeper on the singlenode vagrant machine: <tt>$METRON_HOME/bin/zk_load_configs.sh -z node1:2181 -m PUSH -i $METRON_HOME/config/zookeeper -c GLOBAL</tt></li>
-  
 <li>To push only the PARSER configs into zookeeper on the singlenode vagrant machine: <tt>$METRON_HOME/bin/zk_load_configs.sh -z node1:2181 -m PUSH -i $METRON_HOME/config/zookeeper -c PARSER</tt></li>
-  
 <li>To push only the PARSER &#x2018;bro&#x2019; configs into zookeeper on the singlenode vagrant machine: <tt>$METRON_HOME/bin/zk_load_configs.sh -z node1:2181 -m PUSH -i $METRON_HOME/config/zookeeper -c PARSER -n bro</tt></li>
-  
 <li>To pull all configs from zookeeper to the singlenode vagrant machine disk: <tt>$METRON_HOME/bin/zk_load_configs.sh -z node1:2181 -m PULL -o $METRON_HOME/config/zookeeper -f</tt></li>
 </ul></div>
 <div class="section">
@@ -617,19 +336,13 @@
 <p>or key/value pair:</p>
 <p><tt>$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c GLOBAL -pm ADD -pk foo -pv \&quot;\&quot;bar\&quot;\&quot;</tt></p>
 <p>The options exposed via patch file are the full range of options from RFC-6902:</p>
-
 <ul>
-  
+
 <li>ADD</li>
-  
 <li>REMOVE</li>
-  
 <li>REPLACE</li>
-  
 <li>MOVE</li>
-  
 <li>COPY</li>
-  
 <li>TEST</li>
 </ul>
 <p>whereas with key/value patching, we only current expose ADD and REMOVE. Note that ADD will function as a REPLACE when the key already exists.</p>
@@ -637,19 +350,20 @@
 <h3><a name="Patch_File"></a>Patch File</h3>
 <p>Let&#x2019;s say we want to add a complex JSON object to our configuration with a patch file. e.g.</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;foo&quot; : {
+<div>
+<div>
+<pre class="source">&quot;foo&quot; : {
     &quot;bar&quot; : {
       &quot;baz&quot; : [ &quot;bazval1&quot;, &quot;bazval2&quot; ]
     }
   }
 </pre></div></div>
+
 <p>We would write a patch file &#x201c;/tmp/mypatch.txt&#x201d; with contents:</p>
 
-<div class="source">
-<div class="source">
-<pre>[
+<div>
+<div>
+<pre class="source">[
     {
         &quot;op&quot;: &quot;add&quot;,
         &quot;path&quot;: &quot;/foo&quot;,
@@ -657,37 +371,41 @@
     }
 ]
 </pre></div></div>
+
 <p>And submit via zk_load_configs as follows:</p>
 
-<div class="source">
-<div class="source">
-<pre> $METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c GLOBAL -pf /tmp/mypatch.txt
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source"> $METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c GLOBAL -pf /tmp/mypatch.txt
+</pre></div></div>
+</div>
 <div class="section">
-<h3><a name="Patch_KeyValue"></a>Patch Key/Value</h3>
+<h3><a name="Patch_Key.2FValue"></a>Patch Key/Value</h3>
 <p>Now let&#x2019;s try the same without using a patch file, instead using the patch_key and patch_value options right from the command line utility. This would like like the following.</p>
 
-<div class="source">
-<div class="source">
-<pre>$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c GLOBAL -pm ADD -pk &quot;/foo&quot; -pv &quot;{ \&quot;bar\&quot; : { \&quot;baz\&quot; : [ \&quot;bazval1\&quot;, \&quot;bazval2\&quot; ] } }&quot;
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c GLOBAL -pm ADD -pk &quot;/foo&quot; -pv &quot;{ \&quot;bar\&quot; : { \&quot;baz\&quot; : [ \&quot;bazval1\&quot;, \&quot;bazval2\&quot; ] } }&quot;
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Applying_Multiple_Patches"></a>Applying Multiple Patches</h3>
 <p>Applying multiple patches is also pretty straightforward. You can achieve this in a single command using patch files, or simply execute multiple commands in sequence using the patch_key/value approach.</p>
 <p>Let&#x2019;s say we wanted to add the following to our global config:</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;apache&quot; : &quot;metron&quot;,
+<div>
+<div>
+<pre class="source">&quot;apache&quot; : &quot;metron&quot;,
 &quot;is&quot; : &quot;the best&quot;,
 &quot;streaming&quot; : &quot;analytics platform&quot;
 </pre></div></div>
+
 <p>and remove the /foo key from the previous example.</p>
 <p>Create a patch file /tmp/mypatch.txt with four separate patch operations.</p>
 
-<div class="source">
-<div class="source">
-<pre>[
+<div>
+<div>
+<pre class="source">[
     {
         &quot;op&quot;: &quot;remove&quot;,
         &quot;path&quot;: &quot;/foo&quot;
@@ -709,27 +427,30 @@
     }
 ]
 </pre></div></div>
+
 <p>Now submit again and you should see a Global config with the &#x201c;foo&#x201d; key removed and three new keys added.</p>
 
-<div class="source">
-<div class="source">
-<pre> $METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c GLOBAL -pf /tmp/mypatch.txt
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source"> $METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c GLOBAL -pf /tmp/mypatch.txt
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Notes_On_Patching"></a>Notes On Patching</h3>
 <p>For any given patch key, the last/leaf node in the key&#x2019;s parent <i>must</i> exist, otherwise an exception will be thrown. For example, if you want to add the following:</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;foo&quot;: {
+<div>
+<div>
+<pre class="source">&quot;foo&quot;: {
     &quot;bar&quot;: &quot;baz&quot;
 }
 </pre></div></div>
+
 <p>It is not sufficient to use /foo/bar as a key if foo does not already exist. You would either need to incrementally build the JSON and make this a two step process</p>
 
-<div class="source">
-<div class="source">
-<pre>[
+<div>
+<div>
+<pre class="source">[
     {
         &quot;op&quot;: &quot;add&quot;,
         &quot;path&quot;: &quot;/foo&quot;,
@@ -742,11 +463,12 @@
     }
 ]
 </pre></div></div>
+
 <p>Or provide the value as a complete JSON object.</p>
 
-<div class="source">
-<div class="source">
-<pre>[
+<div>
+<div>
+<pre class="source">[
     {
         &quot;op&quot;: &quot;add&quot;,
         &quot;path&quot;: &quot;/foo&quot;,
@@ -754,14 +476,15 @@
     }
 ]
 </pre></div></div>
+
 <p>The REMOVE operation is idempotent. Running the remove command on the same key multiple times will not fail once the key has been removed.</p>
 <p><a name="Topology_Errors"></a></p>
 <h1>Topology Errors</h1>
 <p>Errors generated in Metron topologies are transformed into JSON format and follow this structure:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;exception&quot;: &quot;java.lang.IllegalStateException: Unable to parse Message: ...&quot;,
   &quot;failed_sensor_type&quot;: &quot;bro&quot;,
   &quot;stack&quot;: &quot;java.lang.IllegalStateException: Unable to parse Message: ...&quot;,
@@ -774,11 +497,12 @@
   &quot;timestamp&quot;: 1488809630698
 }
 </pre></div></div>
-<p>Each topology can be configured to send error messages to a specific Kafka topic. The parser topologies retrieve this setting from the the <tt>parser.error.topic</tt> setting in the global config:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<p>Each topology can be configured to send error messages to a specific Kafka topic.  The parser topologies retrieve this setting from the the <tt>parser.error.topic</tt> setting in the global config:</p>
+
+<div>
+<div>
+<pre class="source">{
   &quot;es.clustername&quot;: &quot;metron&quot;,
   &quot;es.ip&quot;: &quot;node1&quot;,
   &quot;es.port&quot;: &quot;9300&quot;,
@@ -786,16 +510,17 @@
   &quot;parser.error.topic&quot;: &quot;indexing&quot;
 }
 </pre></div></div>
-<p>Error topics for enrichment and threat intel errors are passed into the enrichment topology as flux properties named <tt>enrichment.error.topic</tt> and <tt>threat.intel.error.topic</tt>. These properties can be found in <tt>$METRON_HOME/config/enrichment.properties</tt>.</p>
-<p>The error topic for indexing errors is passed into the indexing topology as a flux property named <tt>index.error.topic</tt>. This property can be found in either <tt>$METRON_HOME/config/elasticsearch.properties</tt> or <tt>$METRON_HOME/config/solr.properties</tt> depending on the search engine selected.</p>
-<p>By default all error messages are sent to the <tt>indexing</tt> topic so that they are indexed and archived, just like other messages. The indexing config for error messages can be found at <tt>$METRON_HOME/config/zookeeper/indexing/error.json</tt>.</p>
+
+<p>Error topics for enrichment and threat intel errors are passed into the enrichment topology as flux properties named <tt>enrichment.error.topic</tt> and <tt>threat.intel.error.topic</tt>.  These properties can be found in <tt>$METRON_HOME/config/enrichment.properties</tt>.</p>
+<p>The error topic for indexing errors is passed into the indexing topology as a flux property named <tt>index.error.topic</tt>.  This property can be found in either <tt>$METRON_HOME/config/elasticsearch.properties</tt> or <tt>$METRON_HOME/config/solr.properties</tt> depending on the search engine selected.</p>
+<p>By default all error messages are sent to the <tt>indexing</tt> topic so that they are indexed and archived, just like other messages.  The indexing config for error messages can be found at <tt>$METRON_HOME/config/zookeeper/indexing/error.json</tt>.</p>
 <p><a name="Performance_Logging"></a></p>
 <h1>Performance Logging</h1>
 <p>The PerformanceLogger class provides functionality that enables developers to debug performance issues. Basic usage looks like the following:</p>
 
-<div class="source">
-<div class="source">
-<pre>// create a simple inner performance class to use for logger instantiation
+<div>
+<div>
+<pre class="source">// create a simple inner performance class to use for logger instantiation
 public static class Perf {}
 // instantiation
 PerformnanceLogger perfLog = new PerformanceLogger(() -&gt; getConfigurations().getGlobalConfig(), Perf.class.getName());
@@ -807,45 +532,37 @@ perfLog.log(&quot;mark1&quot;, &quot;My high performance stuff is very performan
 // log no additional message, just the basics
 perfLog.log(&quot;mark1&quot;);
 </pre></div></div>
+
 <p>The logger maintains a Map&lt;String, Long&gt; of named markers that correspond to start times. Calling mark() performs a put on the underlying timing store. Output includes the mark name, elapsed time in nanoseconds, as well as any custom messaging you provide. A sample log would look like the following:</p>
 
-<div class="source">
-<div class="source">
-<pre>[DEBUG] markName=execute,time(ns)=121411,message=key=7a8dbe44-4cb9-4db2-9d04-7632f543b56c, elapsed time to run execute
+<div>
+<div>
+<pre class="source">[DEBUG] markName=execute,time(ns)=121411,message=key=7a8dbe44-4cb9-4db2-9d04-7632f543b56c, elapsed time to run execute
 </pre></div></div>
+
 <p><b>Configuration</b></p>
 <p>The first argument to the logger is a java.util.function.Supplier&lt;Map&lt;String, Object&gt;&gt;. The offers flexibility in being able to provide multiple configuration &#x201c;suppliers&#x201d; depending on your individual usage requirements. The example above, taken from org.apache.metron.enrichment.bolt.GenericEnrichmentBolt, leverages the global config to dymanically provide configuration from Zookeeper. Any updates to the global config via Zookeeper are reflected live at runtime. Currently, the PerformanceLogger supports the following options:</p>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Property Name </th>
-      
-<th>Type </th>
-      
-<th>Valid Values </th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th>Property Name                              </th>
+<th>Type               </th>
+<th>Valid Values   </th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td>performance.logging.percent.records </td>
-      
-<td>Integer </td>
-      
-<td>0-100 </td>
-    </tr>
-  </tbody>
+<td>performance.logging.percent.records        </td>
+<td>Integer            </td>
+<td>0-100          </td></tr>
+</tbody>
 </table>
 <p><b>Other Usage Details</b></p>
 <p>You can also provide your own format String and provide arguments that will be used when formatting that String. This code avoids expensive String concatenation by only formatting when debugging is enabled. For more complex arguments, e.g. JSON serialization, we expose an isDebugEnabled() method.</p>
 
-<div class="source">
-<div class="source">
-<pre>// log with format String and single argument
+<div>
+<div>
+<pre class="source">// log with format String and single argument
 perfLog.log(&quot;join-message&quot;, &quot;key={}, elapsed time to join messages&quot;, key);
 
 // check if debugging is enabled for the performance logger to avoid more expensive operations
@@ -853,25 +570,51 @@ if (perfLog.isDebugEnabled()) {
     perfLog.log(&quot;join-message&quot;, &quot;key={}, elapsed time to join messages, message={}&quot;, key, rawMessage.toJSONString());
 }
 </pre></div></div>
-<p><b>Side Effects</b></p>
-<p>Calling the mark() method multiple times simply resets the start time to the current nano time. Calling log() with a non-existent mark name will log 0 ns elapsed time with a warning indicating that log has been invoked for a mark name that does not exist. The class is not thread-safe and makes no attempt at keeping multiple threads from modifying the same markers.</p></div></div>
-                  </div>
-            </div>
-          </div>
 
+<p><b>Side Effects</b></p>
+<p>Calling the mark() method multiple times simply resets the start time to the current nano time. Calling log() with a non-existent mark name will log 0 ns elapsed time with a warning indicating that log has been invoked for a mark name that does not exist. The class is not thread-safe and makes no attempt at keeping multiple threads from modifying the same markers.</p>
+<p><a name="Metron_Debugging"></a></p>
+<h1>Metron Debugging</h1>
+<p>A Python script is provided for gathering information useful in debugging your Metron cluster. Run from the node that has Metron installed on it. All options listed below are required.</p>
+<p><i>Note:</i> Be aware that no anonymization/scrubbing is performed on the captured configuration details.</p>
+
+<div>
+<div>
+<pre class="source"># $METRON_HOME/bin/cluster_info.py -h
+Usage: cluster_info.py [options]
+
+Options:
+  -h, --help            show this help message and exit
+  -a HOST:PORT, --ambari-host=HOST:PORT
+                        Connect to Ambari via the supplied host:port
+  -c NAME, --cluster-name=NAME
+                        Name of cluster in Ambari to retrieve info for
+  -o DIRECTORY, --out-dir=DIRECTORY
+                        Write debugging data to specified root directory
+  -s HOST:PORT, --storm-host=HOST:PORT
+                        Connect to Storm via the supplied host:port
+  -b HOST1:PORT,HOST2:PORT, --broker_list=HOST1:PORT,HOST2:PORT
+                        Connect to Kafka via the supplied comma-delimited
+                        host:port list
+  -z HOST1:PORT,HOST2:PORT, --zookeeper_quorum=HOST1:PORT,HOST2:PORT
+                        Connect to Zookeeper via the supplied comma-delimited
+                        host:port quorum list
+  -m DIRECTORY, --metron_home=DIRECTORY
+                        Metron home directory
+  -p DIRECTORY, --hdp_home=DIRECTORY
+                        HDP home directory
+</pre></div></div></div></div>
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[41/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/index.html b/site/current-book/index.html
index 729fff7..f8d6f85 100644
--- a/site/current-book/index.html
+++ b/site/current-book/index.html
@@ -1,274 +1,180 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Apache Metron</title>
-    <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="./css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="./css/site.css" />
     <link rel="stylesheet" href="./css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="./js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Apache Metron</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Apache Metron</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                    
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Metron</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Apache Metron</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Apache Metron</h1>
 <p><a name="Apache_Metron"></a></p>
 <p>Metron integrates a variety of open source big data technologies in order to offer a centralized tool for security monitoring and analysis. Metron provides capabilities for log aggregation, full packet capture indexing, storage, advanced behavioral analytics and data enrichment, while applying the most current threat intelligence information to security telemetry within a single platform.</p>
 <p>For the latest information, please visit our website at <a class="externalLink" href="http://metron.apache.org/">http://metron.apache.org/</a></p>
 <p>Metron can be divided into 4 areas:</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p><b>A mechanism to capture, store, and normalize any type of security telemetry at extremely high rates.</b> Because security telemetry is constantly being generated, it requires a method for ingesting the data at high speeds and pushing it to various processing units for advanced computation and analytics.</p></li>
-  
+
+<p><b>A mechanism to capture, store, and normalize any type of security telemetry at extremely high rates.</b>  Because security telemetry is constantly being generated, it requires a method for ingesting the data at high speeds and pushing it to various processing units for advanced computation and analytics.</p>
+</li>
 <li>
-<p><b>Real time processing and application of enrichments</b> such as threat intelligence, geolocation, and DNS information to telemetry being collected. The immediate application of this information to incoming telemetry provides the context and situational awareness, as well as the who and where information critical for investigation</p></li>
-  
+
+<p><b>Real time processing and application of enrichments</b> such as threat intelligence, geolocation, and DNS information to telemetry being collected. The immediate application of this information to incoming telemetry provides the context and situational awareness, as well as the who and where information critical for investigation</p>
+</li>
 <li>
+
 <p><b>Efficient information storage</b> based on how the information will be used:</p>
-  
 <ul>
-    
+
 <li>Logs and telemetry are stored such that they can be efficiently mined and analyzed for concise security visibility</li>
-    
 <li>The ability to extract and reconstruct full packets helps an analyst answer questions such as who the true attacker was, what data was leaked, and where that data was sent</li>
-    
-<li>Long-term storage not only increases visibility over time, but also enables advanced analytics such as machine learning techniques to be used to create models on the information. Incoming data can then be scored against these stored models for advanced anomaly detection.</li>
-  </ul></li>
-  
+<li>Long-term storage not only increases visibility over time, but also enables advanced analytics such as machine learning techniques to be used to create models on the information.  Incoming data can then be scored against these stored models for advanced anomaly detection.</li>
+</ul>
+</li>
 <li>
-<p><b>An interface that gives a security investigator a centralized view of data and alerts passed through the system.</b> Metron&#x2019;s interface presents alert summaries with threat intelligence and enrichment data specific to that alert on one single page. Furthermore, advanced search capabilities and full packet extraction tools are presented to the analyst for investigation without the need to pivot into additional tools. </p></li>
+
+<p><b>An interface that gives a security investigator a centralized view of data and alerts passed through the system.</b>  Metron&#x2019;s interface presents alert summaries with threat intelligence and enrichment data specific to that alert on one single page.  Furthermore, advanced search capabilities and full packet extraction tools are presented to the analyst for investigation without the need to pivot into additional tools.</p>
+</li>
 </ol>
-<p>Big data is a natural fit for powerful security analytics. The Metron framework integrates a number of elements from the Hadoop ecosystem to provide a scalable platform for security analytics, incorporating such functionality as full-packet capture, stream processing, batch processing, real-time search, and telemetry aggregation. With Metron, our goal is to tie big data into security analytics and drive towards an extensible centralized platform to effectively enable rapid detection and rapid response for advanced security threats. </p>
+<p>Big data is a natural fit for powerful security analytics. The Metron framework integrates a number of elements from the Hadoop ecosystem to provide a scalable platform for security analytics, incorporating such functionality as full-packet capture, stream processing, batch processing, real-time search, and telemetry aggregation.  With Metron, our goal is to tie big data into security analytics and drive towards an extensible centralized platform to effectively enable rapid detection and rapid response for advanced security threats.</p>
 <p><a name="Obtaining_Metron"></a></p>
 <h1>Obtaining Metron</h1>
 <p>To obtain a release of Metron, please visit <a class="externalLink" href="http://metron.apache.org/documentation/#releases">http://metron.apache.org/documentation/#releases</a></p>
 <p>This repository is a collection of submodules for convenience which is regularly updated to point to the latest versions. Github provides multiple ways to obtain Metron&#x2019;s code:</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>git clone &#x2013;recursive <a class="externalLink" href="https://github.com/apache/metron">https://github.com/apache/metron</a></li>
-  
 <li><a class="externalLink" href="https://github.com/apache/metron/archive/master.zip">Download ZIP</a></li>
-  
 <li>Clone or download each repository individually</li>
 </ol>
 <p>Option 3 is more likely to have the latest code.</p>
 <p><a name="Getting_Started"></a></p>
 <h1>Getting Started</h1>
-<p>To start exploring the capabilities of Apache Metron <a href="metron-deployment/vagrant/full-dev-platform/index.html">follow these instructions to launch Metron in a single-node VM</a> on your own hardware. </p>
+<p>To start exploring the capabilities of Apache Metron <a href="metron-deployment/development/centos6/index.html">follow these instructions to launch Metron in a single-node VM</a> on your own hardware.</p>
 <p><a name="Building_Metron"></a></p>
 <h1>Building Metron</h1>
 <p>Build the full project and run tests:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ mvn clean install
+<div>
+<div>
+<pre class="source">$ mvn clean install
 </pre></div></div>
+
 <p>Build without tests:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ mvn clean install -DskipTests
+<div>
+<div>
+<pre class="source">$ mvn clean install -DskipTests
 </pre></div></div>
+
 <p>Build with the HDP profile:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ mvn clean install -PHDP-2.5.0.0
+<div>
+<div>
+<pre class="source">$ mvn clean install -PHDP-2.5.0.0
 </pre></div></div>
+
 <p>You can swap &#x201c;install&#x201d; for &#x201c;package&#x201d; in the commands above if you don&#x2019;t want to deploy the artifacts to your local .m2 repo.</p>
 <p><a name="Build_Metron_Reporting"></a></p>
 <h1>Build Metron Reporting</h1>
 <p>To build and run reporting with code coverage:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ mvn clean install
+<div>
+<div>
+<pre class="source">$ mvn clean install
 $ mvn site site:stage-deploy site:deploy
 </pre></div></div>
+
 <p>Code coverage can be skipped by skipping tests:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ mvn clean install -DskipTests site site:stage-deploy site:deploy
+<div>
+<div>
+<pre class="source">$ mvn clean install -DskipTests site site:stage-deploy site:deploy
 </pre></div></div>
+
 <p>The staged site is deployed to /tmp/metron/site/index.html, and can be viewed by opening the file in a browser.</p>
 <div class="section">
 <h2><a name="Building_with_Docker"></a>Building with Docker</h2>
@@ -277,48 +183,35 @@ $ mvn site site:stage-deploy site:deploy
 <h1>Navigating the Architecture</h1>
 <p>Metron is at its core a Kappa architecture with Apache Storm as the processing component and Apache Kafka as the unified data bus.</p>
 <p>Some high level links to the relevant subparts of the architecture, for more information:</p>
-
 <ul>
-  
+
 <li><a href="metron-platform/metron-parsers/index.html">Parsers</a> : Parsing data from kafka into the Metron data model and passing it downstream to Enrichment.</li>
-  
 <li><a href="metron-platform/metron-enrichment/index.html">Enrichment</a> : Enriching data post-parsing and providing the ability to tag a message as an alert and assign a risk triage level via a custom rule language.</li>
-  
 <li><a href="metron-platform/metron-indexing/index.html">Indexing</a> : Indexing the data post-enrichment into HDFS, Elasticsearch or Solr.</li>
 </ul>
 <p>Some useful utilities that cross all of these parts of the architecture:</p>
-
 <ul>
-  
+
 <li><a href="metron-platform/metron-common/index.html">Stellar</a> : A custom data transformation language that is used throughout metron from simple field transformation to expressing triage rules.</li>
-  
 <li><a href="metron-analytics/metron-maas-service/index.html">Model as a Service</a> : A Yarn application which can deploy machine learning and statistical models onto the cluster along with the associated Stellar functions to be able to call out to them in a scalable manner.</li>
-  
-<li><a href="metron-platform/metron-data-management/index.html">Data management</a> : A set of data management utilities aimed at getting data into HBase in a format which will allow data flowing through metron to be enriched with the results. Contains integrations with threat intelligence feeds exposed via TAXII as well as simple flat file structures.</li>
-  
+<li><a href="metron-platform/metron-data-management/index.html">Data management</a> : A set of data management utilities aimed at getting data into HBase in a format which will allow data flowing through metron to be enriched with the results.  Contains integrations with threat intelligence feeds exposed via TAXII as well as simple flat file structures.</li>
 <li><a href="metron-analytics/metron-profiler/index.html">Profiler</a> : A feature extraction mechanism that can generate a profile describing the behavior of an entity. An entity might be a server, user, subnet or application. Once a profile has been generated defining what normal behavior looks-like, models can be built that identify anomalous behavior.</li>
 </ul>
 <p><a name="Notes_on_Adding_a_New_Sensor"></a></p>
 <h1>Notes on Adding a New Sensor</h1>
 <p>In order to allow for meta alerts to be queries alongside regular alerts in Elasticsearch 2.x, it is necessary to add an additional field to the templates and mapping for existing sensors.</p>
 <p>Please see a description of the steps necessary to make this change in the metron-elasticsearch <a href="./metron-platform/metron-elasticsearch/index.html#Using_Metron_with_Elasticsearch_2.x">Using Metron with Elasticsearch 2.x</a></p></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[09/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-stellar/stellar-common/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-stellar/stellar-common/index.html b/site/current-book/metron-stellar/stellar-common/index.html
index 0465dbb..1160156 100644
--- a/site/current-book/metron-stellar/stellar-common/index.html
+++ b/site/current-book/metron-stellar/stellar-common/index.html
@@ -1,3357 +1,2392 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-stellar/stellar-common/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Stellar Language</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Stellar Language</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Stellar Language</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                            
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Stellar-common</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/3rdPartyStellar.html" title="3rdPartyStellar">
-          <i class="none"></i>
-        3rdPartyStellar</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Stellar-common</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-stellar/stellar-common/3rdPartyStellar.html" title="3rdPartyStellar"><span class="none"></span>3rdPartyStellar</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Stellar Language</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Stellar Language</h1>
 <p><a name="Stellar_Language"></a></p>
-<p>For a variety of components (threat intelligence triage and field transformations) we have the need to do simple computation and transformation using the data from messages as variables. For those purposes, there exists a simple, scaled down DSL created to do simple computation and transformation.</p>
-
+<p>For a variety of components (threat intelligence triage and field transformations) we have the need to do simple computation and transformation using the data from messages as variables.  For those purposes, there exists a simple, scaled down DSL created to do simple computation and transformation.</p>
 <ul>
-  
+
 <li><a href="#Introduction">Introduction</a></li>
-  
 <li><a href="#Stellar_Core_Functions">Stellar Core Functions</a></li>
-  
 <li><a href="#Stellar_Benchmarks">Stellar Benchmarks</a></li>
-  
 <li><a href="#Stellar_Shell">Stellar Shell</a>
-  
 <ul>
-    
+
 <li><a href="#Getting_Started">Getting Started</a></li>
-    
 <li><a href="#Command_Line_Options">Command Line Options</a></li>
-    
 <li><a href="#Variable_Assignment">Variable Assignment</a></li>
-    
 <li><a href="#Magic_Commands">Magic Commands</a></li>
-    
 <li><a href="#Advanced_Usage">Advanced Usage</a></li>
-  </ul></li>
-  
+<li><a href="#Implementation">Implementation</a></li>
+</ul>
+</li>
 <li><a href="#Stellar_Configuration">Stellar Configuration</a></li>
 </ul>
 <div class="section">
 <h2><a name="Introduction"></a>Introduction</h2>
 <p>The Stellar language supports the following:</p>
-
 <ul>
-  
+
 <li>Referencing fields in the enriched JSON</li>
-  
 <li>String literals are quoted with either <tt>'</tt> or <tt>&quot;</tt></li>
-  
 <li>String literals support escaping for <tt>'</tt>, <tt>&quot;</tt>, <tt>\t</tt>, <tt>\r</tt>, <tt>\n</tt>, and backslash
-  
 <ul>
-    
+
 <li>The literal <tt>'\'foo\''</tt> would represent <tt>'foo'</tt></li>
-    
 <li>The literal <tt>&quot;\&quot;foo\&quot;&quot;</tt> would represent <tt>&quot;foo&quot;</tt></li>
-    
 <li>The literal <tt>'foo \\ bar'</tt> would represent <tt>foo \ bar</tt></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Simple boolean operations: <tt>and</tt>, <tt>not</tt>, <tt>or</tt></li>
-  
 <li>Simple arithmetic operations: <tt>*</tt>, <tt>/</tt>, <tt>+</tt>, <tt>-</tt> on real numbers or integers</li>
-  
 <li>Simple comparison operations <tt>&lt;</tt>, <tt>&gt;</tt>, <tt>&lt;=</tt>, <tt>&gt;=</tt></li>
-  
 <li>Simple equality comparison operations <tt>==</tt>, <tt>!=</tt></li>
-  
 <li>if/then/else comparisons (i.e. <tt>if var1 &lt; 10 then 'less than 10' else '10 or more'</tt>)</li>
-  
 <li>Simple match evaluations (i.e. <tt>match{ var1 &lt; 10 =&gt; 'warn', var1 &gt;= 10 =&gt; 'critical', default =&gt; 'info'}</tt></li>
-  
 <li>Determining whether a field exists (via <tt>exists</tt>)</li>
-  
 <li>An <tt>in</tt> operator that works like the <tt>in</tt> in Python</li>
-  
 <li>The ability to have parenthesis to make order of operations explicit</li>
-  
 <li>User defined functions, including Lambda expressions</li>
 </ul>
 <div class="section">
 <h3><a name="Stellar_Language_Keywords"></a>Stellar Language Keywords</h3>
 <p>The following keywords need to be single quote escaped in order to be used in Stellar expressions:</p>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
-<tr class="a">
-      
-<th align="center"> </th>
-      
-<th align="center"> </th>
-      
-<th align="center"> </th>
-      
-<th align="center"> </th>
-      
-<th align="center"> </th>
-    </tr>
-  </thead>
-  <tbody>
-    
-<tr class="b">
-      
-<td align="center">not </td>
-      
-<td align="center">else </td>
-      
-<td align="center">exists </td>
-      
-<td align="center">if </td>
-      
-<td align="center">then </td>
-    </tr>
-    
-<tr class="a">
-      
-<td align="center">and </td>
-      
-<td align="center">or </td>
-      
-<td align="center">in </td>
-      
-<td align="center">NaN </td>
-      
-<td align="center">match </td>
-    </tr>
-    
-<tr class="b">
-      
-<td align="center">default </td>
-      
-<td align="center">== </td>
-      
-<td align="center">!= </td>
-      
-<td align="center">&lt;= </td>
-      
-<td align="center">&gt; </td>
-    </tr>
-    
-<tr class="a">
-      
-<td align="center">&gt;= </td>
-      
-<td align="center">+ </td>
-      
-<td align="center">- </td>
-      
-<td align="center">&lt; </td>
-      
-<td align="center">? </td>
-    </tr>
-    
-<tr class="b">
-      
-<td align="center">* </td>
-      
-<td align="center">/ </td>
-      
-<td align="center">, </td>
-      
-<td align="center">{ </td>
-      
-<td align="center">} </td>
-    </tr>
-    
-<tr class="a">
-      
-<td align="center">\=&gt; </td>
-      
-<td align="center"> </td>
-      
-<td align="center"> </td>
-      
-<td align="center"> </td>
-      
-<td align="center"> </td>
-    </tr>
-  </tbody>
+<thead>
+
+<tr class="a">
+<th align="center">               </th>
+<th align="center">               </th>
+<th align="center">             </th>
+<th align="center">             </th>
+<th align="center">             </th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td align="center"> not           </td>
+<td align="center"> else          </td>
+<td align="center"> exists      </td>
+<td align="center"> if          </td>
+<td align="center"> then        </td></tr>
+<tr class="a">
+<td align="center"> and           </td>
+<td align="center"> or            </td>
+<td align="center"> in          </td>
+<td align="center"> NaN         </td>
+<td align="center"> match       </td></tr>
+<tr class="b">
+<td align="center"> default       </td>
+<td align="center"> ==            </td>
+<td align="center"> !=          </td>
+<td align="center"> &lt;=         </td>
+<td align="center"> &gt;          </td></tr>
+<tr class="a">
+<td align="center"> &gt;=           </td>
+<td align="center"> +            </td>
+<td align="center"> -          </td>
+<td align="center"> &lt;          </td>
+<td align="center"> ?           </td></tr>
+<tr class="b">
+<td align="center"> *            </td>
+<td align="center"> /             </td>
+<td align="center"> ,           </td>
+<td align="center"> {          </td>
+<td align="center"> }          </td></tr>
+<tr class="a">
+<td align="center"> =&gt;           </td>
+<td align="center">               </td>
+<td align="center">             </td>
+<td align="center">             </td>
+<td align="center">             </td></tr>
+</tbody>
 </table>
 <p>Using parens such as: &#x201c;foo&#x201d; : &#x201c;&lt;ok&gt;&#x201d; requires escaping; &#x201c;foo&#x201d;: &#x201c;'&lt;ok&gt;'&#x201d;</p></div>
 <div class="section">
-<h3><a name="Stellar_Language_Inclusion_Checks_in_and_not_in"></a>Stellar Language Inclusion Checks (<tt>in</tt> and <tt>not in</tt>)</h3>
-
+<h3><a name="Stellar_Language_Inclusion_Checks_.28in_and_not_in.29"></a>Stellar Language Inclusion Checks (<tt>in</tt> and <tt>not in</tt>)</h3>
 <ol style="list-style-type: decimal">
-  
+
 <li><tt>in</tt> supports string contains. e.g. <tt>'foo' in 'foobar' == true</tt></li>
-  
 <li><tt>in</tt> supports collection contains. e.g. <tt>'foo' in [ 'foo', 'bar' ] == true</tt></li>
-  
 <li><tt>in</tt> supports map key contains. e.g. <tt>'foo' in { 'foo' : 5} == true</tt></li>
-  
 <li><tt>not in</tt> is the negation of the in expression. e.g. <tt>'grok' not in 'foobar' == true</tt></li>
 </ol></div>
 <div class="section">
-<h3><a name="Stellar_Language_Comparisons____"></a>Stellar Language Comparisons (<tt>&lt;</tt>, <tt>&lt;=</tt>, <tt>&gt;</tt>, <tt>&gt;=</tt>)</h3>
-
+<h3><a name="Stellar_Language_Comparisons_.28.3C.2C_.3C.3D.2C_.3E.2C_.3E.3D.29"></a>Stellar Language Comparisons (<tt>&lt;</tt>, <tt>&lt;=</tt>, <tt>&gt;</tt>, <tt>&gt;=</tt>)</h3>
 <ol style="list-style-type: decimal">
-  
+
 <li>If either side of the comparison is null then return false.</li>
-  
 <li>If both values being compared implement number then the following:
-  
 <ul>
-    
+
 <li>If either side is a double then get double value from both sides and compare using given operator.</li>
-    
 <li>Else if either side is a float then get float value from both sides and compare using given operator.</li>
-    
 <li>Else if either side is a long then get long value from both sides and compare using given operator.</li>
-    
 <li>Otherwise get the int value from both sides and compare using given operator.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>If both sides are of the same type and are comparable then use the compareTo method to compare values.</li>
-  
 <li>If none of the above are met then an exception is thrown.</li>
 </ol></div>
 <div class="section">
-<h3><a name="Stellar_Language_Equality_Check__"></a>Stellar Language Equality Check (<tt>==</tt>, <tt>!=</tt>)</h3>
+<h3><a name="Stellar_Language_Equality_Check_.28.3D.3D.2C_.21.3D.29"></a>Stellar Language Equality Check (<tt>==</tt>, <tt>!=</tt>)</h3>
 <p>Below is how the <tt>==</tt> operator is expected to work:</p>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>If either side of the expression is null then check equality using Java&#x2019;s <tt>==</tt> expression.</li>
-  
 <li>Else if both sides of the expression are of Java&#x2019;s type Number then:
-  
 <ul>
-    
+
 <li>If either side of the expression is a double then use the double value of both sides to test equality.</li>
-    
 <li>Else if either side of the expression is a float then use the float value of both sides to test equality.</li>
-    
 <li>Else if either side of the expression is a long then use long value of both sides to test equality.</li>
-    
 <li>Otherwise use int value of both sides to test equality</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Otherwise use equals method compare the left side with the right side.</li>
 </ol>
 <p>The <tt>!=</tt> operator is the negation of the above.</p></div>
 <div class="section">
 <h3><a name="Stellar_Language_Lambda_Expressions"></a>Stellar Language Lambda Expressions</h3>
-<p>Stellar provides the capability to pass lambda expressions to functions which wish to support that layer of indirection. The syntax is:</p>
-
+<p>Stellar provides the capability to pass lambda expressions to functions which wish to support that layer of indirection.  The syntax is:</p>
 <ul>
-  
+
 <li><tt>(named_variables) -&gt; stellar_expression</tt> : Lambda expression with named variables
-  
 <ul>
-    
+
 <li>For instance, the lambda expression which calls <tt>TO_UPPER</tt> on a named argument <tt>x</tt> could be be expressed as <tt>(x) -&gt; TO_UPPER(x)</tt>.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li><tt>var -&gt; stellar_expression</tt> : Lambda expression with a single named variable, <tt>var</tt>
-  
 <ul>
-    
-<li>For instance, the lambda expression which calls <tt>TO_UPPER</tt> on a named argument <tt>x</tt> could be expressed as <tt>x -&gt; TO_UPPER(x)</tt>. Note, this is more succinct but equivalent to the example directly above.</li>
-  </ul></li>
-  
+
+<li>For instance, the lambda expression which calls <tt>TO_UPPER</tt> on a named argument <tt>x</tt> could be expressed as <tt>x -&gt; TO_UPPER(x)</tt>.  Note, this is more succinct but equivalent to the example directly above.</li>
+</ul>
+</li>
 <li><tt>() -&gt; stellar_expression</tt> : Lambda expression with no named variables.
-  
 <ul>
-    
-<li>If no named variables are needed, you may omit the named variable section. For instance, the lambda expression which returns a constant <tt>false</tt> would be <tt>() -&gt; false</tt></li>
-  </ul></li>
-</ul>
-<p>where </p>
 
+<li>If no named variables are needed, you may omit the named variable section.  For instance, the lambda expression which returns a constant <tt>false</tt> would be <tt>() -&gt; false</tt></li>
+</ul>
+</li>
+</ul>
+<p>where</p>
 <ul>
-  
+
 <li><tt>named_variables</tt> is a comma separated list of variables to use in the Stellar expression</li>
-  
 <li><tt>stellar_expression</tt> is an arbitrary stellar expression</li>
 </ul>
 <p>In the core language functions, we support basic functional programming primitives such as</p>
-
 <ul>
-  
-<li><tt>MAP</tt> - Applies a lambda expression over a list of input. For instance <tt>MAP([ 'foo', 'bar'], (x) -&gt; TO_UPPER(x) )</tt> returns <tt>[ 'FOO', 'BAR' ]</tt></li>
-  
-<li><tt>FILTER</tt> - Filters a list by a predicate in the form of a lambda expression. For instance <tt>FILTER([ 'foo', 'bar'], (x ) -&gt; x == 'foo' )</tt> returns <tt>[ 'foo' ]</tt></li>
-  
-<li><tt>REDUCE</tt> - Applies a function over a list of input. For instance <tt>REDUCE([ 1, 2, 3], (sum, x) -&gt; sum + x, 0 )</tt> returns <tt>6</tt></li>
+
+<li><tt>MAP</tt> - Applies a lambda expression over a list of input.  For instance <tt>MAP([ 'foo', 'bar'], (x) -&gt; TO_UPPER(x) )</tt> returns <tt>[ 'FOO', 'BAR' ]</tt></li>
+<li><tt>FILTER</tt> - Filters a list by a predicate in the form of a lambda expression.  For instance <tt>FILTER([ 'foo', 'bar'], (x ) -&gt; x == 'foo' )</tt> returns <tt>[ 'foo' ]</tt></li>
+<li><tt>REDUCE</tt> - Applies a function over a list of input.  For instance <tt>REDUCE([ 1, 2, 3], (sum, x) -&gt; sum + x, 0 )</tt> returns <tt>6</tt></li>
 </ul></div>
 <div class="section">
 <h3><a name="Stellar_Language_Match_Expression"></a>Stellar Language Match Expression</h3>
 <p>Stellar provides the capability to write match expressions, which are similar to switch statements commonly found in c like languages.</p>
 <p>The syntax is:</p>
-
 <ul>
-  
+
 <li><tt>match{ logical_expression1 =&gt; evaluation expression1, logical_expression2 =&gt; evaluation_expression2, default =&gt; default_expression}</tt></li>
 </ul>
 <p>Where:</p>
-
 <ul>
-  
-<li><tt>logical_expression</tt> is a Stellar expression that evaluates to true or false. For instance <tt>var &gt; 0</tt> or <tt>var &gt; 0 AND var2 == 'foo'</tt> or <tt>IF ... THEN ... ELSE</tt></li>
-  
+
+<li><tt>logical_expression</tt> is a Stellar expression that evaluates to true or false.  For instance <tt>var &gt; 0</tt> or <tt>var &gt; 0 AND var2 == 'foo'</tt> or <tt>IF ... THEN ... ELSE</tt></li>
 <li><tt>evaluation_expression</tt> is a Stellar Expression</li>
-  
 <li><tt>default</tt> is a required default return value, should no logical expression match</li>
 </ul>
-
 <blockquote>
-<p>default is required </p>
+
+<p>default is required</p>
 <p>Lambda expressions are supported, but they must be no argument lambdas such as <tt>() -&gt; STATEMENT</tt></p>
 </blockquote>
-
 <ul>
-  
+
 <li>Only the first clause that evaluates to true will be executed.</li>
 </ul></div></div>
 <div class="section">
 <h2><a name="Stellar_Core_Functions"></a>Stellar Core Functions</h2>
-
 <table border="0" class="table table-striped">
-  <thead>
-    
-<tr class="a">
-      
-<th> </th>
-    </tr>
-  </thead>
-  <tbody>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#ABS"> <tt>ABS</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#APPEND_IF_MISSING"> <tt>APPEND_IF_MISSING</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#BIN"> <tt>BIN</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#BLOOM_ADD"> <tt>BLOOM_ADD</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#BLOOM_EXISTS"> <tt>BLOOM_EXISTS</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#BLOOM_INIT"> <tt>BLOOM_INIT</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#BLOOM_MERGE"> <tt>BLOOM_MERGE</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#CEILING"> <tt>CEILING</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#COS"> <tt>COS</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#CHOP"> <tt>CHOP</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#CHOMP"> <tt>CHOMP</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#COUNT_MATCHES"> <tt>COUNT_MATCHES</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#DAY_OF_MONTH"> <tt>DAY_OF_MONTH</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#DAY_OF_WEEK"> <tt>DAY_OF_WEEK</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#DAY_OF_YEAR"> <tt>DAY_OF_YEAR</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#DECODE"> <tt>DECODE</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#DOMAIN_REMOVE_SUBDOMAINS"> <tt>DOMAIN_REMOVE_SUBDOMAINS</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#DOMAIN_REMOVE_TLD"> <tt>DOMAIN_REMOVE_TLD</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#DOMAIN_TO_TLD"> <tt>DOMAIN_TO_TLD</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#ENCODE"> <tt>ENCODE</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#ENDS_WITH"> <tt>ENDS_WITH</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#ENRICHMENT_EXISTS"> <tt>ENRICHMENT_EXISTS</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#ENRICHMENT_GET"> <tt>ENRICHMENT_GET</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#EXP"> <tt>EXP</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#FILL_LEFT"> <tt>FILL_LEFT</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#FILL_RIGHT"> <tt>FILL_RIGHT</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#FILTER"> <tt>FILTER</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#FLOOR"> <tt>FLOOR</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#FORMAT"> <tt>FORMAT</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#FUZZY_LANGS"> <tt>FUZZY_LANGS</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#FUZZY_SCORE"> <tt>FUZZY_SCORE</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#GEO_GET"> <tt>GEO_GET</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#GEOHASH_CENTROID"> <tt>GEOHASH_CENTROID</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#GEOHASH_DIST"> <tt>GEOHASH_DIST</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#GEOHASH_FROM_LATLONG"> <tt>GEOHASH_FROM_LATLONG</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#GEOHASH_FROM_LOC"> <tt>GEOHASH_FROM_LOC</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#GEOHASH_MAX_DIST"> <tt>GEOHASH_MAX_DIST</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#GEOHASH_TO_LATLONG"> <tt>GEOHASH_TO_LATLONG</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#GET"> <tt>GET</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#GET_FIRST"> <tt>GET_FIRST</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#GET_LAST"> <tt>GET_LAST</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#GET_SUPPORTED_ENCODINGS"> <tt>GET_SUPPORTED_ENCODINGS</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#HASH"> <tt>HASH</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#HLLP_ADD"> <tt>HLLP_ADD</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#HLLP_CARDINALITY"> <tt>HLLP_CARDINALITY</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#HLLP_INIT"> <tt>HLLP_INIT</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#HLLP_MERGE"> <tt>HLLP_MERGE</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#IN_SUBNET"> <tt>IN_SUBNET</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#IS_DATE"> <tt>IS_DATE</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#IS_ENCODING"> <tt>IS_ENCODING</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#IS_DOMAIN"> <tt>IS_DOMAIN</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#IS_EMAIL"> <tt>IS_EMAIL</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#IS_EMPTY"> <tt>IS_EMPTY</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#IS_INTEGER"> <tt>IS_INTEGER</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#IS_IP"> <tt>IS_IP</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#IS_NAN"> <tt>IS_NAN</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#IS_URL"> <tt>IS_URL</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#JOIN"> <tt>JOIN</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#KAFKA_GET"> <tt>KAFKA_GET</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#KAFKA_PROPS"> <tt>KAFKA_PROPS</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#KAFKA_PUT"> <tt>KAFKA_PUT</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#KAFKA_TAIL"> <tt>KAFKA_TAIL</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#LENGTH"> <tt>LENGTH</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#LIST_ADD"> <tt>LIST_ADD</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#LOG2"> <tt>LOG2</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#LOG10"> <tt>LOG10</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#LN"> <tt>LN</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#MAAS_GET_ENDPOINT"> <tt>MAAS_GET_ENDPOINT</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#MAAS_MODEL_APPLY"> <tt>MAAS_MODEL_APPLY</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#MAP"> <tt>MAP</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#MAP_EXISTS"> <tt>MAP_EXISTS</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#MAP_GET"> <tt>MAP_GET</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#MAX"> <tt>MAX</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#MIN"> <tt>MIN</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#MONTH"> <tt>MONTH</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#MULTISET_ADD"> <tt>MULTISET_ADD</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#MULTISET_INIT"> <tt>MULTISET_INIT</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#MULTISET_MERGE"> <tt>MULTISET_MERGE</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#MULTISET_REMOVE"> <tt>MULTISET_REMOVE</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#MULTISET_TO_SET"> <tt>MULTISET_TO_SET</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#PREPEND_IF_MISSING"> <tt>PREPEND_IF_MISSING</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#PROFILE_GET"> <tt>PROFILE_GET</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#PROFILE_FIXED"> <tt>PROFILE_FIXED</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#PROFILE_WINDOW"> <tt>PROFILE_WINDOW</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#PROTOCOL_TO_NAME"> <tt>PROTOCOL_TO_NAME</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#REDUCE"> <tt>REDUCE</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#REGEXP_MATCH"> <tt>REGEXP_MATCH</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#REGEXP_GROUP_VAL"> <tt>REGEXP_GROUP_VAL</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#ROUND"> <tt>ROUND</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#SET_ADD"> <tt>SET_ADD</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#SET_INIT"> <tt>SET_INIT</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#SET_MERGE"> <tt>SET_MERGE</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#SET_REMOVE"> <tt>SET_REMOVE</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#SIN"> <tt>SIN</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#SPLIT"> <tt>SPLIT</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#SQRT"> <tt>SQRT</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#STARTS_WITH"> <tt>STARTS_WITH</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_ADD"> <tt>STATS_ADD</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_BIN"> <tt>STATS_BIN</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_COUNT"> <tt>STATS_COUNT</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_GEOMETRIC_MEAN"> <tt>STATS_GEOMETRIC_MEAN</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_INIT"> <tt>STATS_INIT</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_KURTOSIS"> <tt>STATS_KURTOSIS</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_MAX"> <tt>STATS_MAX</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_MEAN"> <tt>STATS_MEAN</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_MERGE"> <tt>STATS_MERGE</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_MIN"> <tt>STATS_MIN</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_PERCENTILE"> <tt>STATS_PERCENTILE</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_POPULATION_VARIANCE"> <tt>STATS_POPULATION_VARIANCE</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_QUADRATIC_MEAN"> <tt>STATS_QUADRATIC_MEAN</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_SD"> <tt>STATS_SD</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_SKEWNESS"> <tt>STATS_SKEWNESS</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_SUM"> <tt>STATS_SUM</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_SUM_LOGS"> <tt>STATS_SUM_LOGS</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_SUM_SQUARES"> <tt>STATS_SUM_SQUARES</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="../../metron-analytics/metron-statistics/index.html#STATS_VARIANCE"> <tt>STATS_VARIANCE</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#STRING_ENTROPY"> <tt>STRING_ENTROPY</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#SUBSTRING"> <tt>SUBSTRING</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#SYSTEM_ENV_GET"> <tt>SYSTEM_ENV_GET</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#SYSTEM_PROPERTY_GET"> <tt>SYSTEM_PROPERTY_GET</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#TAN"> <tt>TAN</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#TLSH_DIST"> <tt>TLSH_DIST</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#TO_DOUBLE"> <tt>TO_DOUBLE</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#TO_EPOCH_TIMESTAMP"> <tt>TO_EPOCH_TIMESTAMP</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#TO_FLOAT"> <tt>TO_FLOAT</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#TO_INTEGER"> <tt>TO_INTEGER</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#TO_JSON_LIST"> <tt>TO_JSON_LIST</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#TO_JSON_MAP"> <tt>TO_JSON_MAP</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#TO_JSON_OBJECT"> <tt>TO_JSON_OBJECT</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#TO_LONG"> <tt>TO_LONG</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#TO_LOWER"> <tt>TO_LOWER</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#TO_STRING"> <tt>TO_STRING</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#TO_UPPER"> <tt>TO_UPPER</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#TRIM"> <tt>TRIM</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#URL_TO_HOST"> <tt>URL_TO_HOST</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#URL_TO_PATH"> <tt>URL_TO_PATH</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#URL_TO_PORT"> <tt>URL_TO_PORT</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#URL_TO_PROTOCOL"> <tt>URL_TO_PROTOCOL</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#WEEK_OF_MONTH"> <tt>WEEK_OF_MONTH</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#WEEK_OF_YEAR"> <tt>WEEK_OF_YEAR</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#YEAR"> <tt>YEAR</tt></a> </td>
-    </tr>
-    
-<tr class="a">
-      
-<td><a href="#ZIP"> <tt>ZIP</tt></a> </td>
-    </tr>
-    
-<tr class="b">
-      
-<td><a href="#ZIP_LONGEST"> <tt>ZIP_LONGEST</tt></a> </td>
-    </tr>
-  </tbody>
+<thead>
+
+<tr class="a">
+<th>                                                                                                    </th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#ABS"> <tt>ABS</tt></a>                                             </td></tr>
+<tr class="a">
+<td> <a href="#APPEND_IF_MISSING"> <tt>APPEND_IF_MISSING</tt></a>                                                         </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#BIN"> <tt>BIN</tt></a>                                             </td></tr>
+<tr class="a">
+<td> <a href="#BLOOM_ADD"> <tt>BLOOM_ADD</tt></a>                                                                         </td></tr>
+<tr class="b">
+<td> <a href="#BLOOM_EXISTS"> <tt>BLOOM_EXISTS</tt></a>                                                                   </td></tr>
+<tr class="a">
+<td> <a href="#BLOOM_INIT"> <tt>BLOOM_INIT</tt></a>                                                                       </td></tr>
+<tr class="b">
+<td> <a href="#BLOOM_MERGE"> <tt>BLOOM_MERGE</tt></a>                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#CEILING"> <tt>CEILING</tt></a>                                                                             </td></tr>
+<tr class="b">
+<td> <a href="#COS"> <tt>COS</tt></a>                                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#CHOP"> <tt>CHOP</tt></a>                                                                                   </td></tr>
+<tr class="b">
+<td> <a href="#CHOMP"> <tt>CHOMP</tt></a>                                                                                 </td></tr>
+<tr class="a">
+<td> <a href="#COUNT_MATCHES"> <tt>COUNT_MATCHES</tt></a>                                                                 </td></tr>
+<tr class="b">
+<td> <a href="#DAY_OF_MONTH"> <tt>DAY_OF_MONTH</tt></a>                                                                   </td></tr>
+<tr class="a">
+<td> <a href="#DAY_OF_WEEK"> <tt>DAY_OF_WEEK</tt></a>                                                                     </td></tr>
+<tr class="b">
+<td> <a href="#DAY_OF_YEAR"> <tt>DAY_OF_YEAR</tt></a>                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#DECODE"> <tt>DECODE</tt></a>                                                                               </td></tr>
+<tr class="b">
+<td> <a href="#DOMAIN_REMOVE_SUBDOMAINS"> <tt>DOMAIN_REMOVE_SUBDOMAINS</tt></a>                                           </td></tr>
+<tr class="a">
+<td> <a href="#DOMAIN_REMOVE_TLD"> <tt>DOMAIN_REMOVE_TLD</tt></a>                                                         </td></tr>
+<tr class="b">
+<td> <a href="#DOMAIN_TO_TLD"> <tt>DOMAIN_TO_TLD</tt></a>                                                                 </td></tr>
+<tr class="a">
+<td> <a href="#ENCODE"> <tt>ENCODE</tt></a>                                                                               </td></tr>
+<tr class="b">
+<td> <a href="#ENDS_WITH"> <tt>ENDS_WITH</tt></a>                                                                         </td></tr>
+<tr class="a">
+<td> <a href="#ENRICHMENT_EXISTS"> <tt>ENRICHMENT_EXISTS</tt></a>                                                         </td></tr>
+<tr class="b">
+<td> <a href="#ENRICHMENT_GET"> <tt>ENRICHMENT_GET</tt></a>                                                               </td></tr>
+<tr class="a">
+<td> <a href="#EXP"> <tt>EXP</tt></a>                                                                                     </td></tr>
+<tr class="b">
+<td> <a href="#FILL_LEFT"> <tt>FILL_LEFT</tt></a>                                                                         </td></tr>
+<tr class="a">
+<td> <a href="#FILL_RIGHT"> <tt>FILL_RIGHT</tt></a>                                                                       </td></tr>
+<tr class="b">
+<td> <a href="#FILTER"> <tt>FILTER</tt></a>                                                                               </td></tr>
+<tr class="a">
+<td> <a href="#FLOOR"> <tt>FLOOR</tt></a>                                                                                 </td></tr>
+<tr class="b">
+<td> <a href="#FORMAT"> <tt>FORMAT</tt></a>                                                                               </td></tr>
+<tr class="a">
+<td> <a href="#FUZZY_LANGS"> <tt>FUZZY_LANGS</tt></a>                                                                     </td></tr>
+<tr class="b">
+<td> <a href="#FUZZY_SCORE"> <tt>FUZZY_SCORE</tt></a>                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#GEO_GET"> <tt>GEO_GET</tt></a>                                                                             </td></tr>
+<tr class="b">
+<td> <a href="#GEOHASH_CENTROID"> <tt>GEOHASH_CENTROID</tt></a>                                                           </td></tr>
+<tr class="a">
+<td> <a href="#GEOHASH_DIST"> <tt>GEOHASH_DIST</tt></a>                                                                   </td></tr>
+<tr class="b">
+<td> <a href="#GEOHASH_FROM_LATLONG"> <tt>GEOHASH_FROM_LATLONG</tt></a>                                                   </td></tr>
+<tr class="a">
+<td> <a href="#GEOHASH_FROM_LOC"> <tt>GEOHASH_FROM_LOC</tt></a>                                                           </td></tr>
+<tr class="b">
+<td> <a href="#GEOHASH_MAX_DIST"> <tt>GEOHASH_MAX_DIST</tt></a>                                                           </td></tr>
+<tr class="a">
+<td> <a href="#GEOHASH_TO_LATLONG"> <tt>GEOHASH_TO_LATLONG</tt></a>                                                       </td></tr>
+<tr class="b">
+<td> <a href="#GET"> <tt>GET</tt></a>                                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#GET_FIRST"> <tt>GET_FIRST</tt></a>                                                                         </td></tr>
+<tr class="b">
+<td> <a href="#GET_LAST"> <tt>GET_LAST</tt></a>                                                                           </td></tr>
+<tr class="a">
+<td> <a href="#GET_SUPPORTED_ENCODINGS"> <tt>GET_SUPPORTED_ENCODINGS</tt></a>                                             </td></tr>
+<tr class="b">
+<td> <a href="#HASH"> <tt>HASH</tt></a>                                                                                   </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#HLLP_ADD"> <tt>HLLP_ADD</tt></a>                                   </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#HLLP_CARDINALITY"> <tt>HLLP_CARDINALITY</tt></a>                   </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#HLLP_INIT"> <tt>HLLP_INIT</tt></a>                                 </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#HLLP_MERGE"> <tt>HLLP_MERGE</tt></a>                               </td></tr>
+<tr class="a">
+<td> <a href="#IN_SUBNET"> <tt>IN_SUBNET</tt></a>                                                                         </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#IT_ENTROPY"> <tt>IT_ENTROPY</tt></a>                               </td></tr>
+<tr class="a">
+<td> <a href="#IS_DATE"> <tt>IS_DATE</tt></a>                                                                             </td></tr>
+<tr class="b">
+<td> <a href="#IS_ENCODING"> <tt>IS_ENCODING</tt></a>                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#IS_DOMAIN"> <tt>IS_DOMAIN</tt></a>                                                                         </td></tr>
+<tr class="b">
+<td> <a href="#IS_EMAIL"> <tt>IS_EMAIL</tt></a>                                                                           </td></tr>
+<tr class="a">
+<td> <a href="#IS_EMPTY"> <tt>IS_EMPTY</tt></a>                                                                           </td></tr>
+<tr class="b">
+<td> <a href="#IS_INTEGER"> <tt>IS_INTEGER</tt></a>                                                                       </td></tr>
+<tr class="a">
+<td> <a href="#IS_IP"> <tt>IS_IP</tt></a>                                                                                 </td></tr>
+<tr class="b">
+<td> <a href="#IS_NAN"> <tt>IS_NAN</tt></a>                                                                               </td></tr>
+<tr class="a">
+<td> <a href="#IS_URL"> <tt>IS_URL</tt></a>                                                                               </td></tr>
+<tr class="b">
+<td> <a href="#JOIN"> <tt>JOIN</tt></a>                                                                                   </td></tr>
+<tr class="a">
+<td> <a href="#KAFKA_GET"> <tt>KAFKA_GET</tt></a>                                                                         </td></tr>
+<tr class="b">
+<td> <a href="#KAFKA_PROPS"> <tt>KAFKA_PROPS</tt></a>                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#KAFKA_PUT"> <tt>KAFKA_PUT</tt></a>                                                                         </td></tr>
+<tr class="b">
+<td> <a href="#KAFKA_TAIL"> <tt>KAFKA_TAIL</tt></a>                                                                       </td></tr>
+<tr class="a">
+<td> <a href="#LENGTH"> <tt>LENGTH</tt></a>                                                                               </td></tr>
+<tr class="b">
+<td> <a href="#LIST_ADD"> <tt>LIST_ADD</tt></a>                                                                           </td></tr>
+<tr class="a">
+<td> <a href="#LOG2"> <tt>LOG2</tt></a>                                                                                   </td></tr>
+<tr class="b">
+<td> <a href="#LOG10"> <tt>LOG10</tt></a>                                                                                 </td></tr>
+<tr class="a">
+<td> <a href="#LN"> <tt>LN</tt></a>                                                                                       </td></tr>
+<tr class="b">
+<td> <a href="#MAAS_GET_ENDPOINT"> <tt>MAAS_GET_ENDPOINT</tt></a>                                                         </td></tr>
+<tr class="a">
+<td> <a href="#MAAS_MODEL_APPLY"> <tt>MAAS_MODEL_APPLY</tt></a>                                                           </td></tr>
+<tr class="b">
+<td> <a href="#MAP"> <tt>MAP</tt></a>                                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#MAP_EXISTS"> <tt>MAP_EXISTS</tt></a>                                                                       </td></tr>
+<tr class="b">
+<td> <a href="#MAP_GET"> <tt>MAP_GET</tt></a>                                                                             </td></tr>
+<tr class="a">
+<td> <a href="#MAX"> <tt>MAX</tt></a>                                                                                     </td></tr>
+<tr class="b">
+<td> <a href="#MIN"> <tt>MIN</tt></a>                                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#MONTH"> <tt>MONTH</tt></a>                                                                                 </td></tr>
+<tr class="b">
+<td> <a href="#MULTISET_ADD"> <tt>MULTISET_ADD</tt></a>                                                                   </td></tr>
+<tr class="a">
+<td> <a href="#MULTISET_INIT"> <tt>MULTISET_INIT</tt></a>                                                                 </td></tr>
+<tr class="b">
+<td> <a href="#MULTISET_MERGE"> <tt>MULTISET_MERGE</tt></a>                                                               </td></tr>
+<tr class="a">
+<td> <a href="#MULTISET_REMOVE"> <tt>MULTISET_REMOVE</tt></a>                                                             </td></tr>
+<tr class="b">
+<td> <a href="#MULTISET_TO_SET"> <tt>MULTISET_TO_SET</tt></a>                                                             </td></tr>
+<tr class="a">
+<td> <a href="#OBJECT_GET"> <tt>OBJECT_GET</tt></a>                                                                       </td></tr>
+<tr class="b">
+<td> <a href="#PREPEND_IF_MISSING"> <tt>PREPEND_IF_MISSING</tt></a>                                                       </td></tr>
+<tr class="a">
+<td> <a href="#PROFILE_GET"> <tt>PROFILE_GET</tt></a>                                                                     </td></tr>
+<tr class="b">
+<td> <a href="#PROFILE_FIXED"> <tt>PROFILE_FIXED</tt></a>                                                                 </td></tr>
+<tr class="a">
+<td> <a href="#PROFILE_WINDOW"> <tt>PROFILE_WINDOW</tt></a>                                                               </td></tr>
+<tr class="b">
+<td> <a href="#PROTOCOL_TO_NAME"> <tt>PROTOCOL_TO_NAME</tt></a>                                                           </td></tr>
+<tr class="a">
+<td> <a href="#REDUCE"> <tt>REDUCE</tt></a>                                                                               </td></tr>
+<tr class="b">
+<td> <a href="#REGEXP_MATCH"> <tt>REGEXP_MATCH</tt></a>                                                                   </td></tr>
+<tr class="a">
+<td> <a href="#REGEXP_GROUP_VAL"> <tt>REGEXP_GROUP_VAL</tt></a>                                                           </td></tr>
+<tr class="b">
+<td> <a href="#REGEXP_REPLACE"> <tt>REGEXP_REPLACE</tt></a>                                                               </td></tr>
+<tr class="a">
+<td> <a href="#ROUND"> <tt>ROUND</tt></a>                                                                                 </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#SAMPLE_ADD"> <tt>SAMPLE_ADD</tt></a>                               </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#SAMPLE_GET"> <tt>SAMPLE_GET</tt></a>                               </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#SAMPLE_INIT"> <tt>SAMPLE_INIT</tt></a>                             </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#SAMPLE_MERGE"> <tt>SAMPLE_MERGE</tt></a>                           </td></tr>
+<tr class="b">
+<td> <a href="#SET_ADD"> <tt>SET_ADD</tt></a>                                                                             </td></tr>
+<tr class="a">
+<td> <a href="#SET_INIT"> <tt>SET_INIT</tt></a>                                                                           </td></tr>
+<tr class="b">
+<td> <a href="#SET_MERGE"> <tt>SET_MERGE</tt></a>                                                                         </td></tr>
+<tr class="a">
+<td> <a href="#SET_REMOVE"> <tt>SET_REMOVE</tt></a>                                                                       </td></tr>
+<tr class="b">
+<td> <a href="#SIN"> <tt>SIN</tt></a>                                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#SHELL_EDIT"> <tt>SHELL_EDIT</tt></a>                                                                       </td></tr>
+<tr class="b">
+<td> <a href="#SHELL_GET_EXPRESSION"> <tt>SHELL_GET_EXPRESSION</tt></a>                                                   </td></tr>
+<tr class="a">
+<td> <a href="#SHELL_LIST_VARS"> <tt>SHELL_LIST_VARS</tt></a>                                                             </td></tr>
+<tr class="b">
+<td> <a href="#SHELL_MAP2TABLE"> <tt>SHELL_MAP2TABLE</tt></a>                                                             </td></tr>
+<tr class="a">
+<td> <a href="#SHELL_VARS2MAP"> <tt>SHELL_VARS2MAP</tt></a>                                                               </td></tr>
+<tr class="b">
+<td> <a href="#SPLIT"> <tt>SPLIT</tt></a>                                                                                 </td></tr>
+<tr class="a">
+<td> <a href="#SQRT"> <tt>SQRT</tt></a>                                                                                   </td></tr>
+<tr class="b">
+<td> <a href="#STARTS_WITH"> <tt>STARTS_WITH</tt></a>                                                                     </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_ADD"> <tt>STATS_ADD</tt></a>                                 </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_BIN"> <tt>STATS_BIN</tt></a>                                 </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_COUNT"> <tt>STATS_COUNT</tt></a>                             </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_GEOMETRIC_MEAN"> <tt>STATS_GEOMETRIC_MEAN</tt></a>           </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_INIT"> <tt>STATS_INIT</tt></a>                               </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_KURTOSIS"> <tt>STATS_KURTOSIS</tt></a>                       </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_MAX"> <tt>STATS_MAX</tt></a>                                 </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_MEAN"> <tt>STATS_MEAN</tt></a>                               </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_MERGE"> <tt>STATS_MERGE</tt></a>                             </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_MIN"> <tt>STATS_MIN</tt></a>                                 </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_PERCENTILE"> <tt>STATS_PERCENTILE</tt></a>                   </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_POPULATION_VARIANCE"> <tt>STATS_POPULATION_VARIANCE</tt></a> </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_QUADRATIC_MEAN"> <tt>STATS_QUADRATIC_MEAN</tt></a>           </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_SD"> <tt>STATS_SD</tt></a>                                   </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_SKEWNESS"> <tt>STATS_SKEWNESS</tt></a>                       </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_SUM"> <tt>STATS_SUM</tt></a>                                 </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_SUM_LOGS"> <tt>STATS_SUM_LOGS</tt></a>                       </td></tr>
+<tr class="b">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_SUM_SQUARES"> <tt>STATS_SUM_SQUARES</tt></a>                 </td></tr>
+<tr class="a">
+<td> <a href="../../metron-analytics/metron-statistics/index.html#STATS_VARIANCE"> <tt>STATS_VARIANCE</tt></a>                       </td></tr>
+<tr class="b">
+<td> <a href="#STRING_ENTROPY"> <tt>STRING_ENTROPY</tt></a>                                                               </td></tr>
+<tr class="a">
+<td> <a href="#SUBSTRING"> <tt>SUBSTRING</tt></a>                                                                         </td></tr>
+<tr class="b">
+<td> <a href="#SYSTEM_ENV_GET"> <tt>SYSTEM_ENV_GET</tt></a>                                                               </td></tr>
+<tr class="a">
+<td> <a href="#SYSTEM_PROPERTY_GET"> <tt>SYSTEM_PROPERTY_GET</tt></a>                                                     </td></tr>
+<tr class="b">
+<td> <a href="#TAN"> <tt>TAN</tt></a>                                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#TLSH_DIST"> <tt>TLSH_DIST</tt></a>                                                                         </td></tr>
+<tr class="b">
+<td> <a href="#TO_DOUBLE"> <tt>TO_DOUBLE</tt></a>                                                                         </td></tr>
+<tr class="a">
+<td> <a href="#TO_EPOCH_TIMESTAMP"> <tt>TO_EPOCH_TIMESTAMP</tt></a>                                                       </td></tr>
+<tr class="b">
+<td> <a href="#TO_FLOAT"> <tt>TO_FLOAT</tt></a>                                                                           </td></tr>
+<tr class="a">
+<td> <a href="#TO_INTEGER"> <tt>TO_INTEGER</tt></a>                                                                       </td></tr>
+<tr class="b">
+<td> <a href="#TO_JSON_LIST"> <tt>TO_JSON_LIST</tt></a>                                                                   </td></tr>
+<tr class="a">
+<td> <a href="#TO_JSON_MAP"> <tt>TO_JSON_MAP</tt></a>                                                                     </td></tr>
+<tr class="b">
+<td> <a href="#TO_JSON_OBJECT"> <tt>TO_JSON_OBJECT</tt></a>                                                               </td></tr>
+<tr class="a">
+<td> <a href="#TO_LONG"> <tt>TO_LONG</tt></a>                                                                             </td></tr>
+<tr class="b">
+<td> <a href="#TO_LOWER"> <tt>TO_LOWER</tt></a>                                                                           </td></tr>
+<tr class="a">
+<td> <a href="#TO_STRING"> <tt>TO_STRING</tt></a>                                                                         </td></tr>
+<tr class="b">
+<td> <a href="#TO_UPPER"> <tt>TO_UPPER</tt></a>                                                                           </td></tr>
+<tr class="a">
+<td> <a href="#TRIM"> <tt>TRIM</tt></a>                                                                                   </td></tr>
+<tr class="b">
+<td> <a href="#URL_TO_HOST"> <tt>URL_TO_HOST</tt></a>                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#URL_TO_PATH"> <tt>URL_TO_PATH</tt></a>                                                                     </td></tr>
+<tr class="b">
+<td> <a href="#URL_TO_PORT"> <tt>URL_TO_PORT</tt></a>                                                                     </td></tr>
+<tr class="a">
+<td> <a href="#URL_TO_PROTOCOL"> <tt>URL_TO_PROTOCOL</tt></a>                                                             </td></tr>
+<tr class="b">
+<td> <a href="#WEEK_OF_MONTH"> <tt>WEEK_OF_MONTH</tt></a>                                                                 </td></tr>
+<tr class="a">
+<td> <a href="#WEEK_OF_YEAR"> <tt>WEEK_OF_YEAR</tt></a>                                                                   </td></tr>
+<tr class="b">
+<td> <a href="#YEAR"> <tt>YEAR</tt></a>                                                                                   </td></tr>
+<tr class="a">
+<td> <a href="#ZIP"> <tt>ZIP</tt></a>                                                                                     </td></tr>
+<tr class="b">
+<td> <a href="#ZIP_LONGEST"> <tt>ZIP_LONGEST</tt></a>                                                                     </td></tr>
+</tbody>
 </table>
 <div class="section">
 <h3><a name="APPEND_IF_MISSING"></a><tt>APPEND_IF_MISSING</tt></h3>
-
 <ul>
-  
+
 <li>Description: Appends the suffix to the end of the string if the string does not already end with any of the suffixes.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>string - The string to be appended.</li>
-    
 <li>suffix - The string suffix to append to the end of the string.</li>
-    
 <li>additionalsuffix - Optional - Additional string suffix that is a valid terminator.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: A new String if prefix was prepended, the same string otherwise.</li>
 </ul></div>
 <div class="section">
 <h3><a name="BLOOM_ADD"></a><tt>BLOOM_ADD</tt></h3>
-
 <ul>
-  
+
 <li>Description: Adds an element to the bloom filter passed in</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>bloom - The bloom filter</li>
-    
 <li>value(s) - The value(s) to add</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: Bloom Filter</li>
 </ul></div>
 <div class="section">
 <h3><a name="BLOOM_EXISTS"></a><tt>BLOOM_EXISTS</tt></h3>
-
 <ul>
-  
+
 <li>Description: If the bloom filter contains the value</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>bloom - The bloom filter</li>
-    
 <li>value - The value to check</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: True if the filter might contain the value and false otherwise</li>
 </ul></div>
 <div class="section">
 <h3><a name="BLOOM_INIT"></a><tt>BLOOM_INIT</tt></h3>
-
 <ul>
-  
+
 <li>Description: Returns an empty bloom filter</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>expectedInsertions - The expected insertions</li>
-    
 <li>falsePositiveRate - The false positive rate you are willing to tolerate</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: Bloom Filter</li>
 </ul></div>
 <div class="section">
 <h3><a name="BLOOM_MERGE"></a><tt>BLOOM_MERGE</tt></h3>
-
 <ul>
-  
+
 <li>Description: Returns a merged bloom filter</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>bloomfilters - A list of bloom filters to merge</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: Bloom Filter or null if the list is empty</li>
 </ul></div>
 <div class="section">
 <h3><a name="CEILING"></a><tt>CEILING</tt></h3>
-
 <ul>
-  
+
 <li>Description: Returns the ceiling of a number.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>number - The number to take the ceiling of</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The ceiling of the number passed in.</li>
 </ul></div>
 <div class="section">
 <h3><a name="COS"></a><tt>COS</tt></h3>
-
 <ul>
-  
+
 <li>Description: Returns the cosine of a number.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>number - The number to take the cosine of</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The cosine of the number passed in.</li>
 </ul></div>
 <div class="section">
 <h3><a name="CHOP"></a><tt>CHOP</tt></h3>
-
 <ul>
-  
+
 <li>Description: Remove the last character from a String</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>string - the String to chop last character from, may be null</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: String without last character, null if null String input</li>
 </ul></div>
 <div class="section">
 <h3><a name="CHOMP"></a><tt>CHOMP</tt></h3>
-
 <ul>
-  
+
 <li>Description: Removes one newline from end of a String if it&#x2019;s there, otherwise leave it alone. A newline is &#x201c;\n&#x201d;, &#x201c;\r&#x201d;, or &#x201c;\r\n&#x201d;</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>string - the String to chomp a newline from, may be null</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: String without newline, null if null String input</li>
 </ul></div>
 <div class="section">
 <h3><a name="COUNT_MATCHES"></a><tt>COUNT_MATCHES</tt></h3>
-
 <ul>
-  
+
 <li>Description: Counts how many times the substring appears in the larger string.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>string - the CharSequence to check, may be null.</li>
-    
 <li>substring/character - the substring or character to count, may be null.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: the number of non-overlapping occurrences, 0 if either CharSequence is null.</li>
 </ul></div>
 <div class="section">
 <h3><a name="DAY_OF_MONTH"></a><tt>DAY_OF_MONTH</tt></h3>
-
 <ul>
-  
-<li>Description: The numbered day within the month. The first day within the month has a value of 1.</li>
-  
+
+<li>Description: The numbered day within the month.  The first day within the month has a value of 1.</li>
 <li>Input:
-  
 <ul>
-    
+
 <li>dateTime - The datetime as a long representing the milliseconds since unix epoch</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The numbered day within the month.</li>
 </ul></div>
 <div class="section">
 <h3><a name="DAY_OF_WEEK"></a><tt>DAY_OF_WEEK</tt></h3>
-
 <ul>
-  
-<li>Description: The numbered day within the week. The first day of the week, Sunday, has a value of 1.</li>
-  
+
+<li>Description: The numbered day within the week.  The first day of the week, Sunday, has a value of 1.</li>
 <li>Input:
-  
 <ul>
-    
+
 <li>dateTime - The datetime as a long representing the milliseconds since unix epoch</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The numbered day within the week.</li>
 </ul></div>
 <div class="section">
 <h3><a name="DAY_OF_YEAR"></a><tt>DAY_OF_YEAR</tt></h3>
-
 <ul>
-  
-<li>Description: The day number within the year. The first day of the year has value of 1.</li>
-  
+
+<li>Description: The day number within the year.  The first day of the year has value of 1.</li>
 <li>Input:
-  
 <ul>
-    
+
 <li>dateTime - The datetime as a long representing the milliseconds since unix epoch</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The day number within the year.</li>
 </ul></div>
 <div class="section">
 <h3><a name="DECODE"></a><tt>DECODE</tt></h3>
-
 <ul>
-  
-<li>Description: Decodes the passed string with the provided encoding, which  must be one of the encodings returned from <a href="#GET_SUPPORTED_ENCODINGS"> <tt>GET_SUPPORTED_ENCODINGS</tt></a></li>
-  
+
+<li>Description: Decodes the passed string with the provided encoding, which must be one of the encodings returned from <a href="#GET_SUPPORTED_ENCODINGS"> <tt>GET_SUPPORTED_ENCODINGS</tt></a></li>
 <li>Input:
-  
 <ul>
-    
+
 <li>string - the string to decode</li>
-    
 <li>encoding - the encoding to use, must be one of encodings returned from <a href="#GET_SUPPORTED_ENCODINGS"> <tt>GET_SUPPORTED_ENCODINGS</tt></a></li>
-    
-<li>verify - (optional), true or false to determine if string should be verified as being  encoded with the passed encoding</li>
-  </ul></li>
-  
+<li>verify - (optional), true or false to determine if string should be verified as being encoded with the passed encoding</li>
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>The decoded string on success</li>
-    
 <li>The original string the string cannot be decoded</li>
-    
 <li>null on usage error</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h3><a name="DOMAIN_REMOVE_SUBDOMAINS"></a><tt>DOMAIN_REMOVE_SUBDOMAINS</tt></h3>
-
 <ul>
-  
+
 <li>Description: Removes the subdomains from a domain.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>domain - Fully qualified domain name</li>
-  </ul></li>
-  
-<li>Returns: The domain without the subdomains. (for example, DOMAIN_REMOVE_SUBDOMAINS(&#x2018;mail.yahoo.com&#x2019;) yields &#x2018;yahoo.com&#x2019;)</li>
+</ul>
+</li>
+<li>Returns: The domain without the subdomains.  (for example, DOMAIN_REMOVE_SUBDOMAINS(&#x2018;mail.yahoo.com&#x2019;) yields &#x2018;yahoo.com&#x2019;)</li>
 </ul></div>
 <div class="section">
 <h3><a name="DOMAIN_REMOVE_TLD"></a><tt>DOMAIN_REMOVE_TLD</tt></h3>
-
 <ul>
-  
+
 <li>Description: Removes the top level domain (TLD) suffix from a domain.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>domain - Fully qualified domain name</li>
-  </ul></li>
-  
-<li>Returns: The domain without the TLD. (for example, DOMAIN_REMOVE_TLD(&#x2018;mail.yahoo.co.uk&#x2019;) yields &#x2018;mail.yahoo&#x2019;)</li>
+</ul>
+</li>
+<li>Returns: The domain without the TLD.  (for example, DOMAIN_REMOVE_TLD(&#x2018;mail.yahoo.co.uk&#x2019;) yields &#x2018;mail.yahoo&#x2019;)</li>
 </ul></div>
 <div class="section">
 <h3><a name="DOMAIN_TO_TLD"></a><tt>DOMAIN_TO_TLD</tt></h3>
-
 <ul>
-  
+
 <li>Description: Extracts the top level domain from a domain</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>domain - Fully qualified domain name</li>
-  </ul></li>
-  
-<li>Returns: The TLD of the domain. (for example, DOMAIN_TO_TLD(&#x2018;mail.yahoo.co.uk&#x2019;) yields &#x2018;co.uk&#x2019;)</li>
+</ul>
+</li>
+<li>Returns: The TLD of the domain.  (for example, DOMAIN_TO_TLD(&#x2018;mail.yahoo.co.uk&#x2019;) yields &#x2018;co.uk&#x2019;)</li>
 </ul></div>
 <div class="section">
 <h3><a name="ENCODE"></a><tt>ENCODE</tt></h3>
-
 <ul>
-  
-<li>Description: Encodes the passed string with the provided encoding, which  must be one of the encodings returned from <a href="#GET_SUPPORTED_ENCODINGS"> <tt>GET_SUPPORTED_ENCODINGS</tt></a></li>
-  
+
+<li>Description: Encodes the passed string with the provided encoding, which must be one of the encodings returned from <a href="#GET_SUPPORTED_ENCODINGS"> <tt>GET_SUPPORTED_ENCODINGS</tt></a></li>
 <li>Input:
-  
 <ul>
-    
+
 <li>string - the string to encode</li>
-    
 <li>encoding - the encoding to use, must be one of encodings returned from <a href="#GET_SUPPORTED_ENCODINGS"> <tt>GET_SUPPORTED_ENCODINGS</tt></a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns:
-  
 <ul>
-    
+
 <li>The encoded string on success</li>
-    
 <li>null on error</li>
-  </ul></li>
+</ul>
+</li>
 </ul></div>
 <div class="section">
 <h3><a name="ENDS_WITH"></a><tt>ENDS_WITH</tt></h3>
-
 <ul>
-  
+
 <li>Description: Determines whether a string ends with a specified suffix</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>string - The string to test</li>
-    
 <li>suffix - The proposed suffix</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: True if the string ends with the specified suffix and false if otherwise</li>
 </ul></div>
 <div class="section">
 <h3><a name="ENRICHMENT_EXISTS"></a><tt>ENRICHMENT_EXISTS</tt></h3>
-
 <ul>
-  
+
 <li>Description: Interrogates the HBase table holding the simple hbase enrichment data and returns whether the enrichment type and indicator are in the table.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>enrichment_type - The enrichment type</li>
-    
 <li>indicator - The string indicator to look up</li>
-    
 <li>nosql_table - The NoSQL Table to use</li>
-    
 <li>column_family - The Column Family to use</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: True if the enrichment indicator exists and false otherwise</li>
 </ul></div>
 <div class="section">
 <h3><a name="ENRICHMENT_GET"></a><tt>ENRICHMENT_GET</tt></h3>
-
 <ul>
-  
+
 <li>Description: Interrogates the HBase table holding the simple hbase enrichment data and retrieves the tabular value associated with the enrichment type and indicator.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>enrichment_type - The enrichment type</li>
-    
 <li>indicator - The string indicator to look up</li>
-    
 <li>nosql_table - The NoSQL Table to use</li>
-    
 <li>column_family - The Column Family to use</li>
-  </ul></li>
-  
-<li>Returns: A Map associated with the indicator and enrichment type. Empty otherwise.</li>
+</ul>
+</li>
+<li>Returns: A Map associated with the indicator and enrichment type.  Empty otherwise.</li>
 </ul></div>
 <div class="section">
 <h3><a name="EXP"></a><tt>EXP</tt></h3>
-
 <ul>
-  
+
 <li>Description: Returns Euler&#x2019;s number raised to the power of the argument.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>number - The power to which e is raised.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: Euler&#x2019;s number raised to the power of the argument.</li>
 </ul></div>
 <div class="section">
 <h3><a name="FILL_LEFT"></a><tt>FILL_LEFT</tt></h3>
-
 <ul>
-  
+
 <li>Description: Fills or pads a given string with a given character, to a given length on the left</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>input - string</li>
-    
 <li>fill - the fill character</li>
-    
 <li>len - the required length</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: the filled string</li>
 </ul></div>
 <div class="section">
 <h3><a name="FILL_RIGHT"></a><tt>FILL_RIGHT</tt></h3>
-
 <ul>
-  
+
 <li>Description: Fills or pads a given string with a given character, to a given length on the right</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>input - string</li>
-    
 <li>fill - the fill character string</li>
-    
 <li>len - the required length</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: Last element of the list</li>
 </ul></div>
 <div class="section">
 <h3><a name="FILTER"></a><tt>FILTER</tt></h3>
-
 <ul>
-  
+
 <li>Description: Applies a filter in the form of a lambda expression to a list. e.g. <tt>FILTER( [ 'foo', 'bar' ] , (x) -&gt; x == 'foo')</tt> would yield <tt>[ 'foo']</tt></li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>list - List of arguments.</li>
-    
-<li>predicate - The lambda expression to apply. This expression is assumed to take one argument and return a boolean.</li>
-  </ul></li>
-  
+<li>predicate - The lambda expression to apply.  This expression is assumed to take one argument and return a boolean.</li>
+</ul>
+</li>
 <li>Returns: The input list filtered by the predicate.</li>
 </ul></div>
 <div class="section">
 <h3><a name="FLOOR"></a><tt>FLOOR</tt></h3>
-
 <ul>
-  
+
 <li>Description: Returns the floor of a number.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>number - The number to take the floor of</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The floor of the number passed in.</li>
 </ul></div>
 <div class="section">
 <h3><a name="FORMAT"></a><tt>FORMAT</tt></h3>
-
 <ul>
-  
+
 <li>Description: Returns a formatted string using the specified format string and arguments. Uses Java&#x2019;s string formatting conventions.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>format - string</li>
-    
 <li>arguments&#x2026; - object(s)</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: A formatted string.</li>
 </ul></div>
 <div class="section">
 <h3><a name="FUZZY_LANGS"></a><tt>FUZZY_LANGS</tt></h3>
-
 <ul>
-  
+
 <li>Description: Returns a list of IETF BCP 47 available to the system, such as en, fr, de.</li>
-  
 <li>Returns: A list of IEF BGP 47 language tag strings</li>
 </ul></div>
 <div class="section">
 <h3><a name="FUZZY_SCORE"></a><tt>FUZZY_SCORE</tt></h3>
-
 <ul>
-  
+
 <li>Description: Returns the Fuzzy Score which indicates the similarity score between two strings. One point is given for every matched character. Subsequent matches yield two bonus points. A higher score indicates a higher similarity.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>string - The full term that should be matched against.</li>
-    
 <li>string - The query that will be matched against a term.</li>
-    
 <li>string - The IETF BCP 47 language code to use.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: An Integer representing the score.</li>
 </ul></div>
 <div class="section">
 <h3><a name="GEO_GET"></a><tt>GEO_GET</tt></h3>
-
 <ul>
-  
+
 <li>Description: Look up an IPV4 address and returns geographic information about it</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>ip - The IPV4 address to lookup</li>
-    
 <li>fields - Optional list of GeoIP fields to grab. Options are locID, country, city postalCode, dmaCode, latitude, longitude, location_point</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: If a Single field is requested a string of the field, If multiple fields a map of string of the fields, and null otherwise</li>
 </ul></div>
 <div class="section">
 <h3><a name="GEOHASH_CENTROID"></a><tt>GEOHASH_CENTROID</tt></h3>
-
 <ul>
-  
+
 <li>Description: Compute the centroid (geographic midpoint or center of gravity) of a set of <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohashes</a></li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>hashes - A collection of <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohashes</a> or a map associating geohashes to numeric weights</li>
-    
 <li>character_precision? - The number of characters to use in the hash. Default is 12</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The geohash of the centroid</li>
 </ul></div>
 <div class="section">
 <h3><a name="GEOHASH_DIST"></a><tt>GEOHASH_DIST</tt></h3>
-
 <ul>
-  
+
 <li>Description: Compute the distance between <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohashes</a></li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>hash1 - The first point as a geohash</li>
-    
 <li>hash2 - The second point as a geohash</li>
-    
-<li>strategy? - The great circle distance strategy to use. One of <a class="externalLink" href="https://en.wikipedia.org/wiki/Haversine_formula">HAVERSINE</a>, <a class="externalLink" href="https://en.wikipedia.org/wiki/Law_of_cosines#Using_the_distance_formula">LAW_OF_COSINES</a>, or <a class="externalLink" href="https://en.wikipedia.org/wiki/Vincenty%27s_formulae">VICENTY</a>. Haversine is default.</li>
-  </ul></li>
-  
+<li>strategy? - The great circle distance strategy to use.  One of <a class="externalLink" href="https://en.wikipedia.org/wiki/Haversine_formula">HAVERSINE</a>, <a class="externalLink" href="https://en.wikipedia.org/wiki/Law_of_cosines#Using_the_distance_formula">LAW_OF_COSINES</a>, or <a class="externalLink" href="https://en.wikipedia.org/wiki/Vincenty%27s_formulae">VICENTY</a>.  Haversine is default.</li>
+</ul>
+</li>
 <li>Returns: The distance in kilometers between the hashes.</li>
 </ul></div>
 <div class="section">
 <h3><a name="GEOHASH_FROM_LATLONG"></a><tt>GEOHASH_FROM_LATLONG</tt></h3>
-
 <ul>
-  
+
 <li>Description: Compute <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohash</a> given a lat/long</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>latitude - The latitude</li>
-    
 <li>longitude - The longitude</li>
-    
 <li>character_precision? - The number of characters to use in the hash. Default is 12</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: A <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohash</a> of the lat/long</li>
 </ul></div>
 <div class="section">
 <h3><a name="GEOHASH_FROM_LOC"></a><tt>GEOHASH_FROM_LOC</tt></h3>
-
 <ul>
-  
+
 <li>Description: Compute <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohash</a> given a geo enrichment location</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>map - the latitude and logitude in a map (the output of <a href="#GEO_GET">GEO_GET</a> )</li>
-    
 <li>longitude - The longitude</li>
-    
 <li>character_precision? - The number of characters to use in the hash. Default is <tt>12</tt></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: A <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohash</a> of the location</li>
 </ul></div>
 <div class="section">
 <h3><a name="GEOHASH_MAX_DIST"></a><tt>GEOHASH_MAX_DIST</tt></h3>
-
 <ul>
-  
+
 <li>Description: Compute the maximum distance among a list of <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohashes</a></li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>hashes - A set of <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohashes</a></li>
-    
-<li>strategy? - The great circle distance strategy to use. One of <a class="externalLink" href="https://en.wikipedia.org/wiki/Haversine_formula">HAVERSINE</a>, <a class="externalLink" href="https://en.wikipedia.org/wiki/Law_of_cosines#Using_the_distance_formula">LAW_OF_COSINES</a>, or <a class="externalLink" href="https://en.wikipedia.org/wiki/Vincenty%27s_formulae">VICENTY</a>. Haversine is default.</li>
-  </ul></li>
-  
+<li>strategy? - The great circle distance strategy to use. One of <a class="externalLink" href="https://en.wikipedia.org/wiki/Haversine_formula">HAVERSINE</a>, <a class="externalLink" href="https://en.wikipedia.org/wiki/Law_of_cosines#Using_the_distance_formula">LAW_OF_COSINES</a>, or <a class="externalLink" href="https://en.wikipedia.org/wiki/Vincenty%27s_formulae">VICENTY</a>.  Haversine is default.</li>
+</ul>
+</li>
 <li>Returns: The maximum distance in kilometers between any two locations</li>
 </ul></div>
 <div class="section">
 <h3><a name="GEOHASH_TO_LATLONG"></a><tt>GEOHASH_TO_LATLONG</tt></h3>
-
 <ul>
-  
+
 <li>Description: Compute the lat/long of a given <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohash</a></li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>hash - The <a class="externalLink" href="https://en.wikipedia.org/wiki/Geohash">geohash</a></li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: A map containing the latitude and longitude of the hash (keys &#x201c;latitude&#x201d; and &#x201c;longitude&#x201d;)</li>
 </ul></div>
 <div class="section">
 <h3><a name="GET"></a><tt>GE

<TRUNCATED>

[36/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-analytics/metron-profiler/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/metron-profiler/index.html b/site/current-book/metron-analytics/metron-profiler/index.html
index c3bf7ad..fe498af 100644
--- a/site/current-book/metron-analytics/metron-profiler/index.html
+++ b/site/current-book/metron-analytics/metron-profiler/index.html
@@ -1,295 +1,168 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-analytics/metron-profiler/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Metron Profiler</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Metron Profiler</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron Profiler</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                                
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-down"></i>
-        Analytics</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service">
-          <i class="none"></i>
-        Maas-service</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Profiler</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client">
-          <i class="none"></i>
-        Profiler-client</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-analytics/metron-statistics/index.html" title="Statistics">
-          <i class="icon-chevron-right"></i>
-        Statistics</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-down"></span>Analytics</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service"><span class="none"></span>Maas-service</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Profiler</a></li>
+    <li><a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client"><span class="none"></span>Profiler-client</a></li>
+    <li><a href="../../metron-analytics/metron-statistics/index.html" title="Statistics"><span class="icon-chevron-right"></span>Statistics</a></li>
+    </ul>
+</li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Metron Profiler</h1>
-<p><a name="Metron_Profiler"></a></p>
-<p>The Profiler is a feature extraction mechanism that can generate a profile describing the behavior of an entity. An entity might be a server, user, subnet or application. Once a profile has been generated defining what normal behavior looks-like, models can be built that identify anomalous behavior.</p>
-<p>This is achieved by summarizing the streaming telemetry data consumed by Metron over sliding windows. A summary statistic is applied to the data received within a given window. Collecting this summary across many windows results in a time series that is useful for analysis.</p>
-<p>Any field contained within a message can be used to generate a profile. A profile can even be produced by combining fields that originate in different data sources. A user has considerable power to transform the data used in a profile by leveraging the Stellar language. A user only need configure the desired profiles and ensure that the Profiler topology is running.</p>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
 
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron Profiler</h1>
+<p><a name="Metron_Profiler"></a></p>
+<p>The Profiler is a feature extraction mechanism that can generate a profile describing the behavior of an entity.  An entity might be a server, user, subnet or application. Once a profile has been generated defining what normal behavior looks-like, models can be built that identify anomalous behavior.</p>
+<p>This is achieved by summarizing the streaming telemetry data consumed by Metron over sliding windows. A summary statistic is applied to the data received within a given window.  Collecting this summary across many windows results in a time series that is useful for analysis.</p>
+<p>Any field contained within a message can be used to generate a profile.  A profile can even be produced by combining fields that originate in different data sources.  A user has considerable power to transform the data used in a profile by leveraging the Stellar language. A user only need configure the desired profiles and ensure that the Profiler topology is running.</p>
 <ul>
-  
+
 <li><a href="#Installation">Installation</a></li>
-  
 <li><a href="#Creating_Profiles">Creating Profiles</a></li>
-  
 <li><a href="#Deploying_Profiles">Deploying Profiles</a></li>
-  
 <li><a href="#Anatomy_of_a_Profile">Anatomy of a Profile</a></li>
-  
 <li><a href="#Configuring_the_Profiler">Configuring the Profiler</a></li>
-  
 <li><a href="#Examples">Examples</a></li>
-  
 <li><a href="#Implementation">Implementation</a></li>
 </ul>
 <div class="section">
 <h2><a name="Installation"></a>Installation</h2>
 <p>The Profiler can be installed with either of these two methods.</p>
-
 <ul>
-  
+
 <li><a href="#Ambari_Installation">Ambari Installation</a></li>
-  
 <li><a href="#Manual_Installation">Manual Installation</a></li>
 </ul>
 <div class="section">
 <h3><a name="Ambari_Installation"></a>Ambari Installation</h3>
-<p>The Metron Profiler is installed automatically when installing Metron using the Ambari MPack. You can skip the <a href="#Installation">Installation</a> section and move ahead to <a href="#Creating_Profiles">Creating Profiles</a> should this be the case.</p></div>
+<p>The Metron Profiler is installed automatically when installing Metron using the Ambari MPack.  You can skip the <a href="#Installation">Installation</a> section and move ahead to <a href="#Creating_Profiles">Creating Profiles</a> should this be the case.</p></div>
 <div class="section">
 <h3><a name="Manual_Installation"></a>Manual Installation</h3>
-<p>This section will describe the steps necessary to manually install the Profiler on an RPM-based Linux distribution. This assumes that core Metron has already been installed and validated. If you installed Metron using the <a href="#Ambari_MPack">Ambari MPack</a>, then the Profiler has already been installed and you can skip this section.</p>
-
+<p>This section will describe the steps necessary to manually install the Profiler on an RPM-based Linux distribution.  This assumes that core Metron has already been installed and validated.  If you installed Metron using the <a href="#Ambari_MPack">Ambari MPack</a>, then the Profiler has already been installed and you can skip this section.</p>
 <ol style="list-style-type: decimal">
-  
+
 <li>
+
 <p>Build the Metron RPMs (see Building the <a href="../../metron-deployment/index.html#RPMs">RPMs</a>).</p>
 <p>You may have already built the Metron RPMs when core Metron was installed.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ find metron-deployment/ -name &quot;metron-profiler*.rpm&quot;
+
+<div>
+<div>
+<pre class="source">$ find metron-deployment/ -name &quot;metron-profiler*.rpm&quot;
 metron-deployment//packaging/docker/rpm-docker/RPMS/noarch/metron-profiler-0.4.1-201707131420.noarch.rpm
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Copy the Profiler RPM to the installation host. </p>
-<p>The installation host must be the same host on which core Metron was installed. Depending on how you installed Metron, the Profiler RPM might have already been copied to this host with the other Metron RPMs.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[root@node1 ~]# find /localrepo/  -name &quot;metron-profiler*.rpm&quot;
+
+<p>Copy the Profiler RPM to the installation host.</p>
+<p>The installation host must be the same host on which core Metron was installed.  Depending on how you installed Metron, the Profiler RPM might have already been copied to this host with the other Metron RPMs.</p>
+
+<div>
+<div>
+<pre class="source">[root@node1 ~]# find /localrepo/  -name &quot;metron-profiler*.rpm&quot;
 /localrepo/metron-profiler-0.4.1-201707112313.noarch.rpm
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Install the RPM.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[root@node1 ~]# rpm -ivh metron-profiler-*.noarch.rpm
+
+<div>
+<div>
+<pre class="source">[root@node1 ~]# rpm -ivh metron-profiler-*.noarch.rpm
 Preparing...                ########################################### [100%]
    1:metron-profiler        ########################################### [100%]
 </pre></div></div>
-  
-<div class="source">
-<div class="source">
-<pre>[root@node1 ~]# rpm -ql metron-profiler
+
+<div>
+<div>
+<pre class="source">[root@node1 ~]# rpm -ql metron-profiler
 /usr/metron
 /usr/metron/0.4.2
 /usr/metron/0.4.2/bin
@@ -301,67 +174,71 @@ Preparing...                ########################################### [100%]
 /usr/metron/0.4.2/flux/profiler/remote.yaml
 /usr/metron/0.4.2/lib
 /usr/metron/0.4.2/lib/metron-profiler-0.4.2-uber.jar
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Edit the configuration file located at <tt>$METRON_HOME/config/profiler.properties</tt>. </p>
-  
-<div class="source">
-<div class="source">
-<pre>kafka.zk=node1:2181
+
+<p>Edit the configuration file located at <tt>$METRON_HOME/config/profiler.properties</tt>.</p>
+
+<div>
+<div>
+<pre class="source">kafka.zk=node1:2181
 kafka.broker=node1:6667
 </pre></div></div>
-  
+
 <ul>
-    
-<li></li>
-    
-<li></li>
-  </ul></li>
-  
+
+<li>Change <tt>kafka.zk</tt> to refer to Zookeeper in your environment.</li>
+<li>Change <tt>kafka.broker</tt> to refer to a Kafka Broker in your environment.</li>
+</ul>
+</li>
 <li>
-<p>Create a table within HBase that will store the profile data. By default, the table is named <tt>profiler</tt> with a column family <tt>P</tt>. The table name and column family must match the Profiler&#x2019;s configuration (see <a href="#Configuring_the_Profiler">Configuring the Profiler</a>). </p>
-  
-<div class="source">
-<div class="source">
-<pre>$ /usr/hdp/current/hbase-client/bin/hbase shell
+
+<p>Create a table within HBase that will store the profile data. By default, the table is named <tt>profiler</tt> with a column family <tt>P</tt>.  The table name and column family must match the Profiler&#x2019;s configuration (see <a href="#Configuring_the_Profiler">Configuring the Profiler</a>).</p>
+
+<div>
+<div>
+<pre class="source">$ /usr/hdp/current/hbase-client/bin/hbase shell
 hbase(main):001:0&gt; create 'profiler', 'P'
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Start the Profiler topology.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ cd $METRON_HOME
+
+<div>
+<div>
+<pre class="source">$ cd $METRON_HOME
 $ bin/start_profiler_topology.sh
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol>
-<p>At this point the Profiler is running and consuming telemetry messages. We have not defined any profiles yet, so it is not doing anything very useful. The next section walks you through the steps to create your very first &#x201c;Hello, World!&#x201d; profile.</p></div></div>
+<p>At this point the Profiler is running and consuming telemetry messages.  We have not defined any profiles yet, so it is not doing anything very useful.  The next section walks you through the steps to create your very first &#x201c;Hello, World!&#x201d; profile.</p></div></div>
 <div class="section">
 <h2><a name="Creating_Profiles"></a>Creating Profiles</h2>
-<p>This section will describe how to create your very first &#x201c;Hello, World&#x201d; profile. It will also outline a useful workflow for creating, testing, and deploying profiles.</p>
-<p>Creating and refining profiles is an iterative process. Iterating against a live stream of data is slow, difficult and error prone. The Profile Debugger was created to provide a controlled and isolated execution environment to create, refine and troubleshoot profiles.</p>
-
+<p>This section will describe how to create your very first &#x201c;Hello, World&#x201d; profile.  It will also outline a useful workflow for creating, testing, and deploying profiles.</p>
+<p>Creating and refining profiles is an iterative process.  Iterating against a live stream of data is slow, difficult and error prone.  The Profile Debugger was created to provide a controlled and isolated execution environment to create, refine and troubleshoot profiles.</p>
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Launch the Stellar Shell. We will leverage the Profiler Debugger from within the Stellar Shell.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[root@node1 ~]# $METRON_HOME/bin/stellar
+
+<p>Launch the Stellar Shell.  We will leverage the Profiler Debugger from within the Stellar Shell.</p>
+
+<div>
+<div>
+<pre class="source">[root@node1 ~]# $METRON_HOME/bin/stellar
 Stellar, Go!
 [Stellar]&gt;&gt;&gt; %functions PROFILER
 PROFILER_APPLY, PROFILER_FLUSH, PROFILER_INIT
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Create a simple <tt>hello-world</tt> profile that will count the number of messages for each <tt>ip_src_addr</tt>. The <tt>SHELL_EDIT</tt> function will open an editor in which you can copy/paste the following Profiler configuration.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; conf := SHELL_EDIT()
+
+<p>Create a simple <tt>hello-world</tt> profile that will count the number of messages for each <tt>ip_src_addr</tt>.  The <tt>SHELL_EDIT</tt> function will open an editor in which you can copy/paste the following Profiler configuration.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; conf := SHELL_EDIT()
 [Stellar]&gt;&gt;&gt; conf
 {
   &quot;profiles&quot;: [
@@ -375,123 +252,126 @@ PROFILER_APPLY, PROFILER_FLUSH, PROFILER_INIT
     }
   ]
 }
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Create a Profile execution environment; the Profile Debugger. </p>
-<p>The Profiler will output the number of profiles that have been defined, the number of messages that have been applied and the number of routes that have been followed. </p>
+
+<p>Create a Profile execution environment; the Profile Debugger.</p>
+<p>The Profiler will output the number of profiles that have been defined, the number of messages that have been applied and the number of routes that have been followed.</p>
 <p>A route is defined when a message is applied to a specific profile.</p>
-  
 <ul>
-    
+
 <li>If a message is not needed by any profile, then there are no routes.</li>
-    
 <li>If a message is needed by one profile, then one route has been followed.</li>
-    
 <li>If a message is needed by two profiles, then two routes have been followed.</li>
-  </ul>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; p := PROFILER_INIT(conf)
-[Stellar]&gt;&gt;&gt; p
+</ul>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; profiler := PROFILER_INIT(conf)
+[Stellar]&gt;&gt;&gt; profiler
 Profiler{1 profile(s), 0 messages(s), 0 route(s)}
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Create a message that mimics the telemetry that your profile will consume. </p>
-<p>This message can be as simple or complex as you like. For the <tt>hello-world</tt> profile, all you need is a message containing an <tt>ip_src_addr</tt> field.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; msg := SHELL_EDIT()
+
+<p>Create a message that mimics the telemetry that your profile will consume.</p>
+<p>This message can be as simple or complex as you like.  For the <tt>hello-world</tt> profile, all you need is a message containing an <tt>ip_src_addr</tt> field.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; msg := SHELL_EDIT()
 [Stellar]&gt;&gt;&gt; msg
 {
 	&quot;ip_src_addr&quot;: &quot;10.0.0.1&quot;
 }
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Apply the message to your Profiler, as many times as you like.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; PROFILER_APPLY(msg, p)
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; PROFILER_APPLY(msg, profiler)
 Profiler{1 profile(s), 1 messages(s), 1 route(s)}
 </pre></div></div>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; PROFILER_APPLY(msg, p)
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; PROFILER_APPLY(msg, profiler)
 Profiler{1 profile(s), 2 messages(s), 2 route(s)}
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Flush the Profiler. </p>
-<p>A flush is what occurs at the end of each 15 minute period in the Profiler. The result is a list of Profile Measurements. Each measurement is a map containing detailed information about the profile data that has been generated. The <tt>value</tt> field is what is written to HBase when running this profile in the Profiler topology. </p>
-<p>There will always be one measurement for each [profile, entity] pair. This profile simply counts the number of messages by IP source address. Notice that the value is &#x2018;3&#x2019; for the entity &#x2018;10.0.0.1&#x2019; as we applied 3 messages with an &#x2018;ip_src_addr&#x2019; of &#x2019;10.0.0.1&#x2019;.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; values := PROFILER_FLUSH(profiler)
+
+<p>Flush the Profiler.</p>
+<p>A flush is what occurs at the end of each 15 minute period in the Profiler.  The result is a list of Profile Measurements. Each measurement is a map containing detailed information about the profile data that has been generated. The <tt>value</tt> field is what is written to HBase when running this profile in the Profiler topology.</p>
+<p>There will always be one measurement for each [profile, entity] pair.  This profile simply counts the number of messages by IP source address. Notice that the value is &#x2018;3&#x2019; for the entity &#x2018;10.0.0.1&#x2019; as we applied 3 messages with an &#x2018;ip_src_addr&#x2019; of &#x2019;10.0.0.1&#x2019;.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; values := PROFILER_FLUSH(profiler)
 [Stellar]&gt;&gt;&gt; values
 [{period={duration=900000, period=1669628, start=1502665200000, end=1502666100000},
 profile=hello-world, groups=[], value=3, entity=10.0.0.1}]
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>Apply real, live telemetry to your profile.</p>
-<p>Once you are happy with your profile against a controlled data set, it can be useful to introduce more complex, live data. This example extracts 10 messages of live, enriched telemetry to test your profile(s).</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; %define bootstrap.servers := &quot;node1:6667&quot; 
+<p>Once you are happy with your profile against a controlled data set, it can be useful to introduce more complex, live data.  This example extracts 10 messages of live, enriched telemetry to test your profile(s).</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; %define bootstrap.servers := &quot;node1:6667&quot;
 node1:6667
-[Stellar]&gt;&gt;&gt; msgs := KAFKA_GET(&quot;indexing&quot;, 10) 
+[Stellar]&gt;&gt;&gt; msgs := KAFKA_GET(&quot;indexing&quot;, 10)
 [Stellar]&gt;&gt;&gt; LENGTH(msgs)
 10
 </pre></div></div>
+
 <p>Apply those 10 messages to your profile(s).</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; PROFILER_APPLY(msgs, p)
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; PROFILER_APPLY(msgs, profiler)
   Profiler{1 profile(s), 10 messages(s), 10 route(s)}
-</pre></div></div></li>
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h2><a name="Deploying_Profiles"></a>Deploying Profiles</h2>
-<p>This section will describe the steps required to get your first &#x201c;Hello, World!&#x201d;&quot; profile running. This assumes that you have a successful Profiler <a href="#Installation">Installation</a> and have it running. You can deploy profiles in two different ways.</p>
-
+<p>This section will describe the steps required to get your first &#x201c;Hello, World!&#x201d;&quot; profile running.  This assumes that you have a successful Profiler <a href="#Installation">Installation</a> and have it running.  You can deploy profiles in two different ways.</p>
 <ul>
-  
+
 <li><a href="#Deploying_Profiles_with_the_Stellar_Shell">Deploying Profiles with the Stellar Shell</a></li>
-  
 <li><a href="#Deploying_Profiles_from_the_Command_Line">Deploying Profiles from the Command Line</a></li>
 </ul>
 <div class="section">
 <h3><a name="Deploying_Profiles_with_the_Stellar_Shell"></a>Deploying Profiles with the Stellar Shell</h3>
-<p>Continuing the previous running example, at this point, you have seen how your profile behaves against real, live telemetry in a controlled execution environment. The next step is to deploy your profile to the live, actively running Profiler topology.</p>
-
+<p>Continuing the previous running example, at this point, you have seen how your profile behaves against real, live telemetry in a controlled execution environment.  The next step is to deploy your profile to the live, actively running Profiler topology.</p>
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Start the Stellar Shell with the <tt>-z ZK:2181</tt> command line argument. This is required when deploying a new profile to the active Profiler topology. Replace <tt>ZK:2181</tt> with a URL that is appropriate to your environment.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[root@node1 ~]# $METRON_HOME/bin/stellar -z ZK:2181
+
+<p>Start the Stellar Shell with the <tt>-z ZK:2181</tt> command line argument.  This is required when deploying a new profile to the active Profiler topology.  Replace <tt>ZK:2181</tt> with a URL that is appropriate to your environment.</p>
+
+<div>
+<div>
+<pre class="source">[root@node1 ~]# $METRON_HOME/bin/stellar -z ZK:2181
 Stellar, Go!
 [Stellar]&gt;&gt;&gt;
 [Stellar]&gt;&gt;&gt; %functions CONFIG CONFIG_GET, CONFIG_PUT
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
+
 <p>If you haven&#x2019;t already, define your profile.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; conf := SHELL_EDIT()
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; conf := SHELL_EDIT()
 [Stellar]&gt;&gt;&gt; conf
 {
   &quot;profiles&quot;: [
@@ -505,37 +385,40 @@ Stellar, Go!
     }
   ]
 }
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Check what is already deployed. </p>
-<p>Pushing a new profile configuration is destructive. It will overwrite any existing configuration. Check what you have out there. Manually merge the existing configuration with your new profile definition.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; existing := CONFIG_GET(&quot;PROFILER&quot;)
-</pre></div></div></li>
-  
+
+<p>Check what is already deployed.</p>
+<p>Pushing a new profile configuration is destructive.  It will overwrite any existing configuration.  Check what you have out there.  Manually merge the existing configuration with your new profile definition.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; existing := CONFIG_GET(&quot;PROFILER&quot;)
+</pre></div></div>
+</li>
 <li>
-<p>Deploy your profile. This will push the configuration to to the live, actively running Profiler topology. This will overwrite any existing profile definitions.</p>
-  
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; CONFIG_PUT(&quot;PROFILER&quot;, conf)
-</pre></div></div></li>
+
+<p>Deploy your profile.  This will push the configuration to to the live, actively running Profiler topology.  This will overwrite any existing profile definitions.</p>
+
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; CONFIG_PUT(&quot;PROFILER&quot;, conf)
+</pre></div></div>
+</li>
 </ol></div>
 <div class="section">
 <h3><a name="Deploying_Profiles_from_the_Command_Line"></a>Deploying Profiles from the Command Line</h3>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Create the profile definition in a file located at <tt>$METRON_HOME/config/zookeeper/profiler.json</tt>. This file will likely not exist, if you have never created Profiles before.</p>
+
+<p>Create the profile definition in a file located at <tt>$METRON_HOME/config/zookeeper/profiler.json</tt>.  This file will likely not exist, if you have never created Profiles before.</p>
 <p>The following example will create a profile that simply counts the number of messages per <tt>ip_src_addr</tt>.</p>
-  
-<div class="source">
-<div class="source">
-<pre>{
+
+<div>
+<div>
+<pre class="source">{
   &quot;profiles&quot;: [
     {
       &quot;profile&quot;: &quot;hello-world&quot;,
@@ -547,21 +430,23 @@ Stellar, Go!
     }
   ]
 }
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Upload the profile definition to Zookeeper. Change <tt>node1:2181</tt> to refer the actual Zookeeper host in your environment.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ cd $METRON_HOME
+
+<p>Upload the profile definition to Zookeeper.  Change <tt>node1:2181</tt> to refer the actual Zookeeper host in your environment.</p>
+
+<div>
+<div>
+<pre class="source">$ cd $METRON_HOME
 $ bin/zk_load_configs.sh -m PUSH -i config/zookeeper/ -z node1:2181
 </pre></div></div>
+
 <p>You can validate this by reading back the Metron configuration from Zookeeper using the same script. The result should look-like the following.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ bin/zk_load_configs.sh -m DUMP -z node1:2181
+
+<div>
+<div>
+<pre class="source">$ bin/zk_load_configs.sh -m DUMP -z node1:2181
 ...
 PROFILER Config: profiler
 {
@@ -576,378 +461,386 @@ PROFILER Config: profiler
     }
   ]
 }
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Ensure that test messages are being sent to the Profiler&#x2019;s input topic in Kafka. The Profiler will consume messages from the input topic defined in the Profiler&#x2019;s configuration (see <a href="#Configuring_the_Profiler">Configuring the Profiler</a>). By default this is the <tt>indexing</tt> topic.</p></li>
-  
+
+<p>Ensure that test messages are being sent to the Profiler&#x2019;s input topic in Kafka.  The Profiler will consume messages from the input topic defined in the Profiler&#x2019;s configuration (see <a href="#Configuring_the_Profiler">Configuring the Profiler</a>).  By default this is the <tt>indexing</tt> topic.</p>
+</li>
 <li>
-<p>Check the HBase table to validate that the Profiler is writing the profile. Remember that the Profiler is flushing the profile every 15 minutes. You will need to wait at least this long to start seeing profile data in HBase.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ /usr/hdp/current/hbase-client/bin/hbase shell
+
+<p>Check the HBase table to validate that the Profiler is writing the profile.  Remember that the Profiler is flushing the profile every 15 minutes.  You will need to wait at least this long to start seeing profile data in HBase.</p>
+
+<div>
+<div>
+<pre class="source">$ /usr/hdp/current/hbase-client/bin/hbase shell
 hbase(main):001:0&gt; count 'profiler'
-</pre></div></div></li>
-  
+</pre></div></div>
+</li>
 <li>
-<p>Use the <a href="../metron-profiler-client/index.html">Profiler Client</a> to read the profile data. The following <tt>PROFILE_GET</tt> command will read the data written by the <tt>hello-world</tt> profile. This assumes that <tt>10.0.0.1</tt> is one of the values for <tt>ip_src_addr</tt> contained within the telemetry consumed by the Profiler.</p>
-  
-<div class="source">
-<div class="source">
-<pre>$ bin/stellar -z node1:2181
+
+<p>Use the <a href="../metron-profiler-client/index.html">Profiler Client</a> to read the profile data.  The following <tt>PROFILE_GET</tt> command will read the data written by the <tt>hello-world</tt> profile. This assumes that <tt>10.0.0.1</tt> is one of the values for <tt>ip_src_addr</tt> contained within the telemetry consumed by the Profiler.</p>
+
+<div>
+<div>
+<pre class="source">$ bin/stellar -z node1:2181
 [Stellar]&gt;&gt;&gt; PROFILE_GET( &quot;hello-world&quot;, &quot;10.0.0.1&quot;, PROFILE_FIXED(30, &quot;MINUTES&quot;))
 [451, 448]
 </pre></div></div>
-<p>This result indicates that over the past 30 minutes, the Profiler stored two values related to the source IP address &#x201c;10.0.0.1&#x201d;. In the first 15 minute period, the IP <tt>10.0.0.1</tt> was seen in 451 telemetry messages. In the second 15 minute period, the same IP was seen in 448 telemetry messages.</p>
-<p>It is assumed that the <tt>PROFILE_GET</tt> client is correctly configured to match the Profile configuration before using it to read that Profile. More information on configuring and using the Profiler client can be found <a href="../metron-profiler-client/index.html">here</a>. </p></li>
+
+<p>This result indicates that over the past 30 minutes, the Profiler stored two values related to the source IP address &#x201c;10.0.0.1&#x201d;.  In the first 15 minute period, the IP <tt>10.0.0.1</tt> was seen in 451 telemetry messages.  In the second 15 minute period, the same IP was seen in 448 telemetry messages.</p>
+<p>It is assumed that the <tt>PROFILE_GET</tt> client is correctly configured to match the Profile configuration before using it to read that Profile.  More information on configuring and using the Profiler client can be found <a href="../metron-profiler-client/index.html">here</a>.</p>
+</li>
 </ol></div></div>
 <div class="section">
 <h2><a name="Anatomy_of_a_Profile"></a>Anatomy of a Profile</h2>
-<p>A profile definition requires a JSON-formatted set of elements, many of which can contain Stellar code. The specification contains the following elements. (For the impatient, skip ahead to the <a href="#Examples">Examples</a>.)</p>
+<div class="section">
+<h3><a name="Profiler"></a>Profiler</h3>
+<p>The Profiler configuration contains only two fields; only one of which is required.</p>
+
+<div>
+<div>
+<pre class="source">{
+    &quot;profiles&quot;: [
+        { &quot;profile&quot;: &quot;one&quot;, ... },
+        { &quot;profile&quot;: &quot;two&quot;, ... }
+    ],
+    &quot;timestampField&quot;: &quot;timestamp&quot;
+}
+</pre></div></div>
 
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
 <tr class="a">
-      
-<th>Name </th>
-      
-<th> </th>
-      
-<th>Description</th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<th> Name                              </th>
+<th>               </th>
+<th> Description</th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td><a href="#profile">profile</a> </td>
-      
-<td>Required </td>
-      
-<td>Unique name identifying the profile.</td>
-    </tr>
-    
+<td> <a href="#profiles">profiles</a>             </td>
+<td> Required      </td>
+<td> A list of zero or more Profile definitions.</td></tr>
 <tr class="a">
-      
-<td><a href="#foreach">foreach</a> </td>
-      
-<td>Required </td>
-      
-<td>A separate profile is maintained &#x201c;for each&#x201d; of these.</td>
-    </tr>
-    
+<td> <a href="#timestampField">timestampField</a> </td>
+<td> Optional      </td>
+<td> Indicates whether processing time or event time should be used. By default, processing time is enabled.</td></tr>
+</tbody>
+</table>
+<div class="section">
+<h4><a name="profiles"></a><tt>profiles</tt></h4>
+<p><i>Required</i></p>
+<p>A list of zero or more Profile definitions.</p></div>
+<div class="section">
+<h4><a name="timestampField"></a><tt>timestampField</tt></h4>
+<p><i>Optional</i></p>
+<p>Indicates whether processing time or event time is used. By default, processing time is enabled.</p>
+<div class="section">
+<h5><a name="Processing_Time"></a>Processing Time</h5>
+<p>By default, no <tt>timestampField</tt> is defined.  In this case, the Profiler uses system time when generating profiles.  This means that the profiles are generated based on when the data has been processed by the Profiler.  This is also known as &#x2018;processing time&#x2019;.</p>
+<p>This is the simplest mode of operation, but has some draw backs.  If the Profiler is consuming live data and all is well, the processing and event times will likely remain similar and consistent. If processing time diverges from event time, then the Profiler will generate skewed profiles.</p>
+<p>There are a few scenarios that might cause skewed profiles when using processing time.  For example when a system has undergone a scheduled maintenance window and is restarted, a high volume of messages will need to be processed by the Profiler. The output of the Profiler might indicate an increase in activity during this time, although no change in activity actually occurred on the target network. The same situation could occur if an upstream system which provides telemetry undergoes an outage.</p>
+<p><a href="#Event_Time">Event Time</a> can be used to mitigate these problems.</p></div>
+<div class="section">
+<h5><a name="Event_Time"></a>Event Time</h5>
+<p>Alternatively, a <tt>timestampField</tt> can be defined.  This must be the name of a field contained within the telemetry processed by the Profiler.  The Profiler will extract and use the timestamp contained within this field.</p>
+<ul>
+
+<li>
+
+<p>If a message does not contain this field, it will be dropped.</p>
+</li>
+<li>
+
+<p>The field must contain a timestamp in epoch milliseconds expressed as either a numeric or string. Otherwise, the message will be dropped.</p>
+</li>
+<li>
+
+<p>The Profiler will use the same field across all telemetry sources and for all profiles.</p>
+</li>
+<li>
+
+<p>Be aware of clock skew across telemetry sources.  If your profile is processing telemetry from multiple sources where the clock differs significantly, the Profiler may assume that some of those messages are late and will be ignored.  Adjusting the <a href="#profiler.window.duration"><tt>profiler.window.duration</tt></a> and <a href="#profiler.window.lag"><tt>profiler.window.lag</tt></a> can help accommodate skewed clocks.</p>
+</li>
+</ul></div></div></div>
+<div class="section">
+<h3><a name="Profiles"></a>Profiles</h3>
+<p>A profile definition requires a JSON-formatted set of elements, many of which can contain Stellar code.  The specification contains the following elements.  (For the impatient, skip ahead to the <a href="#Examples">Examples</a>.)</p>
+<table border="0" class="table table-striped">
+<thead>
+
+<tr class="a">
+<th> Name                          </th>
+<th>               </th>
+<th> Description</th></tr>
+</thead><tbody>
+
 <tr class="b">
-      
-<td><a href="#onlyif">onlyif</a> </td>
-      
-<td>Optional </td>
-      
-<td>Boolean expression that determines if a message should be applied to the profile.</td>
-    </tr>
-    
+<td> <a href="#profile">profile</a>           </td>
+<td> Required      </td>
+<td> Unique name identifying the profile.</td></tr>
 <tr class="a">
-      
-<td><a href="#groupBy">groupBy</a> </td>
-      
-<td>Optional </td>
-      
-<td>One or more Stellar expressions used to group the profile measurements when persisted.</td>
-    </tr>
-    
+<td> <a href="#foreach">foreach</a>           </td>
+<td> Required      </td>
+<td> A separate profile is maintained &#x201c;for each&#x201d; of these.</td></tr>
 <tr class="b">
-      
-<td><a href="#init">init</a> </td>
-      
-<td>Optional </td>
-      
-<td>One or more expressions executed at the start of a window period.</td>
-    </tr>
-    
+<td> <a href="#onlyif">onlyif</a>             </td>
+<td> Optional      </td>
+<td> Boolean expression that determines if a message should be applied to the profile.</td></tr>
 <tr class="a">
-      
-<td><a href="#update">update</a> </td>
-      
-<td>Required </td>
-      
-<td>One or more expressions executed when a message is applied to the profile.</td>
-    </tr>
-    
+<td> <a href="#groupBy">groupBy</a>           </td>
+<td> Optional      </td>
+<td> One or more Stellar expressions used to group the profile measurements when persisted.</td></tr>
 <tr class="b">
-      
-<td><a href="#result">result</a> </td>
-      
-<td>Required </td>
-      
-<td>Stellar expressions that are executed when the window period expires.</td>
-    </tr>
-    
+<td> <a href="#init">init</a>                 </td>
+<td> Optional      </td>
+<td> One or more expressions executed at the start of a window period.</td></tr>
 <tr class="a">
-      
-<td><a href="#expires">expires</a> </td>
-      
-<td>Optional </td>
-      
-<td>Profile data is purged after this period of time, specified in days.</td>
-    </tr>
-  </tbody>
-</table>
+<td> <a href="#update">update</a>             </td>
+<td> Required      </td>
+<td> One or more expressions executed when a message is applied to the profile.</td></tr>
+<tr class="b">
+<td> <a href="#result">result</a>             </td>
+<td> Required      </td>
+<td> Stellar expressions that are executed when the window period expires.</td></tr>
+<tr class="a">
+<td> <a href="#expires">expires</a>           </td>
+<td> Optional      </td>
+<td> Profile data is purged after this period of time, specified in days.</td></tr>
+</tbody>
+</table></div>
 <div class="section">
 <h3><a name="profile"></a><tt>profile</tt></h3>
 <p><i>Required</i></p>
-<p>A unique name identifying the profile. The field is treated as a string.</p></div>
+<p>A unique name identifying the profile.  The field is treated as a string.</p></div>
 <div class="section">
 <h3><a name="foreach"></a><tt>foreach</tt></h3>
 <p><i>Required</i></p>
-<p>A separate profile is maintained &#x2018;for each&#x2019; of these. This is effectively the entity that the profile is describing. The field is expected to contain a Stellar expression whose result is the entity name. </p>
+<p>A separate profile is maintained &#x2018;for each&#x2019; of these.  This is effectively the entity that the profile is describing.  The field is expected to contain a Stellar expression whose result is the entity name.</p>
 <p>For example, if <tt>ip_src_addr</tt> then a separate profile would be maintained for each unique IP source address in the data; 10.0.0.1, 10.0.0.2, etc.</p></div>
 <div class="section">
 <h3><a name="onlyif"></a><tt>onlyif</tt></h3>
 <p><i>Optional</i></p>
-<p>An expression that determines if a message should be applied to the profile. A Stellar expression that returns a Boolean is expected. A message is only applied to a profile if this expression is true. This allows a profile to filter the messages that get applied to it.</p></div>
+<p>An expression that determines if a message should be applied to the profile.  A Stellar expression that returns a Boolean is expected.  A message is only applied to a profile if this expression is true. This allows a profile to filter the messages that get applied to it.</p></div>
 <div class="section">
 <h3><a name="groupBy"></a><tt>groupBy</tt></h3>
 <p><i>Optional</i></p>
-<p>One or more Stellar expressions used to group the profile measurements when persisted. This can be used to sort the Profile data to allow for a contiguous scan when accessing subsets of the data. This is also one way to deal with calendar effects. For example, where activity on a weekday can be very different from a weekend.</p>
-<p>A common use case would be grouping by day of week. This allows a contiguous scan to access all profile data for Mondays only. Using the following definition would achieve this.</p>
+<p>One or more Stellar expressions used to group the profile measurements when persisted. This can be used to sort the Profile data to allow for a contiguous scan when accessing subsets of the data.  This is also one way to deal with calendar effects.  For example, where activity on a weekday can be very different from a weekend.</p>
+<p>A common use case would be grouping by day of week.  This allows a contiguous scan to access all profile data for Mondays only.  Using the following definition would achieve this.</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;groupBy&quot;: [ &quot;DAY_OF_WEEK(start)&quot; ]
+<div>
+<div>
+<pre class="source">&quot;groupBy&quot;: [ &quot;DAY_OF_WEEK(start)&quot; ]
 </pre></div></div>
-<p>The expression can reference any of these variables.</p>
 
+<p>The expression can reference any of these variables.</p>
 <ul>
-  
+
 <li>Any variable defined by the profile in its <tt>init</tt> or <tt>update</tt> expressions.</li>
-  
 <li><tt>profile</tt> The name of the profile.</li>
-  
 <li><tt>entity</tt> The name of the entity being profiled.</li>
-  
 <li><tt>start</tt> The start time of the profile period in epoch milliseconds.</li>
-  
 <li><tt>end</tt> The end time of the profile period in epoch milliseconds.</li>
-  
 <li><tt>duration</tt> The duration of the profile period in milliseconds.</li>
-  
 <li><tt>result</tt> The result of executing the <tt>result</tt> expression.</li>
 </ul></div>
 <div class="section">
 <h3><a name="init"></a><tt>init</tt></h3>
 <p><i>Optional</i></p>
-<p>One or more expressions executed at the start of a window period. A map is expected where the key is the variable name and the value is a Stellar expression. The map can contain zero or more variable:expression pairs. At the start of each window period, each expression is executed once and stored in the given variable. Note that constant init values such as &#x201c;0&#x201d; must be in quotes regardless of their type, as the init value must be a string to be executed by Stellar.</p>
+<p>One or more expressions executed at the start of a window period.  A map is expected where the key is the variable name and the value is a Stellar expression.  The map can contain zero or more variable:expression pairs. At the start of each window period, each expression is executed once and stored in the given variable. Note that constant init values such as &#x201c;0&#x201d; must be in quotes regardless of their type, as the init value must be a string to be executed by Stellar.</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;init&quot;: {
+<div>
+<div>
+<pre class="source">&quot;init&quot;: {
   &quot;var1&quot;: &quot;0&quot;,
   &quot;var2&quot;: &quot;1&quot;
 }
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="update"></a><tt>update</tt></h3>
 <p><i>Required</i></p>
-<p>One or more expressions executed when a message is applied to the profile. A map is expected where the key is the variable name and the value is a Stellar expression. The map can include 0 or more variables/expressions. When each message is applied to the profile, the expression is executed and stored in a variable with the given name.</p>
+<p>One or more expressions executed when a message is applied to the profile.  A map is expected where the key is the variable name and the value is a Stellar expression.  The map can include 0 or more variables/expressions. When each message is applied to the profile, the expression is executed and stored in a variable with the given name.</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;update&quot;: {
+<div>
+<div>
+<pre class="source">&quot;update&quot;: {
   &quot;var1&quot;: &quot;var1 + 1&quot;,
   &quot;var2&quot;: &quot;var2 + 1&quot;
 }
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="result"></a><tt>result</tt></h3>
 <p><i>Required</i></p>
-<p>Stellar expressions that are executed when the window period expires. The expressions are expected to summarize the messages that were applied to the profile over the window period. In the most basic form a single result is persisted for later retrieval.</p>
+<p>Stellar expressions that are executed when the window period expires.  The expressions are expected to summarize the messages that were applied to the profile over the window period.  In the most basic form a single result is persisted for later retrieval.</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;result&quot;: &quot;var1 + var2&quot;
+<div>
+<div>
+<pre class="source">&quot;result&quot;: &quot;var1 + var2&quot;
 </pre></div></div>
-<p>For more advanced use cases, a profile can generate two types of results. A profile can define one or both of these result types at the same time.</p>
 
+<p>For more advanced use cases, a profile can generate two types of results.  A profile can define one or both of these result types at the same time.</p>
 <ul>
-  
-<li><tt>profile</tt>: A required expression that defines a value that is persisted for later retrieval.</li>
-  
+
+<li><tt>profile</tt>:  A required expression that defines a value that is persisted for later retrieval.</li>
 <li><tt>triage</tt>: An optional expression that defines values that are accessible within the Threat Triage process.</li>
 </ul>
 <p><b>profile</b></p>
-<p>A required Stellar expression that results in a value that is persisted in the profile store for later retrieval. The expression can result in any object that is Kryo serializable. These values can be retrieved for later use with the <a href="../metron-profiler-client/index.html">Profiler Client</a>.</p>
+<p>A required Stellar expression that results in a value that is persisted in the profile store for later retrieval.  The expression can result in any object that is Kryo serializable.  These values can be retrieved for later use with the <a href="../metron-profiler-client/index.html">Profiler Client</a>.</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;result&quot;: {
+<div>
+<div>
+<pre class="source">&quot;result&quot;: {
     &quot;profile&quot;: &quot;2 + 2&quot;
 }
 </pre></div></div>
+
 <p>An alternative, simplified form is also acceptable.</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;result&quot;: &quot;2 + 2&quot;
+<div>
+<div>
+<pre class="source">&quot;result&quot;: &quot;2 + 2&quot;
 </pre></div></div>
+
 <p><b>triage</b></p>
-<p>An optional map of one or more Stellar expressions. The value of each expression is made available to the Threat Triage process under the given name. Each expression must result in a either a primitive type, like an integer, long, or short, or a String. All other types will result in an error.</p>
-<p>In the following example, three values, the minimum, the maximum and the mean are appended to a message. This message is consumed by Metron, like other sources of telemetry, and each of these values are accessible from within the Threat Triage process using the given field names; <tt>min</tt>, <tt>max</tt>, and <tt>mean</tt>.</p>
+<p>An optional map of one or more Stellar expressions. The value of each expression is made available to the Threat Triage process under the given name.  Each expression must result in a either a primitive type, like an integer, long, or short, or a String.  All other types will result in an error.</p>
+<p>In the following example, three values, the minimum, the maximum and the mean are appended to a message.  This message is consumed by Metron, like other sources of telemetry, and each of these values are accessible from within the Threat Triage process using the given field names; <tt>min</tt>, <tt>max</tt>, and <tt>mean</tt>.</p>
 
-<div class="source">
-<div class="source">
-<pre>&quot;result&quot;: {
+<div>
+<div>
+<pre class="source">&quot;result&quot;: {
     &quot;triage&quot;: {
         &quot;min&quot;: &quot;STATS_MIN(stats)&quot;,
         &quot;max&quot;: &quot;STATS_MAX(stats)&quot;,
         &quot;mean&quot;: &quot;STATS_MEAN(stats)&quot;
     }
 }
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="expires"></a><tt>expires</tt></h3>
 <p><i>Optional</i></p>
-<p>A numeric value that defines how many days the profile data is retained. After this time, the data expires and is no longer accessible. If no value is defined, the data does not expire.</p>
+<p>A numeric value that defines how many days the profile data is retained.  After this time, the data expires and is no longer accessible.  If no value is defined, the data does not expire.</p>
 <p>The REPL can be a powerful for developing profiles. Read all about <a href="../metron-profiler-client/index.html#developing_profiles">Developing Profiles</a>.</p></div></div>
 <div class="section">
 <h2><a name="Configuring_the_Profiler"></a>Configuring the Profiler</h2>
-<p>The Profiler runs as an independent Storm topology. The configuration for the Profiler topology is stored in local filesystem at <tt>$METRON_HOME/config/profiler.properties</tt>. The values can be changed on disk and then the Profiler topology must be restarted.</p>
-
+<p>The Profiler runs as an independent Storm topology.  The configuration for the Profiler topology is stored in local filesystem at <tt>$METRON_HOME/config/profiler.properties</tt>. After changing these values, the Profiler topology must be restarted for the changes to take effect.</p>
 <table border="0" class="table table-striped">
-  <thead>
-    
+<thead>
+
+<tr class="a">
+<th> Setting                                                                       </th>
+<th> Description</th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td> <a href="#profiler.input.topic"><tt>profiler.input.topic</tt></a>                                 </td>
+<td> The name of the input Kafka topic.</td></tr>
+<tr class="a">
+<td> <a href="#profiler.output.topic"><tt>profiler.output.topic</tt></a>                               </td>
+<td> The name of the output Kafka topic.</td></tr>
+<tr class="b">
+<td> <a href="#profiler.period.duration"><tt>profiler.period.duration</tt></a>                         </td>
+<td> The duration of each profile period.</td></tr>
 <tr class="a">
-      
-<th>Setting </th>
-      
-<th>Description</th>
-    </tr>
-  </thead>
-  <tbody>
-    
+<td> <a href="#profiler.period.duration.units"><tt>profiler.period.duration.units</tt></a>              </td>
+<td> The units used to specify the <a href="#profiler.period.duration"><tt>profiler.period.duration</tt></a>.</td></tr>
 <tr class="b">
-      
-<td><a href="#profiler.input.topic"><tt>profiler.input.topic</tt></a> </td>
-      
-<td>The name of the Kafka topic from which to consume data.</td>
-    </tr>
-    
+<td> <a href="#profiler.window.duration"><tt>profiler.window.duration</tt></a>                         </td>
+<td> The duration of each profile window.</td></tr>
 <tr class="a">
-      
-<td><a href="#profiler.output.topic"><tt>profiler.output.topic</tt></a> </td>
-      
-<td>The name of the Kafka topic to which profile data is written. Only used with profiles that define the <a href="#result"><tt>triage</tt> result field</a>.</td>
-    </tr>
-    
+<td> <a href="#profilerpwindowdurationunits"><tt>profiler.window.duration.units</tt></a>             </td>
+<td> The units used to specify the <a href="#profiler.window.duration"><tt>profiler.window.duration</tt></a>.</td></tr>
 <tr class="b">
-      
-<td><a href="#profiler.period.duration"><tt>profiler.period.duration</tt></a> </td>
-      
-<td>The duration of each profile period.</td>
-    </tr>
-    
+<td> <a href="#profiler.window.lag"><tt>profiler.window.lag</tt></a>                                   </td>
+<td> The maximum time lag for timestamps.</td></tr>
 <tr class="a">
-      
-<td><a href="#profiler.period.duration.units"><tt>profiler.period.duration.units</tt></a> </td>
-      
-<td>The units used to specify the <a href="#profiler.period.duration"><tt>profiler.period.duration</tt></a>.</td>
-    </tr>
-    
+<td> <a href="#profilerpwindowlagunits"><tt>profiler.window.lag.units</tt></a>                       </td>
+<td> The units used to specify the <a href="#profiler.window.lag"><tt>profiler.window.lag</tt></a>.</td></tr>
 <tr class="b">
-      
-<td><a href="#profiler.workers"><tt>profiler.workers</tt></a> </td>
-      
-<td>The number of worker processes for the topology.</td>
-    </tr>
-    
+<td> <a href="#profiler.workers"><tt>profiler.workers</tt></a>                                        </td>
+<td> The number of worker processes for the topology.</td></tr>
 <tr class="a">
-      
-<td><a href="#profiler.executors"><tt>profiler.executors</tt></a> </td>
-      
-<td>The number of executors to spawn per component.</td>
-    </tr>
-    
+<td> <a href="#profiler.executors"><tt>profiler.executors</tt></a>                                    </td>
+<td> The number of executors to spawn per component.</td></tr>
 <tr class="b">
-      
-<td><a href="#profiler.ttl"><tt>profiler.ttl</tt></a> </td>
-      
-<td>If a message has not been applied to a Profile in this period of time, the Profile will be forgotten and its resources will be cleaned up.</td>
-    </tr>
-    
+<td> <a href="#profiler.ttl"><tt>profiler.ttl</tt></a>                                                </td>
+<td> If a message has not been applied to a Profile in this period of time, the Profile will be forgotten and its resources will be cleaned up.</td></tr>
 <tr class="a">
-      
-<td><a href="#profiler.ttl.units"><tt>profiler.ttl.units</tt></a> </td>
-      
-<td>The units used to specify the <tt>profiler.ttl</tt>.</td>
-    </tr>
-    
+<td> <a href="#profiler.ttl.units"><tt>profiler.ttl.units</tt></a>                                     </td>
+<td> The units used to specify the <tt>profiler.ttl</tt>.</td></tr>
 <tr class="b">
-      
-<td><a href="#profiler.hbase.salt.divisor"><tt>profiler.hbase.salt.divisor</tt></a> </td>
-      
-<td>A salt is prepended to the row key to help prevent hotspotting.</td>
-    </tr>
-    
+<td> <a href="#profiler.hbase.salt.divisor"><tt>profiler.hbase.salt.divisor</tt></a>                    </td>
+<td> A salt is prepended to the row key to help prevent hot-spotting.</td></tr>
 <tr class="a">
-      
-<td><a href="#profiler.hbase.table"><tt>profiler.hbase.table</tt></a> </td>
-      
-<td>The name of the HBase table that profiles are written to.</td>
-    </tr>
-    
+<td> <a href="#profiler.hbase.table"><tt>profiler.hbase.table</tt></a>                                 </td>
+<td> The name of the HBase table that profiles are written to.</td></tr>
 <tr class="b">
-      
-<td><a href="#profiler.hbase.column.family"><tt>profiler.hbase.column.family</tt></a> </td>
-      
-<td>The column family used to store profiles.</td>
-    </tr>
-    
+<td> <a href="#profiler.hbase.column.family"><tt>profiler.hbase.column.family</tt></a>                  </td>
+<td> The column family used to store profiles.</td></tr>
 <tr class="a">
-      
-<td><a href="#profiler.hbase.batch"><tt>profiler.hbase.batch</tt></a> </td>
-      
-<td>The number of puts that are written to HBase in a single batch.</td>
-    </tr>
-    
+<td> <a href="#profiler.hbase.batch"><tt>profiler.hbase.batch</tt></a>                                 </td>
+<td> The number of puts that are written to HBase in a single batch.</td></tr>
 <tr class="b">
-      
-<td><a href="#profiler.hbase.flush.interval.seconds"><tt>profiler.hbase.flush.interval.seconds</tt></a> </td>
-      
-<td>The maximum number of seconds between batch writes to HBase.</td>
-    </tr>
-  </tbody>
+<td> <a href="#profiler.hbase.flush.interval.seconds"><tt>profiler.hbase.flush.interval.seconds</tt></a> </td>
+<td> The maximum number of seconds between batch writes to HBase.</td></tr>
+<tr class="a">
+<td> <a href="#topology.kryo.register"><tt>topology.kryo.register</tt></a>                             </td>
+<td> Storm will use Kryo serialization for these classes.</td></tr>
+</tbody>
 </table>
 <div class="section">
 <h3><a name="profiler.input.topic"></a><tt>profiler.input.topic</tt></h3>
 <p><i>Default</i>: indexing</p>
-<p>The name of the Kafka topic from which to consume data. By default, the Profiler consumes data from the <tt>indexing</tt> topic so that it has access to fully enriched telemetry.</p></div>
+<p>The name of the Kafka topic from which to consume data.  By default, the Profiler consumes data from the <tt>indexing</tt> topic so that it has access to fully enriched telemetry.</p></div>
 <div class="section">
 <h3><a name="profiler.output.topic"></a><tt>profiler.output.topic</tt></h3>
 <p><i>Default</i>: enrichments</p>
-<p>The name of the Kafka topic to which profile data is written. This property is only applicable to profiles that define the <a href="#result"><tt>result</tt> <tt>triage</tt> field</a>. This allows Profile data to be selectively triaged like any other source of telemetry in Metron.</p></div>
+<p>The name of the Kafka topic to which profile data is written.  This property is only applicable to profiles that define  the <a href="#result"><tt>result</tt> <tt>triage</tt> field</a>.  This allows Profile data to be selectively triaged like any other source of telemetry in Metron.</p></div>
 <div class="section">
 <h3><a name="profiler.period.duration"></a><tt>profiler.period.duration</tt></h3>
 <p><i>Default</i>: 15</p>
-<p>The duration of each profile period. This value should be defined along with <a href="#profiler.period.duration.units"><tt>profiler.period.duration.units</tt></a>.</p>
-<p><i>Important</i>: To read a profile using the <a href="metron-analytics/metron-profiler-client/index.html">Profiler Client</a>, the Profiler Client&#x2019;s <tt>profiler.client.period.duration</tt> property must match this value. Otherwise, the Profiler Client will be unable to read the profile data. </p></div>
+<p>The duration of each profile period.  This value should be defined along with <a href="#profiler.period.duration.units"><tt>profiler.period.duration.units</tt></a>.</p>
+<p><i>Important</i>: To read a profile using the <a href="metron-analytics/metron-profiler-client/index.html">Profiler Client</a>, the Profiler Client&#x2019;s <tt>profiler.client.period.duration</tt> property must match this value.  Otherwise, the Profiler Client will be unable to read the profile data.</p></div>
 <div class="section">
 <h3><a name="profiler.period.duration.units"></a><tt>profiler.period.duration.units</tt></h3>
 <p><i>Default</i>: MINUTES</p>
-<p>The units used to specify the <tt>profiler.period.duration</tt>. This value should be defined along with <a href="#profiler.period.duration"><tt>profiler.period.duration</tt></a>.</p>
-<p><i>Important</i>: To read a profile using the Profiler Client, the Profiler Client&#x2019;s <tt>profiler.client.period.duration.units</tt> property must match this value. Otherwise, the <a href="metron-analytics/metron-profiler-client/index.html">Profiler Client</a> will be unable to read the profile data.</p></div>
+<p>The units used to specify the <tt>profiler.period.duration</tt>.  This value should be defined along with <a href="#profiler.period.duration"><tt>profiler.period.duration</tt></a>.</p>
+<p><i>Important</i>: To read a profile using the Profiler Client, the Profiler Client&#x2019;s <tt>profiler.client.period.duration.units</tt> property must match this value.  Otherwise, the <a href="metron-analytics/metron-profiler-client/index.html">Profiler Client</a> will be unable to read the profile data.</p></div>
+<div class="section">
+<h3><a name="profiler.window.duration"></a><tt>profiler.window.duration</tt></h3>
+<p><i>Default</i>: 30</p>
+<p>The duration of each profile window.  Telemetry that arrives within a slice of time is processed within a single window.</p>
+<p>Many windows of telemetry will be processed during a single profile period.  This does not change the output of the Profiler, it only changes how the Profiler processes data. The window defines how much data the Profiler processes in a single pass.</p>
+<p>This value should be defined along with <a href="#profiler.window.duration.units"><tt>profiler.window.duration.units</tt></a>.</p>
+<p>This value must be less than the period duration as defined by <a href="#profiler.period.duration"><tt>profiler.period.duration</tt></a> and <a href="#profiler.period.duration.units"><tt>profiler.period.duration.units</tt></a>.</p></div>
+<div class="section">
+<h3><a name="profiler.window.duration.units"></a><tt>profiler.window.duration.units</tt></h3>
+<p><i>Default</i>: SECONDS</p>
+<p>The units used to specify the <tt>profiler.window.duration</tt>.  This value should be defined along with <a href="#profiler.window.duration"><tt>profiler.window.duration</tt></a>.</p></div>
+<div class="section">
+<h3><a name="profiler.window.lag"></a><tt>profiler.window.lag</tt></h3>
+<p><i>Default</i>: 1</p>
+<p>The maximum time lag for timestamps. Timestamps cannot arrive out-of-order by more than this amount. This value should be defined along with <a href="#profiler.window.lag.units"><tt>profiler.window.lag.units</tt></a>.</p></div>
+<div class="section">
+<h3><a name="profiler.window.lag.units"></a><tt>profiler.window.lag.units</tt></h3>
+<p><i>Default</i>: SECONDS</p>
+<p>The units used to specify the <tt>profiler.window.lag</tt>.  This value should be defined along with <a href="#profiler.window.lag"><tt>profiler.window.lag</tt></a>.</p></div>
 <div class="section">
 <h3><a name="profiler.workers"></a><tt>profiler.workers</tt></h3>
 <p><i>Default</i>: 1</p>
-<p>The number of worker processes to create for the Profiler topology. This property is useful for performance tuning the Profiler.</p></div>
+<p>The number of worker processes to create for the Profiler topology.  This property is useful for performance tuning the Profiler.</p></div>
 <div class="section">
 <h3><a name="profiler.executors"></a><tt>profiler.executors</tt></h3>
 <p><i>Default</i>: 0</p>
-<p>The number of executors to spawn per component for the Profiler topology. This property is useful for performance tuning the Profiler.</p></div>
+<p>The number of executors to spawn per component for the Profiler topology.  This property is useful for performance tuning the Profiler.</p></div>
 <div class="section">
 <h3><a name="profiler.ttl"></a><tt>profiler.ttl</tt></h3>
 <p><i>Default</i>: 30</p>
 <p>If a message has not been applied to a Profile in this period of time, the Profile will be terminated and its resources will be cleaned up. This value should be defined along with <a href="#profiler.ttl.units"><tt>profiler.ttl.units</tt></a>.</p>
-<p>This time-to-live does not affect the persisted Profile data in HBase. It only affects the state stored in memory during the execution of the latest profile period. This state will be deleted if the time-to-live is exceeded.</p></div>
+<p>This time-to-live does not affect the persisted Profile data in HBase.  It only affects the state stored in memory during the execution of the latest profile period.  This state will be deleted if the time-to-live is exceeded.</p></div>
 <div class="section">
 <h3><a name="profiler.ttl.units"></a><tt>profiler.ttl.units</tt></h3>
 <p><i>Default</i>: MINUTES</p>
@@ -955,11 +848,11 @@ hbase(main):001:0&gt; count 'profiler'
 <div class="section">
 <h3><a name="profiler.hbase.salt.divisor"></a><tt>profiler.hbase.salt.divisor</tt></h3>
 <p><i>Default</i>: 1000</p>
-<p>A salt is prepended to the row key to help prevent hotspotting. This constant is used to generate the salt. This constant should be roughly equal to the number of nodes in the Hbase cluster to ensure even distribution of data.</p></div>
+<p>A salt is prepended to the row key to help prevent hotspotting.  This constant is used to generate the salt.  This constant should be roughly equal to the number of nodes in the Hbase cluster to ensure even distribution of data.</p></div>
 <div class="section">
 <h3><a name="profiler.hbase.table"></a><tt>profiler.hbase.table</tt></h3>
 <p><i>Default</i>: profiler</p>
-<p>The name of the HBase table that profile data is written to. The Profiler expects that the table exists and is writable. It will not create the table.</p></div>
+<p>The name of the HBase table that profile data is written to.  The Profiler expects that the table exists and is writable.  It will not create the table.</p></div>
 <div class="section">
 <h3><a name="profiler.hbase.column.family"></a><tt>profiler.hbase.column.family</tt></h3>
 <p><i>Default</i>: P</p>
@@ -971,15 +864,37 @@ hbase(main):001:0&gt; count 'profiler'
 <div class="section">
 <h3><a name="profiler.hbase.flush.interval.seconds"></a><tt>profiler.hbase.flush.interval.seconds</tt></h3>
 <p><i>Default</i>: 30</p>
-<p>The maximum number of seconds between batch writes to HBase.</p></div></div>
+<p>The maximum number of seconds between batch writes to HBase.</p></div>
+<div class="section">
+<h3><a name="topology.kryo.register"></a><tt>topology.kryo.register</tt></h3>
+<p><i>Default</i>:</p>
+
+<div>
+<div>
+<pre class="source">[ org.apache.metron.profiler.ProfileMeasurement, \
+  org.apache.metron.profiler.ProfilePeriod, \
+  org.apache.metron.common.configuration.profiler.ProfileResult, \
+  org.apache.metron.common.configuration.profiler.ProfileResultExpressions, \
+  org.apache.metron.common.configuration.profiler.ProfileTriageExpressions, \
+  org.apache.metron.common.configuration.profiler.ProfilerConfig, \
+  org.apache.metron.common.configuration.profiler.ProfileConfig, \
+  org.json.simple.JSONObject, \
+  java.util.LinkedHashMap, \
+  org.apache.metron.statistics.OnlineStatisticsProvider ]
+</pre></div></div>
+
+<p>Storm will use Kryo serialization for these classes. Kryo serialization is more performant than Java serialization, in most cases.</p>
+<p>For these classes, Storm will uses Kryo&#x2019;s <tt>FieldSerializer</tt> as defined in the <a class="externalLink" href="http://storm.apache.org/releases/1.1.2/Serialization.html">Storm Serialization docs</a>.  For all other classes not in this list, Storm defaults to using Java serialization which is slower and not recommended for a production topology.</p>
+<p>This value should only need altered if you have defined a profile that results in a non-primitive, user-defined type that is not in this list.  If the class is not defined in this list, Java serialization will be used and the class must adhere to Java&#x2019;s serialization requirements.</p>
+<p>The performance of the entire Profiler topology can be negatively impacted if any profile produces results that undergo Java serialization.</p></div></div>
 <div class="section">
 <h2><a name="Examples"></a>Examples</h2>
 <p>The following examples are intended to highlight the functionality provided by the Profiler. Try out these examples easily in the Stellar Shell as described in the <a href="#Creating_Profiles">Creating Profiles</a> section.</p>
-<p>These examples assume a fictitious input message stream that looks like the following. </p>
+<p>These examples assume a fictitious input message stream that looks like the following.</p>
 
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; msgs := SHELL_EDIT()
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; msgs := SHELL_EDIT()
 [Stellar]&gt;&gt;&gt; msgs
 [
   {
@@ -1002,13 +917,14 @@ hbase(main):001:0&gt; count 'profiler'
   }
 ]
 </pre></div></div>
+
 <div class="section">
 <h3><a name="Example_1"></a>Example 1</h3>
 <p>The total number of bytes of HTTP data for each host. The following configuration would be used to generate this profile.</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;profiles&quot;: [
     {
       &quot;profile&quot;: &quot;example1&quot;,
@@ -1026,31 +942,25 @@ hbase(main):001:0&gt; count 'profiler'
   ]
 }
 </pre></div></div>
-<p>This creates a profile&#x2026;</p>
 
+<p>This creates a profile&#x2026;</p>
 <ul>
-  
+
 <li>Named &#x2018;example1&#x2019;</li>
-  
 <li>That for each IP source address</li>
-  
 <li>Only if the &#x2018;protocol&#x2019; field equals &#x2018;HTTP&#x2019;</li>
-  
 <li>Initializes a counter &#x2018;total_bytes&#x2019; to zero</li>
-  
 <li>Adds to &#x2018;total_bytes&#x2019; the value of the message&#x2019;s &#x2018;bytes_in&#x2019; field</li>
-  
 <li>Returns &#x2018;total_bytes&#x2019; as the result</li>
-  
 <li>The profile data will expire in 30 days</li>
 </ul></div>
 <div class="section">
 <h3><a name="Example_2"></a>Example 2</h3>
 <p>The ratio of DNS traffic to HTTP traffic for each host. The following configuration would be used to generate this profile.</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;profiles&quot;: [
     {
       &quot;profile&quot;: &quot;example2&quot;,
@@ -1069,29 +979,24 @@ hbase(main):001:0&gt; count 'profiler'
   ]
 }
 </pre></div></div>
-<p>This creates a profile&#x2026;</p>
 
+<p>This creates a profile&#x2026;</p>
 <ul>
-  
+
 <li>Named &#x2018;example2&#x2019;</li>
-  
 <li>That for each IP source address</li>
-  
 <li>Only if the &#x2018;protocol&#x2019; field equals &#x2018;HTTP&#x2019; or &#x2018;DNS&#x2019;</li>
-  
 <li>Accumulates the number of DNS requests</li>
-  
 <li>Accumulates the number of HTTP requests</li>
-  
 <li>Returns the ratio of these as the result</li>
 </ul></div>
 <div class="section">
 <h3><a name="Example_3"></a>Example 3</h3>
 <p>The average of the <tt>length</tt> field of HTTP traffic. The following configuration would be used to generate this profile.</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;profiles&quot;: [
     {
       &quot;profile&quot;: &quot;example3&quot;,
@@ -1103,28 +1008,24 @@ hbase(main):001:0&gt; count 'profiler'
   ]
 }
 </pre></div></div>
-<p>This creates a profile&#x2026;</p>
 
+<p>This creates a profile&#x2026;</p>
 <ul>
-  
+
 <li>Named &#x2018;example3&#x2019;</li>
-  
 <li>That for each IP source address</li>
-  
 <li>Only if the &#x2018;protocol&#x2019; field is &#x2018;HTTP&#x2019;</li>
-  
 <li>Adds the <tt>length</tt> field from each message</li>
-  
 <li>Calculates the average as the result</li>
 </ul></div>
 <div class="section">
 <h3><a name="Example_4"></a>Example 4</h3>
-<p>It is important to note that the Profiler can persist any serializable Object, not just numeric values. An alternative to the previous example could take advantage of this. </p>
-<p>Instead of storing the mean of the lengths, the profile could store a statistical summarization of the lengths. This summary can then be used at a later time to calculate the mean, min, max, percentiles, or any other sensible metric. This provides a much greater degree of flexibility.</p>
+<p>It is important to note that the Profiler can persist any serializable Object, not just numeric values.  An alternative to the previous example could take advantage of this.</p>
+<p>Instead of storing the mean of the lengths, the profile could store a statistical summarization of the lengths.  This summary can then be used at a later time to calculate the mean, min, max, percentiles, or any other sensible metric.  This provides a much greater degree of flexibility.</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;profiles&quot;: [
     {
       &quot;profile&quot;: &quot;example4&quot;,
@@ -1136,94 +1037,101 @@ hbase(main):001:0&gt; count 'profiler'
   ]
 }
 </pre></div></div>
+
 <p>The following Stellar REPL session shows how you might use this summary to calculate different metrics with the same underlying profile data. It is assumed that the PROFILE_GET client is configured as described <a href="../metron-profiler-client/index.html">here</a>.</p>
 <p>Retrieve the last 30 minutes of profile measurements for a specific host.</p>
 
-<div class="source">
-<div class="source">
-<pre>$ bin/stellar -z node1:2181
+<div>
+<div>
+<pre class="source">$ bin/stellar -z node1:2181
 
 [Stellar]&gt;&gt;&gt; stats := PROFILE_GET( &quot;example4&quot;, &quot;10.0.0.1&quot;, PROFILE_FIXED(30, &quot;MINUTES&quot;))
 [Stellar]&gt;&gt;&gt; stats
 [org.apache.metron.common.math.stats.OnlineStatisticsProvider@79fe4ab9, ...]
 </pre></div></div>
+
 <p>Calculate different metrics with the same profile data.</p>
 
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; STATS_MEAN( GET_FIRST( stats))
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; STATS_MEAN( GET_FIRST( stats))
 15979.0625
 
 [Stellar]&gt;&gt;&gt; STATS_PERCENTILE( GET_FIRST(stats), 90)
 30310.958
 </pre></div></div>
+
 <p>Merge all of the profile measurements over the past 30 minutes into a single summary and calculate the 90th percentile.</p>
 
-<div class="source">
-<div class="source">
-<pre>[Stellar]&gt;&gt;&gt; merged := STATS_MERGE( stats)
+<div>
+<div>
+<pre class="source">[Stellar]&gt;&gt;&gt; merged := STATS_MERGE( stats)
 [Stellar]&gt;&gt;&gt; STATS_PERCENTILE(merged, 90)
 29810.992
 </pre></div></div>
+
 <p>More information on accessing profile data can be found in the <a href="../metron-profiler-client/index.html">Profiler Client</a>.</p>
 <p>More information on using the <a href="../../metron-platform/metron-common/index.html"><tt>STATS_*</tt> functions in Stellar can be found here</a>.</p></div></div>
 <div class="section">
 <h2><a name="Implementation"></a>Implementation</h2></div>
 <div class="section">
 <h2><a name="Key_Classes"></a>Key Classes</h2>
-
 <ul>
-  
+
 <li>
-<p><tt>ProfileMeasurement</tt> - Represents a single data point within a Profile. A Profile is effectively a time series. To this end a Profile is composed of many ProfileMeasurement values which in aggregate form a time series.</p></li>
-  
+
+<p><tt>ProfileMeasurement</tt> - Represents a single data point within a Profile.  A Profile is effectively a time series.  To this end a Profile is composed of many ProfileMeasurement values which in aggregate form a time series.</p>
+</li>
 <li>
-<p><tt>ProfilePeriod</tt> - The Profiler captures one <tt>ProfileMeasurement</tt> each <tt>ProfilePeriod</tt>. A <tt>ProfilePeriod</tt> will occur at fixed, deterministic points in time. This allows for efficient retrieval of profile data.</p></li>
-  
+
+<p><tt>ProfilePeriod</tt> - The Profiler captures one <tt>ProfileMeasurement</tt> each <tt>ProfilePeriod</tt>.  A <tt>ProfilePeriod</tt> will occur at fixed, deterministic points in time.  This allows for efficient retrieval of profile data.</p>
+</li>
 <li>
-<p><tt>RowKeyBuilder</tt> - Builds row keys that can be used to read or write profile data to HBase.</p></li>
-  
+
+<p><tt>RowKeyBuilder</tt> - Builds row keys that can be used to read or write profile data to HBase.</p>
+</li>
 <li>
-<p><tt>ColumnBuilder</tt> - Defines the columns of data stored with a profile measurement.</p></li>
-  
+
+<p><tt>ColumnBuilder</tt> - Defines the columns of data stored with a profile measurement.</p>
+</li>
 <li>
-<p><tt>ProfileHBaseMapper</tt> - Defines for the <tt>HBaseBolt</tt> how profile measurements are stored in HBase. This class leverages a <tt>RowKeyBuilder</tt> and <tt>ColumnBuilder</tt>.</p></li>
+
+<p><tt>ProfileHBaseMapper</tt> - Defines for the <tt>HBaseBolt</tt> how profile measurements are stored in HBase.  This class leverages a <tt>RowKeyBuilder</tt> and <tt>ColumnBuilder</tt>.</p>
+</li>
 </ul></div>
 <div class="section">
 <h2><a name="Storm_Topology"></a>Storm Topology</h2>
 <p>The Profiler is implemented as a Storm topology using the following bolts and spouts.</p>
-
 <ul>
-  
+
 <li>
-<p><tt>KafkaSpout</tt> - A spout that consumes messages from a single Kafka topic. In most cases, the Profiler topology will consume messages from the <tt>indexing</tt> topic. This topic contains fully enriched messages that are ready to be indexed. This ensures that profiles can take advantage of all the available data elements.</p></li>
-  
+
+<p><tt>KafkaSpout</tt> - A spout that consumes messages from a single Kafka topic.  In most cases, the Profiler topology will consume messages from the <tt>indexing</tt> topic.  This topic contains fully enriched messages that are ready to be indexed.  This ensures that profiles can take advantage of all the available data elements.</p>
+</li>
 <li>
-<p><tt>ProfileSplitterBolt</tt> - The bolt responsible for filtering incoming messages and directing each to the one or more downstream bolts that are responsible for building a profile. Each message may be needed by 0, 1 or even many profiles. Each emitted tuple contain

<TRUNCATED>

[25/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/roles/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/index.html b/site/current-book/metron-deployment/roles/index.html
deleted file mode 100644
index 53372ea..0000000
--- a/site/current-book/metron-deployment/roles/index.html
+++ /dev/null
@@ -1,332 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Ansible Roles</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../css/site.css" />
-    <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Ansible Roles</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                    
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Roles</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/roles/monit/index.html" title="Monit">
-          <i class="none"></i>
-        Monit</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
-          <i class="none"></i>
-        Opentaxii</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
-          <i class="none"></i>
-        Pcap_replay</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
-          <i class="none"></i>
-        Sensor-stubs</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
-          <i class="none"></i>
-        Sensor-test-mode</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Ansible Roles</h1>
-<p><a name="Ansible_Roles"></a></p>
-
-<ul>
-  
-<li>Monit</li>
-  
-<li>OpenTaxii</li>
-  
-<li>Pcap Replay</li>
-  
-<li>Sensor Stubs</li>
-  
-<li>Sensor Test Mode</li>
-</ul>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/roles/monit/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/monit/index.html b/site/current-book/metron-deployment/roles/monit/index.html
deleted file mode 100644
index 09684dc..0000000
--- a/site/current-book/metron-deployment/roles/monit/index.html
+++ /dev/null
@@ -1,388 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Monit Integration</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../../css/site.css" />
-    <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Monit Integration</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-down"></i>
-        Roles</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Monit</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
-          <i class="none"></i>
-        Opentaxii</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
-          <i class="none"></i>
-        Pcap_replay</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
-          <i class="none"></i>
-        Sensor-stubs</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
-          <i class="none"></i>
-        Sensor-test-mode</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Monit Integration</h1>
-<p><a name="Monit_Integration"></a></p>
-<p>This role will leverage Monit as a process watchdog to manage sensors, topologies, and core services. </p>
-
-<ul>
-  
-<li>Monit can be used to start, stop, or check status of any of the sensors or topologies.</li>
-  
-<li>When monitoring is enabled (on by default) if a process dies, it will be restarted.</li>
-  
-<li>The Monit web interface is exposed at <a class="externalLink" href="http://hostname:2812">http://hostname:2812</a>.</li>
-  
-<li>The web interface username and password is defined by the <tt>monit_user</tt> and <tt>monit_password</tt> variables. These default to <tt>admin</tt>/<tt>monit</tt>.</li>
-  
-<li>Monit CLI tools can also be used to simplify the process of managing Metron components.</li>
-  
-<li>
-<p>The post-deployment report for Amazon-EC2 provides links to Monit&#x2019;s web interface labeled as &#x2018;Sensor Status&#x2019; and &#x2018;Topology Status.&#x2019;</p>
-  
-<div class="source">
-<div class="source">
-<pre>  ok: [localhost] =&gt; {
-&quot;Success&quot;: [
-    &quot;Apache Metron deployed successfully&quot;,
-    &quot;   Metron          @ http://ec2-52-39-143-62.us-west-2.compute.amazonaws.com:5000&quot;,
-    &quot;   Ambari          @ http://ec2-52-39-4-93.us-west-2.compute.amazonaws.com:8080&quot;,
-    &quot;   Sensor Status   @ http://ec2-52-39-4-93.us-west-2.compute.amazonaws.com:2812&quot;,
-    &quot;   Topology Status @ http://ec2-52-39-130-62.us-west-2.compute.amazonaws.com:2812&quot;,
-    &quot;For additional information, see https://metron.apache.org/'&quot;
-]
-  }
-</pre></div></div></li>
-</ul>
-<div class="section">
-<h2><a name="Usage"></a>Usage</h2>
-<p>Start all Metron components</p>
-
-<div class="source">
-<div class="source">
-<pre>monit start all
-</pre></div></div>
-<p>Stop all Metron components</p>
-
-<div class="source">
-<div class="source">
-<pre>monit stop all
-</pre></div></div>
-<p>Start an individual Metron component</p>
-
-<div class="source">
-<div class="source">
-<pre>monit start bro-parser
-</pre></div></div>
-<p>Start all components required to ingest Bro data</p>
-
-<div class="source">
-<div class="source">
-<pre>monit -g bro start
-</pre></div></div>
-<p>Start all parsers</p>
-
-<div class="source">
-<div class="source">
-<pre>monit -g parsers start
-</pre></div></div>
-<p>What is running?</p>
-
-<div class="source">
-<div class="source">
-<pre>monit summary
-</pre></div></div></div>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/roles/opentaxii/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/opentaxii/index.html b/site/current-book/metron-deployment/roles/opentaxii/index.html
deleted file mode 100644
index 29d8700..0000000
--- a/site/current-book/metron-deployment/roles/opentaxii/index.html
+++ /dev/null
@@ -1,480 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; OpenTAXII</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../../css/site.css" />
-    <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">OpenTAXII</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-down"></i>
-        Roles</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/monit/index.html" title="Monit">
-          <i class="none"></i>
-        Monit</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Opentaxii</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
-          <i class="none"></i>
-        Pcap_replay</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
-          <i class="none"></i>
-        Sensor-stubs</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
-          <i class="none"></i>
-        Sensor-test-mode</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>OpenTAXII</h1>
-<p><a name="OpenTAXII"></a></p>
-<p>Installs <a class="externalLink" href="https://github.com/EclecticIQ/OpenTAXII">OpenTAXII</a> as a deamon that can be launched via a SysV service script. The complementary client implementation, <a class="externalLink" href="https://github.com/EclecticIQ/cabby">Cabby</a> is also installed.</p>
-<p>OpenTAXII is a robust Python implementation of TAXII Services that delivers a rich feature set and friendly pythonic API. <a class="externalLink" href="https://stixproject.github.io/">TAXII</a> (Trusted Automated eXchange of Indicator Information) is a collection of specifications defining a set of services and message exchanges used for sharing cyber threat intelligence information between parties.</p>
-<div class="section">
-<h2><a name="Getting_Started"></a>Getting Started</h2>
-<p>After deployment completes the OpenTAXII service is installed and running. A set of <a class="externalLink" href="http://hailataxii.com/">Hail a TAXII</a> threat intel collections have been defined and configured. Use the <tt>status</tt> option to view the collections that have been defined.</p>
-
-<div class="source">
-<div class="source">
-<pre>$ service opentaxii status
-Checking opentaxii...                             Running
-guest.phishtank_com                                0
-guest.Abuse_ch                                     0
-guest.CyberCrime_Tracker                           0
-guest.EmergingThreats_rules                        0
-guest.Lehigh_edu                                   0
-guest.MalwareDomainList_Hostlist                   0
-guest.blutmagie_de_torExits                        0
-guest.dataForLast_7daysOnly                        0
-guest.dshield_BlockList                            0
-</pre></div></div>
-<p>Notice that each collections contain zero records. None of the data is automatically synced during deployment. To sync the data manually use the <tt>sync</tt> option as defined below. The following example does not provide a begin and end time so the data will be fetched for the current day only.</p>
-
-<div class="source">
-<div class="source">
-<pre># service opentaxii sync guest.blutmagie_de_torExits
-2016-04-21 20:34:42,511 INFO: Starting new HTTP connection (1): localhost
-2016-04-21 20:34:42,540 INFO: Response received for Inbox_Message from http://localhost:9000/services/inbox
-2016-04-21 20:34:42,542 INFO: Sending Inbox_Message to http://localhost:9000/services/inbox
-...
-2016-04-21 20:34:42,719 INFO: Response received for Poll_Request from http://localhost:9000/services/poll
-2016-04-21 20:34:42,719 INFO: Content blocks count: 1618, is partial: False
-</pre></div></div>
-<p>The OpenTAXII service now contains 1,618 threat intel records indicating Tor Exit nodes.</p>
-
-<div class="source">
-<div class="source">
-<pre>[root@source ~]# service opentaxii status
-Checking opentaxii...                             Running
-guest.phishtank_com                                0
-guest.Abuse_ch                                     0
-guest.CyberCrime_Tracker                           0
-guest.EmergingThreats_rules                        0
-guest.Lehigh_edu                                   0
-guest.MalwareDomainList_Hostlist                   0
-guest.blutmagie_de_torExits                        1618
-guest.dataForLast_7daysOnly                        0
-guest.dshield_BlockList                            0
-</pre></div></div></div>
-<div class="section">
-<h2><a name="Usage"></a>Usage</h2>
-<p>A standard SysV script has been installed to manage OpenTAXII. The following functions are available.</p>
-<p><tt>start</tt> <tt>stop</tt> <tt>restart</tt> the OpenTAXII service</p>
-<p><tt>status</tt> of the OpenTAXII service. The command displays the collections that have been defined and the number of records in each.</p>
-
-<div class="source">
-<div class="source">
-<pre>$ service opentaxii status
-Checking opentaxii...                             Running
-guest.phishtank_com                                984
-guest.Abuse_ch                                     45
-guest.CyberCrime_Tracker                           482
-guest.EmergingThreats_rules                        0
-guest.Lehigh_edu                                   1030
-guest.MalwareDomainList_Hostlist                   84
-guest.blutmagie_de_torExits                        3236
-guest.dataForLast_7daysOnly                        3377
-guest.dshield_BlockList                            0
-</pre></div></div>
-<p><tt>setup</tt> Initializes the services and collections required to operate the OpenTAXII service. This will destroy all existing data. The user is prompted to continue before any data is destroyed.</p>
-
-<div class="source">
-<div class="source">
-<pre># service opentaxii setup
-WARNING: force reset and destroy all opentaxii data? [Ny]: y
-Stopping opentaxii                                ..Ok
-2016-04-21T19:56:01.886157Z [opentaxii.server] info: api.persistence.loaded {timestamp=2016-04-21T19:56:01.886157Z, logger=opentaxii.server, api_class=SQLDatabaseAPI, event=api.persistence.loaded, level=info}
-2016-04-21T19:56:01.896503Z [opentaxii.server] info: api.auth.loaded {timestamp=2016-04-21T19:56:01.896503Z, logger=opentaxii.server, api_class=SQLDatabaseAPI, event=api.auth.loaded, level=info}
-2016-04-21T19:56:01.896655Z [opentaxii.server] info: taxiiserver.configured {timestamp=2016-04-21T19:56:01.896655Z, logger=opentaxii.server, event=taxiiserver.configured, level=info}
-...
-Ok
-</pre></div></div>
-<p><tt>sync [collection] [begin-at] [end-at]</tt> Syncs the threat intel data available at <a class="externalLink" href="http://hailataxii.com/">Hail a TAXII</a>. If no begin and end date is provided then data is synced over the current day only.</p>
-
-<ul>
-  
-<li><tt>collection</tt> Name of the collection to sync.</li>
-  
-<li><tt>begin-at</tt> Exclusive begin of time window; ISO8601</li>
-  
-<li><tt>end-at</tt> Inclusive end of time window; ISO8601</li>
-</ul>
-
-<div class="source">
-<div class="source">
-<pre>$ service opentaxii sync guest.phishtank_com
-+ /usr/local/opentaxii/opentaxii-venv/bin/taxii-proxy --poll-path http://hailataxii.com/taxii-data --poll-collection guest.phishtank_com --inbox-path http://localhost:9000/services/guest.phishtank_com-inbox --inbox-collection guest.phishtank_com --binding urn:stix.mitre.org:xml:1.1.1 --begin 2016-04-21 --end 2016-04-22
-2016-04-21 17:36:23,778 INFO: Sending Poll_Request to http://hailataxii.com/taxii-data
-2016-04-21 17:36:23,784 INFO: Starting new HTTP connection (1): hailataxii.com
-2016-04-21 17:36:24,175 INFO: Response received for Poll_Request from http://hailataxii.com/taxii-data
-2016-04-21 17:36:24,274 INFO: Sending Inbox_Message to http://localhost:9000/services/guest.phishtank_com-inbox
-...
-2016-04-21 17:36:34,867 INFO: Response received for Poll_Request from http://localhost:9000/services/guest.phishtank_com-poll
-2016-04-21 17:36:34,868 INFO: Content blocks count: 6993, is partial: False
-</pre></div></div>
-<div class="section">
-<h3><a name="Troubleshooting"></a>Troubleshooting</h3>
-<p>Should you need to explore the installation, here are instructions on doing so.</p>
-<p>OpenTAXII is installed in a virtual environment. Before exploring the environment run the following commands to perform the necessary setup. The specific paths may change depending on your Ansible settings.</p>
-
-<div class="source">
-<div class="source">
-<pre>export LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64
-export OPENTAXII_CONFIG=/usr/local/opentaxii/etc/opentaxii-conf.yml
-cd /usr/local/opentaxii
-. opentaxii-venv/bin/activate
-</pre></div></div>
-<p>Discover available services.</p>
-
-<div class="source">
-<div class="source">
-<pre>taxii-discovery --discovery http://localhost:9000/services/discovery
-taxii-discovery --discovery http://hailataxii.com/taxii-data
-</pre></div></div>
-<p>Explore available collections.</p>
-
-<div class="source">
-<div class="source">
-<pre>taxii-collections --discovery http://localhost:9000/services/discovery
-taxii-collections --discovery http://hailataxii.com/taxii-data
-</pre></div></div>
-<p>Read data from a collection.</p>
-
-<div class="source">
-<div class="source">
-<pre>taxii-poll --discovery http://localhost:9000/services/discovery -c guest.phishtank_com
-taxii-poll --discovery http://hailataxii.com/taxii-data -c guest.phishtank_com --begin 2016-04-20
-</pre></div></div>
-<p>Manually load data into a collection.</p>
-
-<div class="source">
-<div class="source">
-<pre>taxii-push \
-  --discovery http://localhost:9000/services/discovery \
-  --dest phishtank \
-  --content-file data.xml \
-  --username guest \
-  --password guest
-</pre></div></div>
-<p>Fetch data from a remote service and mirror it locally.</p>
-
-<div class="source">
-<div class="source">
-<pre>taxii-proxy --poll-path http://hailataxii.com/taxii-data \
-            --poll-collection guest.phishtank_com \
-            --inbox-path http://localhost:9000/services/guest.phishtank_com-inbox \
-            --inbox-collection guest.phishtank_com \
-            --binding urn:stix.mitre.org:xml:1.1.1 \
-            --inbox-username guest \
-            --inbox-password guest \
-            --begin 2016-04-20
-</pre></div></div></div></div>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/roles/pcap_replay/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/pcap_replay/index.html b/site/current-book/metron-deployment/roles/pcap_replay/index.html
deleted file mode 100644
index 22bf7c0..0000000
--- a/site/current-book/metron-deployment/roles/pcap_replay/index.html
+++ /dev/null
@@ -1,355 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Pcap Replay</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../../css/site.css" />
-    <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Pcap Replay</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-down"></i>
-        Roles</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/monit/index.html" title="Monit">
-          <i class="none"></i>
-        Monit</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
-          <i class="none"></i>
-        Opentaxii</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Pcap_replay</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/sensor-stubs/index.html" title="Sensor-stubs">
-          <i class="none"></i>
-        Sensor-stubs</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
-          <i class="none"></i>
-        Sensor-test-mode</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Pcap Replay</h1>
-<p>This project enables packet capture data to be replayed through a network interface to simulate live network traffic. This can be used to support functional, performance, and load testing of Apache Metron.</p>
-<div class="section">
-<h2><a name="Getting_Started"></a>Getting Started</h2>
-<p>To replay packet capture data, simply start the <tt>pcap-replay</tt> SysV service. To do this run the following command.</p>
-
-<div class="source">
-<div class="source">
-<pre>service pcap-replay start
-</pre></div></div>
-<p>All additional options accepted by <tt>tcpreplay</tt> can be passed to the service script to modify how the network data is replayed. For example, this makes it simple to control the amount and rate of data replayed during functional, performance and load testing.</p>
-<p>Example: Replay data at a rate of 10 mbps.</p>
-
-<div class="source">
-<div class="source">
-<pre>service pcap-replay start --mbps 10
-</pre></div></div>
-<p>Example: Replay data at a rate of 10 packets per second.</p>
-
-<div class="source">
-<div class="source">
-<pre>service pcap-replay start --pps 10
-</pre></div></div>
-<p>All nodes on the same subnet with their network interface set to promiscuous mode will then be able to capture the network traffic being replayed. To validate, simply run something like the following.</p>
-
-<div class="source">
-<div class="source">
-<pre>tcpdump -i eth1
-</pre></div></div></div>
-<div class="section">
-<h2><a name="Data"></a>Data</h2>
-<p>An example packet capture file has been installed at <tt>/opt/pcap-replay/example.pcap</tt>. By default, the network traffic contained within this file is continually replayed. </p>
-<p>To replay your own packet capture data, simply add any number of files containing <tt>libpcap</tt> formatted packet capture data to <tt>/opt/pcap-replay</tt>. The files must end with the <tt>.pcap</tt> extension. To pick up newly installed files, simply restart the service.</p>
-
-<div class="source">
-<div class="source">
-<pre>service pcap-replay restart
-</pre></div></div></div>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/roles/sensor-stubs/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/roles/sensor-stubs/index.html b/site/current-book/metron-deployment/roles/sensor-stubs/index.html
deleted file mode 100644
index 436a523..0000000
--- a/site/current-book/metron-deployment/roles/sensor-stubs/index.html
+++ /dev/null
@@ -1,401 +0,0 @@
-<!DOCTYPE html>
-<!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
--->
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-  <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
-    <meta http-equiv="Content-Language" content="en" />
-    <title>Metron &#x2013; Sensor Stubs</title>
-    <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.3.0.min.css" />
-    <link rel="stylesheet" href="../../../css/site.css" />
-    <link rel="stylesheet" href="../../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
-        <div class="clear"><hr/></div>
-      </div>
-
-      <div id="breadcrumbs">
-        <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Sensor Stubs</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
-      </div>
-
-            
-      <div class="row-fluid">
-        <div id="leftColumn" class="span3">
-          <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup">
-          <i class="none"></i>
-        Kerberos-ambari-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-down"></i>
-        Roles</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/monit/index.html" title="Monit">
-          <i class="none"></i>
-        Monit</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/opentaxii/index.html" title="Opentaxii">
-          <i class="none"></i>
-        Opentaxii</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/pcap_replay/index.html" title="Pcap_replay">
-          <i class="none"></i>
-        Pcap_replay</a>
-            </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Sensor-stubs</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-deployment/roles/sensor-test-mode/index.html" title="Sensor-test-mode">
-          <i class="none"></i>
-        Sensor-test-mode</a>
-            </li>
-              </ul>
-        </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
-          </div>
-        </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Sensor Stubs</h1>
-<p><a name="Sensor_Stubs"></a></p>
-<p>A service has been created to simulate the behavior of a sensor by sending canned telemetry data to a Kafka topic. These &#x201c;Sensor Stubs&#x201d; consume fewer resources than the actual sensor that they replace.</p>
-<div class="section">
-<div class="section">
-<h3><a name="aQ_How_do_the_sensor_stubs_work"></a>(Q) How do the sensor stubs work?</h3>
-<p>The stubs are installed with a set of canned data for each sensor type; Bro, Snort and YAF. A subset of this canned data is randomly selected and sent to the Kafka topic in batches. The timestamp of each message is updated to match current system time. </p></div>
-<div class="section">
-<h3><a name="aQ_How_do_I_configure_the_message_rate"></a>(Q) How do I configure the message rate?</h3>
-<p>The number of telemetry messages sent in each batch, along with the time delay between batches is configurable. Before installation, these values can be configured by redefining <tt>sensor_stubs_delay</tt> and <tt>sensor_stubs_count</tt>. The values can also be configured by altering the deployed system service script at <tt>/etc/init.d/sensor-stubs</tt>.</p></div>
-<div class="section">
-<h3><a name="aQ_How_do_I_install_the_sensor_stubs"></a>(Q) How do I install the sensor stubs?</h3>
-<p>Using the default playbooks, this role can be installed by using the Ansible tag <tt>sensor-stubs</tt>. This service is installed on the same hosts where the sensors would be; defined by the <tt>sensors</tt> host group.</p>
-<p>The defaults for the &#x201c;Full Dev&#x201d; environment have been changed so that the Sensor Stubs are installed by default, rather than the sensors themselves. The Amazon EC2 environment continues to install the original sensors by default.</p></div>
-<div class="section">
-<h3><a name="aQ_How_do_I_use_the_sensor_stubs"></a>(Q) How do I use the sensor stubs?</h3>
-<p>Start all sensor stubs. The output includes the PID for each running sensor stub.</p>
-
-<div class="source">
-<div class="source">
-<pre>$ service sensor-stubs start
-Starting sensor-stubs...
-       bro: Ok [26505]
-       yaf: Ok [26507]
-     snort: Ok [26509]
-</pre></div></div>
-<p>Check the status of each sensor stub.</p>
-
-<div class="source">
-<div class="source">
-<pre>$ service sensor-stubs status
-Checking sensor-stubs...
-       bro: Running [26505]
-       yaf: Running [26507]
-     snort: Running [26509]
-</pre></div></div>
-<p>Stop all sensor stubs.</p>
-
-<div class="source">
-<div class="source">
-<pre>$ service sensor-stubs stop
-Stopping sensor-stubs...
-..       bro: Ok [26505]
-..       yaf: Ok [26507]
-..     snort: Ok [26509]
-</pre></div></div>
-<p>Check the status. All sensor stubs should be stopped.</p>
-
-<div class="source">
-<div class="source">
-<pre>$ service sensor-stubs status
-Checking sensor-stubs...
-       bro: Not running
-       yaf: Not running
-     snort: Not running
-</pre></div></div>
-<p>Start only the Bro sensor stub.</p>
-
-<div class="source">
-<div class="source">
-<pre>$ service sensor-stubs start bro
-Starting sensor-stubs...
-       bro: OK [11616]
-</pre></div></div>
-<p>Stop the Bro sensor stub.</p>
-
-<div class="source">
-<div class="source">
-<pre>$ service sensor-stubs stop bro
-Stopping sensor-stubs...
-..       bro: Ok [11616]
-</pre></div></div></div>
-<div class="section">
-<h3><a name="aQ_How_do_I_install_the_original_sensors"></a>(Q) How do I install the original sensors?</h3>
-<p>The default behavior can be changed by skipping the <tt>sensor-stubs</tt> flag and including the <tt>sensors</tt> flag. For example, to deploy &#x201c;Full Dev&#x201d; with the original sensors run the following command.</p>
-
-<div class="source">
-<div class="source">
-<pre>cd metron-deployment/vagrant/full-dev-platform
-vagrant --ansible-skip-tags=&quot;sensor-stubs,solr&quot; up
-</pre></div></div></div>
-<div class="section">
-<h3><a name="aQ_Where_does_the_mock_data_come_from"></a>(Q) Where does the mock data come from?</h3>
-<p>The data produced by the sensor stubs was generated by running the sensors against the example pcap file that is distributed with Metron. This ensures that the data produced by the sensor stubs is similar to the data produced when using the actual sensors.</p></div></div>
-                  </div>
-            </div>
-          </div>
-
-    <hr/>
-
-    <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
-      </div>
-
-                          
-        
-                </div>
-    </footer>
-  </body>
-</html>


[35/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-analytics/metron-statistics/HLLP.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/metron-statistics/HLLP.html b/site/current-book/metron-analytics/metron-statistics/HLLP.html
index f56beba..fdbf631 100644
--- a/site/current-book/metron-analytics/metron-statistics/HLLP.html
+++ b/site/current-book/metron-analytics/metron-statistics/HLLP.html
@@ -1,234 +1,109 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-analytics/metron-statistics/HLLP.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; HyperLogLogPlus</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">HyperLogLogPlus</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">HyperLogLogPlus</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                                
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-down"></i>
-        Analytics</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service">
-          <i class="none"></i>
-        Maas-service</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-profiler/index.html" title="Profiler">
-          <i class="none"></i>
-        Profiler</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client">
-          <i class="none"></i>
-        Profiler-client</a>
-            </li>
-                                                                                  
-      <li>
-    
-                          <a href="../../metron-analytics/metron-statistics/index.html" title="Statistics">
-          <i class="icon-chevron-down"></i>
-        Statistics</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>HLLP</a>
-          </li>
-              </ul>
-        </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-down"></span>Analytics</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service"><span class="none"></span>Maas-service</a></li>
+    <li><a href="../../metron-analytics/metron-profiler/index.html" title="Profiler"><span class="none"></span>Profiler</a></li>
+    <li><a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client"><span class="none"></span>Profiler-client</a></li>
+    <li><a href="../../metron-analytics/metron-statistics/index.html" title="Statistics"><span class="icon-chevron-down"></span>Statistics</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>HLLP</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>HyperLogLogPlus</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>HyperLogLogPlus</h1>
 <p><a name="HyperLogLogPlus"></a></p>
 <div class="section">
-<h2><a name="Preliminaries_-_Distinct_Value_cardinality_Estimation"></a>Preliminaries - Distinct Value (cardinality) Estimation</h2>
+<h2><a name="Preliminaries_-_Distinct_Value_.28cardinality.29_Estimation"></a>Preliminaries - Distinct Value (cardinality) Estimation</h2>
 <p>Calculating the number of distinct values (DV) in a specific finite-sized list of values using brute force (e.g. a HashSet) techniques is fairly straight forward provided that linear scaling to your number of values fits within your memory constraints. For many cases, the simple calculation serves just fine. However, in many other instances the simple linear approach simply will not scale as well as we would like, and a more robust algorithm is desirable. Enter probabilistic distinct value estimation algorithms: k-minimum-values (KMV), LogLog, HyperLogLog (HLL), and most recently HyperLogLog++ (HLLP). As data volume grows it becomes impractical to store all values in memory in order to reasonably calculate set cardinality. This is where probabilistic estimation algorithms become extremely useful. Often, we are happy to trade a handful of memory and better performance in exchange for an inexact, yet reasonably close approximation of the actual cardinality.</p></div>
 <div class="section">
 <h2><a name="The_Case_For_An_Improvement"></a>The Case For An Improvement</h2>
@@ -239,31 +114,30 @@
 <h3><a name="Overview"></a>Overview</h3>
 <p>Below is a table detailing the performance characteristics when running HLLP over multiple data set sizes. There are three primary variables manipulated during the test: cardinality, sparse set precision, and normal set precision. Each of these values will have an impact on error rates, execution time, and total memory consumption.</p></div>
 <div class="section">
-<h3><a name="Choosing_p_sp"></a>Choosing p, sp</h3>
+<h3><a name="Choosing_p.2C_sp"></a>Choosing p, sp</h3>
 <p>There is a tradeoff when choosing values for p and sp. A higher value for sp means a lower error rate for instances when the sparse set is being used. However, a higher value for sp also requires more memory, which means the algorithm will cut over to the dense representation more quickly. The Google paper (listed above) recommends settings of p=25, sp=14 for optimal memory use and algorithm accuracy, and these are the defaults provided by the Apache Metron implementation in Stellar. You&#x2019;ll notice in the tests below, memory consumption tops out around 12KB for this setting, regardless of cardinality size.</p></div>
 <div class="section">
-<h3><a name="Key"></a>Key</h3>
-<hr />
+<h3><a name="Key"></a>Key</h3><hr />
 
-<div class="source">
-<div class="source">
-<pre>card: cardinality&lt;br&gt;
+<div>
+<div>
+<pre class="source">card: cardinality&lt;br&gt;
 sp: sparse precision value&lt;br&gt;
 p: normal (dense) precision value&lt;br&gt;
 err: error as a percent of the expected cardinality&lt;br&gt;
 time: total time to add values to the hllp estimator and calculate a cardinality estimate&lt;br&gt;
 size: size of the hllp set in bytes once all values have been added for the specified cardinality&lt;br&gt;
 l=low, m=mid(based on percentile chosen), h=high, std=standard deviation&lt;br&gt;
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
-<h3><a name="Table_Cardinality_200-1000_step_by_200"></a>Table Cardinality 200-1000, step by 200</h3>
-<hr />
+<h3><a name="Table_Cardinality_200-1000.2C_step_by_200"></a>Table Cardinality 200-1000, step by 200</h3><hr />
 <div class="section">
 <h4><a name="Options_Used"></a>Options Used</h4>
 
-<div class="source">
-<div class="source">
-<pre>num trials: 5000
+<div>
+<div>
+<pre class="source">num trials: 5000
 card min: 200
 card max: 1000
 card step: 200
@@ -278,2735 +152,1288 @@ format err as %: true
 </pre></div></div>
 
 <table border="0" class="table table-striped">
-  <thead>
-    
-<tr class="a">
-      
-<th>card </th>
-      
-<th>sp </th>
-      
-<th>p </th>
-      
-<th>err l/m/h/std (% of actual) </th>
-      
-<th>time l/m/h/std (ms) </th>
-      
-<th>size l/m/h/std (b) </th>
-    </tr>
-  </thead>
-  <tbody>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>4 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.000 / 180.000 / 17.470 </td>
-      
-<td>0.070 / 0.118 / 10.092 / 0.204 </td>
-      
-<td>218 / 220 / 220 / 1 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>8 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.000 / 209.500 / 18.060 </td>
-      
-<td>0.071 / 0.085 / 8.786 / 0.128 </td>
-      
-<td>219 / 221 / 221 / 1 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>8 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.500 / 21.500 / 3.185 </td>
-      
-<td>0.082 / 0.085 / 8.375 / 0.119 </td>
-      
-<td>531 / 587 / 637 / 14 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>12 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.000 / 174.000 / 17.376 </td>
-      
-<td>0.070 / 0.085 / 8.708 / 0.128 </td>
-      
-<td>219 / 221 / 221 / 1 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>12 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.000 / 25.500 / 2.847 </td>
-      
-<td>0.086 / 0.119 / 8.984 / 0.151 </td>
-      
-<td>353 / 372 / 627 / 74 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>12 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.500 / 5.500 / 0.659 </td>
-      
-<td>0.082 / 0.096 / 17.208 / 0.243 </td>
-      
-<td>775 / 816 / 834 / 9 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>16 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.500 / 172.000 / 16.622 </td>
-      
-<td>0.068 / 0.075 / 9.553 / 0.137 </td>
-      
-<td>220 / 222 / 222 / 1 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>16 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.500 / 18.000 / 2.899 </td>
-      
-<td>0.095 / 0.118 / 0.663 / 0.031 </td>
-      
-<td>354 / 373 / 390 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>16 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 1.500 / 0.276 </td>
-      
-<td>0.085 / 0.108 / 13.977 / 0.199 </td>
-      
-<td>802 / 833 / 852 / 6 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>16 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 2.500 / 0.279 </td>
-      
-<td>0.079 / 0.085 / 0.271 / 0.017 </td>
-      
-<td>990 / 1027 / 1044 / 6 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>20 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.000 / 209.500 / 17.424 </td>
-      
-<td>0.068 / 0.072 / 0.250 / 0.014 </td>
-      
-<td>221 / 223 / 223 / 1 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>20 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.500 / 18.000 / 2.944 </td>
-      
-<td>0.096 / 0.106 / 13.581 / 0.192 </td>
-      
-<td>357 / 374 / 388 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>20 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 1.000 / 0.066 </td>
-      
-<td>0.084 / 0.088 / 1.096 / 0.019 </td>
-      
-<td>915 / 945 / 972 / 8 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>20 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.070 </td>
-      
-<td>0.083 / 0.087 / 10.272 / 0.145 </td>
-      
-<td>912 / 953 / 982 / 8 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>20 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 1.000 / 0.069 </td>
-      
-<td>0.079 / 0.081 / 0.210 / 0.017 </td>
-      
-<td>1028 / 1056 / 1058 / 2 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>24 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.500 / 136.000 / 16.589 </td>
-      
-<td>0.068 / 0.071 / 0.952 / 0.025 </td>
-      
-<td>221 / 223 / 223 / 1 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>24 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.500 / 18.000 / 2.955 </td>
-      
-<td>0.095 / 0.106 / 10.387 / 0.148 </td>
-      
-<td>353 / 373 / 390 / 5 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>24 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.016 </td>
-      
-<td>0.082 / 0.083 / 0.235 / 0.014 </td>
-      
-<td>1035 / 1048 / 1056 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>24 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.012 </td>
-      
-<td>0.082 / 0.084 / 10.904 / 0.154 </td>
-      
-<td>1038 / 1051 / 1059 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>24 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.010 </td>
-      
-<td>0.082 / 0.085 / 0.410 / 0.019 </td>
-      
-<td>1049 / 1064 / 1082 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>24 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.023 </td>
-      
-<td>0.079 / 0.084 / 11.926 / 0.168 </td>
-      
-<td>1247 / 1264 / 1275 / 4 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>28 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.500 / 229.000 / 17.294 </td>
-      
-<td>0.068 / 0.072 / 0.369 / 0.015 </td>
-      
-<td>222 / 224 / 224 / 1 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>28 </td>
-      
-<td>8 </td>
-      
-<td>83.000 / 83.000 / 84.000 / 0.118 </td>
-      
-<td>0.093 / 0.098 / 10.918 / 0.157 </td>
-      
-<td>272 / 276 / 278 / 1 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>28 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.016 </td>
-      
-<td>0.082 / 0.083 / 10.157 / 0.143 </td>
-      
-<td>1032 / 1067 / 1075 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>28 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.007 </td>
-      
-<td>0.082 / 0.083 / 0.617 / 0.020 </td>
-      
-<td>1058 / 1069 / 1077 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>28 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.010 </td>
-      
-<td>0.082 / 0.085 / 11.619 / 0.165 </td>
-      
-<td>1060 / 1071 / 1080 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>28 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.007 </td>
-      
-<td>0.083 / 0.085 / 0.516 / 0.022 </td>
-      
-<td>1065 / 1083 / 1100 / 4 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>28 </td>
-      
-<td>28 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.019 </td>
-      
-<td>0.080 / 0.087 / 11.278 / 0.160 </td>
-      
-<td>1250 / 1267 / 1278 / 4 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>32 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 18.000 / 175.500 / 17.157 </td>
-      
-<td>0.068 / 0.078 / 0.223 / 0.019 </td>
-      
-<td>222 / 224 / 224 / 1 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>32 </td>
-      
-<td>8 </td>
-      
-<td>99.000 / 99.000 / 99.000 / 0.000 </td>
-      
-<td>0.093 / 0.107 / 11.695 / 0.167 </td>
-      
-<td>258 / 258 / 258 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>32 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.014 </td>
-      
-<td>0.081 / 0.084 / 8.906 / 0.126 </td>
-      
-<td>1060 / 1071 / 1075 / 2 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>32 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.010 </td>
-      
-<td>0.082 / 0.085 / 1.458 / 0.027 </td>
-      
-<td>1061 / 1073 / 1077 / 2 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>32 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.012 </td>
-      
-<td>0.082 / 0.084 / 10.499 / 0.148 </td>
-      
-<td>1065 / 1074 / 1079 / 2 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>32 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.012 </td>
-      
-<td>0.082 / 0.085 / 0.420 / 0.020 </td>
-      
-<td>1066 / 1075 / 1082 / 2 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>200 </td>
-      
-<td>32 </td>
-      
-<td>28 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.016 </td>
-      
-<td>0.082 / 0.086 / 8.753 / 0.124 </td>
-      
-<td>1076 / 1088 / 1102 / 4 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>200 </td>
-      
-<td>32 </td>
-      
-<td>32 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.010 </td>
-      
-<td>0.080 / 0.081 / 13.248 / 0.188 </td>
-      
-<td>1249 / 1268 / 1279 / 4 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>4 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.500 / 175.250 / 17.530 </td>
-      
-<td>0.136 / 0.142 / 0.591 / 0.035 </td>
-      
-<td>218 / 220 / 220 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>8 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.000 / 148.500 / 17.174 </td>
-      
-<td>0.136 / 0.156 / 12.882 / 0.241 </td>
-      
-<td>219 / 221 / 221 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>8 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.500 / 21.250 / 3.215 </td>
-      
-<td>0.159 / 0.186 / 10.396 / 0.151 </td>
-      
-<td>375 / 387 / 802 / 62 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>12 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.750 / 166.000 / 17.363 </td>
-      
-<td>0.135 / 0.142 / 11.607 / 0.165 </td>
-      
-<td>219 / 221 / 221 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>12 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.250 / 23.750 / 3.133 </td>
-      
-<td>0.166 / 0.179 / 8.713 / 0.130 </td>
-      
-<td>376 / 387 / 401 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>12 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.750 / 4.250 / 0.706 </td>
-      
-<td>0.159 / 0.163 / 10.537 / 0.189 </td>
-      
-<td>1334 / 1387 / 1434 / 14 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>16 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.125 / 163.250 / 17.208 </td>
-      
-<td>0.135 / 0.138 / 12.403 / 0.178 </td>
-      
-<td>220 / 222 / 222 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>16 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.250 / 23.500 / 3.126 </td>
-      
-<td>0.165 / 0.179 / 9.985 / 0.147 </td>
-      
-<td>378 / 388 / 401 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>16 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.250 / 1.500 / 0.195 </td>
-      
-<td>0.166 / 0.173 / 10.434 / 0.150 </td>
-      
-<td>1414 / 1446 / 1475 / 8 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>16 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.250 / 1.750 / 0.194 </td>
-      
-<td>0.160 / 0.177 / 1.300 / 0.048 </td>
-      
-<td>1761 / 1798 / 1825 / 8 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>20 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.750 / 270.750 / 17.592 </td>
-      
-<td>0.135 / 0.149 / 8.814 / 0.128 </td>
-      
-<td>221 / 223 / 223 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>20 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.500 / 23.000 / 3.103 </td>
-      
-<td>0.165 / 0.183 / 9.271 / 0.139 </td>
-      
-<td>379 / 389 / 398 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>20 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.066 </td>
-      
-<td>0.164 / 0.182 / 11.093 / 0.162 </td>
-      
-<td>1617 / 1652 / 1683 / 10 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>20 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.070 </td>
-      
-<td>0.166 / 0.170 / 8.676 / 0.128 </td>
-      
-<td>1624 / 1665 / 1706 / 10 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>20 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.750 / 0.069 </td>
-      
-<td>0.159 / 0.162 / 12.846 / 0.183 </td>
-      
-<td>1840 / 1853 / 1856 / 2 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>24 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 16.750 / 174.750 / 16.786 </td>
-      
-<td>0.135 / 0.145 / 8.586 / 0.128 </td>
-      
-<td>221 / 223 / 223 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>24 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.500 / 24.500 / 3.090 </td>
-      
-<td>0.165 / 0.179 / 11.344 / 0.164 </td>
-      
-<td>377 / 389 / 399 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>24 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.016 </td>
-      
-<td>0.164 / 0.183 / 10.708 / 0.159 </td>
-      
-<td>1828 / 1841 / 1852 / 4 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>24 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.017 </td>
-      
-<td>0.164 / 0.172 / 11.180 / 0.161 </td>
-      
-<td>1830 / 1844 / 1855 / 4 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>24 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.015 </td>
-      
-<td>0.166 / 0.170 / 9.589 / 0.140 </td>
-      
-<td>1843 / 1868 / 1890 / 6 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>24 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.022 </td>
-      
-<td>0.160 / 0.162 / 9.137 / 0.134 </td>
-      
-<td>2213 / 2232 / 2247 / 5 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>28 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 18.250 / 164.000 / 16.587 </td>
-      
-<td>0.135 / 0.144 / 14.520 / 0.209 </td>
-      
-<td>222 / 224 / 224 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>28 </td>
-      
-<td>8 </td>
-      
-<td>33.000 / 47.750 / 56.750 / 3.588 </td>
-      
-<td>0.164 / 0.168 / 1.305 / 0.052 </td>
-      
-<td>355 / 375 / 394 / 5 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>28 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.006 </td>
-      
-<td>0.165 / 0.178 / 11.264 / 0.167 </td>
-      
-<td>1837 / 1848 / 1854 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>28 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.014 </td>
-      
-<td>0.164 / 0.167 / 8.689 / 0.130 </td>
-      
-<td>1838 / 1850 / 1857 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>28 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.011 </td>
-      
-<td>0.164 / 0.167 / 10.679 / 0.159 </td>
-      
-<td>1839 / 1852 / 1863 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>28 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.011 </td>
-      
-<td>0.166 / 0.175 / 12.887 / 0.190 </td>
-      
-<td>1857 / 1875 / 1896 / 5 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>28 </td>
-      
-<td>28 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.015 </td>
-      
-<td>0.160 / 0.196 / 1.478 / 0.071 </td>
-      
-<td>2217 / 2235 / 2252 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>32 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.500 / 183.750 / 19.308 </td>
-      
-<td>0.135 / 0.142 / 9.949 / 0.148 </td>
-      
-<td>222 / 224 / 224 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>32 </td>
-      
-<td>8 </td>
-      
-<td>46.500 / 56.250 / 63.250 / 2.194 </td>
-      
-<td>0.164 / 0.181 / 1.209 / 0.063 </td>
-      
-<td>349 / 370 / 390 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>32 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.012 </td>
-      
-<td>0.164 / 0.171 / 11.529 / 0.170 </td>
-      
-<td>1837 / 1848 / 1854 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>32 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.016 </td>
-      
-<td>0.164 / 0.167 / 10.276 / 0.154 </td>
-      
-<td>1836 / 1850 / 1856 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>32 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.015 </td>
-      
-<td>0.164 / 0.184 / 11.611 / 0.227 </td>
-      
-<td>1839 / 1851 / 1858 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>32 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.012 </td>
-      
-<td>0.165 / 0.171 / 9.313 / 0.143 </td>
-      
-<td>1842 / 1853 / 1862 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>400 </td>
-      
-<td>32 </td>
-      
-<td>28 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.013 </td>
-      
-<td>0.166 / 0.180 / 10.708 / 0.162 </td>
-      
-<td>1860 / 1877 / 1900 / 5 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>400 </td>
-      
-<td>32 </td>
-      
-<td>32 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.014 </td>
-      
-<td>0.160 / 0.166 / 1.205 / 0.057 </td>
-      
-<td>2217 / 2235 / 2249 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>4 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.333 / 219.667 / 17.222 </td>
-      
-<td>0.202 / 0.240 / 13.390 / 0.237 </td>
-      
-<td>218 / 220 / 220 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>8 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.417 / 231.333 / 17.340 </td>
-      
-<td>0.202 / 0.224 / 9.662 / 0.150 </td>
-      
-<td>220 / 221 / 221 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>8 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.500 / 24.333 / 3.240 </td>
-      
-<td>0.234 / 0.271 / 11.443 / 0.216 </td>
-      
-<td>382 / 392 / 403 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>12 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.167 / 188.167 / 17.324 </td>
-      
-<td>0.202 / 0.210 / 1.320 / 0.055 </td>
-      
-<td>219 / 221 / 221 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>12 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.500 / 23.833 / 3.243 </td>
-      
-<td>0.235 / 0.246 / 10.821 / 0.164 </td>
-      
-<td>382 / 392 / 403 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>12 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.667 / 4.500 / 0.694 </td>
-      
-<td>0.240 / 0.259 / 12.004 / 0.183 </td>
-      
-<td>2015 / 2101 / 2178 / 21 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>16 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.500 / 170.000 / 17.882 </td>
-      
-<td>0.202 / 0.218 / 9.208 / 0.140 </td>
-      
-<td>221 / 222 / 222 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>16 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.667 / 22.667 / 3.276 </td>
-      
-<td>0.234 / 0.251 / 12.301 / 0.237 </td>
-      
-<td>383 / 393 / 408 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>16 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.167 / 1.000 / 0.170 </td>
-      
-<td>0.250 / 0.284 / 12.000 / 0.180 </td>
-      
-<td>2199 / 2243 / 2282 / 12 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>16 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.167 / 1.167 / 0.171 </td>
-      
-<td>0.239 / 0.291 / 21.163 / 0.333 </td>
-      
-<td>2811 / 2859 / 2901 / 13 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>20 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.667 / 176.667 / 18.162 </td>
-      
-<td>0.202 / 0.223 / 16.405 / 0.275 </td>
-      
-<td>221 / 223 / 223 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>20 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.500 / 20.833 / 3.180 </td>
-      
-<td>0.234 / 0.243 / 5.621 / 0.104 </td>
-      
-<td>385 / 394 / 405 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>20 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.066 </td>
-      
-<td>0.246 / 0.281 / 19.279 / 0.390 </td>
-      
-<td>2541 / 2604 / 2663 / 15 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>20 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.667 / 0.071 </td>
-      
-<td>0.248 / 0.258 / 24.627 / 0.387 </td>
-      
-<td>2569 / 2627 / 2682 / 15 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>20 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.069 </td>
-      
-<td>0.239 / 0.246 / 13.606 / 0.248 </td>
-      
-<td>2938 / 2954 / 2972 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>24 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.333 / 210.333 / 18.157 </td>
-      
-<td>0.202 / 0.218 / 11.991 / 0.180 </td>
-      
-<td>222 / 223 / 223 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>24 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.667 / 21.167 / 3.184 </td>
-      
-<td>0.234 / 0.258 / 11.130 / 0.238 </td>
-      
-<td>385 / 394 / 406 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>24 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.017 </td>
-      
-<td>0.246 / 0.272 / 12.032 / 0.191 </td>
-      
-<td>2912 / 2935 / 2954 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>24 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.019 </td>
-      
-<td>0.246 / 0.268 / 19.358 / 0.282 </td>
-      
-<td>2916 / 2940 / 2959 / 6 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>24 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.015 </td>
-      
-<td>0.248 / 0.259 / 20.293 / 0.343 </td>
-      
-<td>2948 / 2980 / 3009 / 9 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>24 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.017 </td>
-      
-<td>0.239 / 0.252 / 11.721 / 0.230 </td>
-      
-<td>3586 / 3618 / 3643 / 7 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>28 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.667 / 182.667 / 17.309 </td>
-      
-<td>0.202 / 0.210 / 11.957 / 0.177 </td>
-      
-<td>223 / 224 / 224 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>28 </td>
-      
-<td>8 </td>
-      
-<td>10.333 / 28.000 / 42.833 / 3.687 </td>
-      
-<td>0.233 / 0.238 / 10.166 / 0.210 </td>
-      
-<td>377 / 392 / 403 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>28 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.333 / 0.013 </td>
-      
-<td>0.246 / 0.252 / 9.907 / 0.158 </td>
-      
-<td>3005 / 3024 / 3041 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>28 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.012 </td>
-      
-<td>0.246 / 0.256 / 18.673 / 0.273 </td>
-      
-<td>3007 / 3026 / 3045 / 5 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>28 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.011 </td>
-      
-<td>0.246 / 0.283 / 17.762 / 0.355 </td>
-      
-<td>3005 / 3029 / 3047 / 6 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>28 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.011 </td>
-      
-<td>0.248 / 0.254 / 8.821 / 0.138 </td>
-      
-<td>3035 / 3065 / 3098 / 8 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>28 </td>
-      
-<td>28 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.011 </td>
-      
-<td>0.239 / 0.242 / 12.171 / 0.177 </td>
-      
-<td>3589 / 3621 / 3646 / 7 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>32 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.333 / 200.000 / 18.294 </td>
-      
-<td>0.202 / 0.205 / 9.036 / 0.138 </td>
-      
-<td>223 / 224 / 224 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>32 </td>
-      
-<td>8 </td>
-      
-<td>25.167 / 37.167 / 46.833 / 3.185 </td>
-      
-<td>0.232 / 0.236 / 9.061 / 0.144 </td>
-      
-<td>377 / 389 / 403 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>32 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.013 </td>
-      
-<td>0.246 / 0.256 / 11.119 / 0.169 </td>
-      
-<td>3028 / 3043 / 3055 / 4 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>32 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.010 </td>
-      
-<td>0.246 / 0.257 / 11.557 / 0.213 </td>
-      
-<td>3030 / 3045 / 3057 / 4 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>32 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.011 </td>
-      
-<td>0.246 / 0.266 / 20.353 / 0.295 </td>
-      
-<td>3030 / 3047 / 3071 / 4 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>32 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.012 </td>
-      
-<td>0.246 / 0.270 / 11.461 / 0.183 </td>
-      
-<td>3035 / 3050 / 3073 / 4 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>600 </td>
-      
-<td>32 </td>
-      
-<td>28 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.013 </td>
-      
-<td>0.248 / 0.260 / 11.755 / 0.242 </td>
-      
-<td>3060 / 3085 / 3108 / 7 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>600 </td>
-      
-<td>32 </td>
-      
-<td>32 </td>
-      
-<td>0.000 / 0.000 / 0.167 / 0.012 </td>
-      
-<td>0.239 / 0.248 / 10.380 / 0.157 </td>
-      
-<td>3593 / 3621 / 3645 / 7 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>4 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.250 / 188.250 / 17.248 </td>
-      
-<td>0.269 / 0.280 / 10.818 / 0.212 </td>
-      
-<td>219 / 220 / 220 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>8 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.875 / 184.125 / 17.721 </td>
-      
-<td>0.270 / 0.298 / 16.706 / 0.252 </td>
-      
-<td>220 / 221 / 221 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>8 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.625 / 22.000 / 3.420 </td>
-      
-<td>0.303 / 0.315 / 12.489 / 0.243 </td>
-      
-<td>386 / 395 / 406 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>12 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.750 / 196.500 / 16.870 </td>
-      
-<td>0.269 / 0.289 / 13.533 / 0.235 </td>
-      
-<td>220 / 221 / 221 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>12 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.750 / 25.875 / 3.359 </td>
-      
-<td>0.303 / 0.330 / 18.306 / 0.365 </td>
-      
-<td>386 / 395 / 408 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>12 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.750 / 4.375 / 0.677 </td>
-      
-<td>0.322 / 0.357 / 20.131 / 0.350 </td>
-      
-<td>2449 / 2603 / 2694 / 27 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>16 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.750 / 195.250 / 17.455 </td>
-      
-<td>0.269 / 0.274 / 10.472 / 0.205 </td>
-      
-<td>221 / 222 / 222 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>16 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.750 / 20.375 / 3.357 </td>
-      
-<td>0.302 / 0.308 / 10.637 / 0.165 </td>
-      
-<td>386 / 396 / 409 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>16 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.125 / 1.375 / 0.170 </td>
-      
-<td>0.334 / 0.344 / 10.745 / 0.221 </td>
-      
-<td>2712 / 2836 / 2882 / 15 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>16 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.250 / 1.375 / 0.169 </td>
-      
-<td>0.321 / 0.334 / 13.979 / 0.252 </td>
-      
-<td>3390 / 3625 / 3668 / 19 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>20 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.750 / 220.750 / 17.953 </td>
-      
-<td>0.269 / 0.281 / 9.422 / 0.148 </td>
-      
-<td>222 / 223 / 223 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>20 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.750 / 21.875 / 3.305 </td>
-      
-<td>0.302 / 0.358 / 24.073 / 0.435 </td>
-      
-<td>387 / 397 / 408 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>20 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.067 </td>
-      
-<td>0.331 / 0.345 / 13.461 / 0.211 </td>
-      
-<td>3249 / 3304 / 3360 / 17 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>20 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.069 </td>
-      
-<td>0.333 / 0.349 / 8.271 / 0.146 </td>
-      
-<td>3190 / 3331 / 3392 / 17 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>20 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.375 / 0.069 </td>
-      
-<td>0.320 / 0.335 / 11.307 / 0.239 </td>
-      
-<td>3539 / 3752 / 3771 / 5 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>24 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.125 / 184.500 / 18.184 </td>
-      
-<td>0.269 / 0.283 / 1.541 / 0.079 </td>
-      
-<td>222 / 223 / 223 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>24 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.625 / 23.125 / 3.301 </td>
-      
-<td>0.302 / 0.317 / 10.845 / 0.169 </td>
-      
-<td>388 / 397 / 409 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>24 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.017 </td>
-      
-<td>0.330 / 0.352 / 10.221 / 0.244 </td>
-      
-<td>3703 / 3729 / 3749 / 6 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>24 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.018 </td>
-      
-<td>0.330 / 0.344 / 13.454 / 0.239 </td>
-      
-<td>3711 / 3734 / 3753 / 6 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>24 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.018 </td>
-      
-<td>0.333 / 0.348 / 16.987 / 0.374 </td>
-      
-<td>3751 / 3786 / 3826 / 10 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>24 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.018 </td>
-      
-<td>0.320 / 0.333 / 17.097 / 0.288 </td>
-      
-<td>4567 / 4605 / 4630 / 8 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>28 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.375 / 326.125 / 18.121 </td>
-      
-<td>0.269 / 0.274 / 9.600 / 0.149 </td>
-      
-<td>223 / 224 / 224 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>28 </td>
-      
-<td>8 </td>
-      
-<td>0.375 / 19.875 / 34.500 / 4.445 </td>
-      
-<td>0.302 / 0.313 / 10.025 / 0.181 </td>
-      
-<td>387 / 397 / 409 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>28 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.125 / 0.011 </td>
-      
-<td>0.330 / 0.343 / 11.347 / 0.216 </td>
-      
-<td>3797 / 3821 / 3839 / 6 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>28 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.125 / 0.010 </td>
-      
-<td>0.330 / 0.344 / 10.748 / 0.219 </td>
-      
-<td>3801 / 3823 / 3843 / 6 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>28 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.125 / 0.011 </td>
-      
-<td>0.330 / 0.355 / 15.268 / 0.314 </td>
-      
-<td>3802 / 3827 / 3845 / 6 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>28 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.125 / 0.011 </td>
-      
-<td>0.334 / 0.360 / 11.192 / 0.243 </td>
-      
-<td>3841 / 3874 / 3905 / 9 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>28 </td>
-      
-<td>28 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.011 </td>
-      
-<td>0.321 / 0.334 / 1.757 / 0.095 </td>
-      
-<td>4575 / 4609 / 4635 / 8 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>32 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.125 / 161.000 / 18.370 </td>
-      
-<td>0.269 / 0.279 / 20.700 / 0.344 </td>
-      
-<td>223 / 224 / 224 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>32 </td>
-      
-<td>8 </td>
-      
-<td>10.500 / 27.750 / 39.500 / 3.883 </td>
-      
-<td>0.301 / 0.333 / 18.781 / 0.332 </td>
-      
-<td>385 / 395 / 406 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>32 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.011 </td>
-      
-<td>0.330 / 0.371 / 18.883 / 0.317 </td>
-      
-<td>3823 / 3840 / 3853 / 4 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>32 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.125 / 0.011 </td>
-      
-<td>0.330 / 0.353 / 9.319 / 0.195 </td>
-      
-<td>3824 / 3842 / 3855 / 4 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>32 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.125 / 0.012 </td>
-      
-<td>0.330 / 0.353 / 15.582 / 0.305 </td>
-      
-<td>3826 / 3843 / 3859 / 4 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>32 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.125 / 0.013 </td>
-      
-<td>0.330 / 0.349 / 11.346 / 0.234 </td>
-      
-<td>3830 / 3847 / 3862 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>800 </td>
-      
-<td>32 </td>
-      
-<td>28 </td>
-      
-<td>0.000 / 0.000 / 0.250 / 0.013 </td>
-      
-<td>0.333 / 0.368 / 14.482 / 0.309 </td>
-      
-<td>3866 / 3894 / 3925 / 8 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>800 </td>
-      
-<td>32 </td>
-      
-<td>32 </td>
-      
-<td>0.000 / 0.000 / 0.125 / 0.012 </td>
-      
-<td>0.320 / 0.335 / 17.771 / 0.295 </td>
-      
-<td>4576 / 4608 / 4633 / 8 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>4 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.800 / 187.500 / 18.576 </td>
-      
-<td>0.337 / 0.374 / 10.384 / 0.179 </td>
-      
-<td>219 / 220 / 220 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>8 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.600 / 181.100 / 18.035 </td>
-      
-<td>0.336 / 0.372 / 13.220 / 0.262 </td>
-      
-<td>220 / 221 / 221 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>8 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.900 / 23.100 / 3.425 </td>
-      
-<td>0.371 / 0.398 / 9.495 / 0.175 </td>
-      
-<td>386 / 398 / 409 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>12 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.600 / 178.900 / 17.559 </td>
-      
-<td>0.336 / 0.351 / 11.176 / 0.265 </td>
-      
-<td>220 / 221 / 221 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>12 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.800 / 22.700 / 3.438 </td>
-      
-<td>0.371 / 0.392 / 10.942 / 0.188 </td>
-      
-<td>387 / 398 / 411 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>12 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.800 / 4.200 / 0.684 </td>
-      
-<td>0.403 / 0.443 / 11.219 / 0.279 </td>
-      
-<td>2971 / 3242 / 3364 / 39 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>16 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.600 / 184.300 / 17.148 </td>
-      
-<td>0.336 / 0.368 / 21.656 / 0.350 </td>
-      
-<td>221 / 222 / 222 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>16 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.800 / 21.900 / 3.350 </td>
-      
-<td>0.369 / 0.385 / 15.840 / 0.277 </td>
-      
-<td>388 / 399 / 412 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>16 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.200 / 1.200 / 0.174 </td>
-      
-<td>0.422 / 0.438 / 16.035 / 0.335 </td>
-      
-<td>3428 / 3530 / 3628 / 16 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>16 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.200 / 1.200 / 0.173 </td>
-      
-<td>0.405 / 0.420 / 13.370 / 0.305 </td>
-      
-<td>4314 / 4484 / 4573 / 20 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>20 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.600 / 223.000 / 16.968 </td>
-      
-<td>0.336 / 0.372 / 17.746 / 0.392 </td>
-      
-<td>222 / 223 / 223 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>20 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.800 / 21.400 / 3.384 </td>
-      
-<td>0.369 / 0.419 / 16.486 / 0.257 </td>
-      
-<td>389 / 400 / 412 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>20 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.400 / 0.068 </td>
-      
-<td>0.417 / 0.446 / 12.313 / 0.265 </td>
-      
-<td>3954 / 4099 / 4158 / 18 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>20 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.500 / 0.069 </td>
-      
-<td>0.422 / 0.439 / 11.284 / 0.262 </td>
-      
-<td>4016 / 4134 / 4211 / 19 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>20 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.600 / 0.069 </td>
-      
-<td>0.404 / 0.429 / 16.987 / 0.335 </td>
-      
-<td>4517 / 4641 / 4651 / 5 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>24 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.600 / 226.700 / 17.761 </td>
-      
-<td>0.339 / 0.351 / 12.804 / 0.275 </td>
-      
-<td>222 / 223 / 223 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>24 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 3.800 / 24.100 / 3.403 </td>
-      
-<td>0.371 / 0.406 / 15.710 / 0.298 </td>
-      
-<td>390 / 400 / 412 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>24 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.018 </td>
-      
-<td>0.417 / 0.449 / 12.253 / 0.294 </td>
-      
-<td>4494 / 4614 / 4636 / 7 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>24 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.100 / 0.018 </td>
-      
-<td>0.417 / 0.434 / 11.046 / 0.254 </td>
-      
-<td>4594 / 4620 / 4641 / 7 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>24 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.019 </td>
-      
-<td>0.422 / 0.458 / 13.433 / 0.270 </td>
-      
-<td>4645 / 4682 / 4722 / 11 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>24 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.016 </td>
-      
-<td>0.405 / 0.435 / 11.470 / 0.189 </td>
-      
-<td>5635 / 5672 / 5703 / 9 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>28 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 17.200 / 148.500 / 17.046 </td>
-      
-<td>0.338 / 0.363 / 20.446 / 0.380 </td>
-      
-<td>224 / 224 / 224 / 0 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>28 </td>
-      
-<td>8 </td>
-      
-<td>0.000 / 15.100 / 30.700 / 4.794 </td>
-      
-<td>0.371 / 0.389 / 18.272 / 0.359 </td>
-      
-<td>390 / 400 / 411 / 3 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>28 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.100 / 0.013 </td>
-      
-<td>0.417 / 0.432 / 13.575 / 0.264 </td>
-      
-<td>4677 / 4700 / 4719 / 6 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>28 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.013 </td>
-      
-<td>0.417 / 0.435 / 18.060 / 0.381 </td>
-      
-<td>4545 / 4702 / 4723 / 6 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>28 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.013 </td>
-      
-<td>0.417 / 0.448 / 17.596 / 0.344 </td>
-      
-<td>4679 / 4707 / 4728 / 6 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>28 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.012 </td>
-      
-<td>0.421 / 0.454 / 13.807 / 0.297 </td>
-      
-<td>4733 / 4764 / 4801 / 10 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>28 </td>
-      
-<td>28 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.013 </td>
-      
-<td>0.406 / 0.456 / 20.533 / 0.353 </td>
-      
-<td>5646 / 5675 / 5704 / 9 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>32 </td>
-      
-<td>4 </td>
-      
-<td>0.000 / 18.150 / 177.600 / 19.537 </td>
-      
-<td>0.338 / 0.352 / 12.293 / 0.252 </td>
-      
-<td>223 / 224 / 224 / 0 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>32 </td>
-      
-<td>8 </td>
-      
-<td>1.500 / 22.100 / 35.100 / 4.399 </td>
-      
-<td>0.371 / 0.407 / 17.782 / 0.386 </td>
-      
-<td>388 / 398 / 412 / 3 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>32 </td>
-      
-<td>12 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.011 </td>
-      
-<td>0.417 / 0.435 / 11.830 / 0.247 </td>
-      
-<td>4698 / 4717 / 4730 / 5 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>32 </td>
-      
-<td>16 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.013 </td>
-      
-<td>0.417 / 0.443 / 18.214 / 0.276 </td>
-      
-<td>4700 / 4719 / 4733 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>32 </td>
-      
-<td>20 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.012 </td>
-      
-<td>0.417 / 0.437 / 11.208 / 0.232 </td>
-      
-<td>4703 / 4720 / 4737 / 5 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>32 </td>
-      
-<td>24 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.013 </td>
-      
-<td>0.417 / 0.448 / 18.288 / 0.405 </td>
-      
-<td>4706 / 4724 / 4740 / 5 </td>
-    </tr>
-    
-<tr class="b">
-      
-<td>1000 </td>
-      
-<td>32 </td>
-      
-<td>28 </td>
-      
-<td>0.000 / 0.000 / 0.200 / 0.011 </td>
-      
-<td>0.420 / 0.437 / 13.389 / 0.245 </td>
-      
-<td>4613 / 4782 / 4814 / 9 </td>
-    </tr>
-    
-<tr class="a">
-      
-<td>1000 </td>
-      
-<td>32 </td>
-      
-<td>32 </td>
-      
-<td>0.000 / 0.000 / 0.100 / 0.012 </td>
-      
-<td>0.406 / 0.424 / 12.513 / 0.232 </td>
-      
-<td>5641 / 5675 / 5704 / 9 </td>
-    </tr>
-  </tbody>
+<thead>
+
+<tr class="a">
+<th>card      </th>
+<th>sp        </th>
+<th>p         </th>
+<th>err l/m/h/std (% of actual)             </th>
+<th>time l/m/h/std (ms)                     </th>
+<th>size l/m/h/std (b)            </th></tr>
+</thead><tbody>
+
+<tr class="b">
+<td>200       </td>
+<td>4         </td>
+<td>4         </td>
+<td>0.000 / 17.000 / 180.000 / 17.470       </td>
+<td>0.070 / 0.118 / 10.092 / 0.204          </td>
+<td>218 / 220 / 220 / 1           </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>8         </td>
+<td>4         </td>
+<td>0.000 / 17.000 / 209.500 / 18.060       </td>
+<td>0.071 / 0.085 / 8.786 / 0.128           </td>
+<td>219 / 221 / 221 / 1           </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>8         </td>
+<td>8         </td>
+<td>0.000 / 3.500 / 21.500 / 3.185          </td>
+<td>0.082 / 0.085 / 8.375 / 0.119           </td>
+<td>531 / 587 / 637 / 14          </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>12        </td>
+<td>4         </td>
+<td>0.000 / 17.000 / 174.000 / 17.376       </td>
+<td>0.070 / 0.085 / 8.708 / 0.128           </td>
+<td>219 / 221 / 221 / 1           </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>12        </td>
+<td>8         </td>
+<td>0.000 / 3.000 / 25.500 / 2.847          </td>
+<td>0.086 / 0.119 / 8.984 / 0.151           </td>
+<td>353 / 372 / 627 / 74          </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>12        </td>
+<td>12        </td>
+<td>0.000 / 0.500 / 5.500 / 0.659           </td>
+<td>0.082 / 0.096 / 17.208 / 0.243          </td>
+<td>775 / 816 / 834 / 9           </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>16        </td>
+<td>4         </td>
+<td>0.000 / 17.500 / 172.000 / 16.622       </td>
+<td>0.068 / 0.075 / 9.553 / 0.137           </td>
+<td>220 / 222 / 222 / 1           </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>16        </td>
+<td>8         </td>
+<td>0.000 / 3.500 / 18.000 / 2.899          </td>
+<td>0.095 / 0.118 / 0.663 / 0.031           </td>
+<td>354 / 373 / 390 / 5           </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>16        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 1.500 / 0.276           </td>
+<td>0.085 / 0.108 / 13.977 / 0.199          </td>
+<td>802 / 833 / 852 / 6           </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>16        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 2.500 / 0.279           </td>
+<td>0.079 / 0.085 / 0.271 / 0.017           </td>
+<td>990 / 1027 / 1044 / 6         </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>20        </td>
+<td>4         </td>
+<td>0.000 / 17.000 / 209.500 / 17.424       </td>
+<td>0.068 / 0.072 / 0.250 / 0.014           </td>
+<td>221 / 223 / 223 / 1           </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>20        </td>
+<td>8         </td>
+<td>0.000 / 3.500 / 18.000 / 2.944          </td>
+<td>0.096 / 0.106 / 13.581 / 0.192          </td>
+<td>357 / 374 / 388 / 5           </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>20        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 1.000 / 0.066           </td>
+<td>0.084 / 0.088 / 1.096 / 0.019           </td>
+<td>915 / 945 / 972 / 8           </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>20        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.500 / 0.070           </td>
+<td>0.083 / 0.087 / 10.272 / 0.145          </td>
+<td>912 / 953 / 982 / 8           </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>20        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 1.000 / 0.069           </td>
+<td>0.079 / 0.081 / 0.210 / 0.017           </td>
+<td>1028 / 1056 / 1058 / 2        </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>24        </td>
+<td>4         </td>
+<td>0.000 / 17.500 / 136.000 / 16.589       </td>
+<td>0.068 / 0.071 / 0.952 / 0.025           </td>
+<td>221 / 223 / 223 / 1           </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>24        </td>
+<td>8         </td>
+<td>0.000 / 3.500 / 18.000 / 2.955          </td>
+<td>0.095 / 0.106 / 10.387 / 0.148          </td>
+<td>353 / 373 / 390 / 5           </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>24        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.500 / 0.016           </td>
+<td>0.082 / 0.083 / 0.235 / 0.014           </td>
+<td>1035 / 1048 / 1056 / 3        </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>24        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.500 / 0.012           </td>
+<td>0.082 / 0.084 / 10.904 / 0.154          </td>
+<td>1038 / 1051 / 1059 / 3        </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>24        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.500 / 0.010           </td>
+<td>0.082 / 0.085 / 0.410 / 0.019           </td>
+<td>1049 / 1064 / 1082 / 5        </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>24        </td>
+<td>24        </td>
+<td>0.000 / 0.000 / 0.500 / 0.023           </td>
+<td>0.079 / 0.084 / 11.926 / 0.168          </td>
+<td>1247 / 1264 / 1275 / 4        </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>28        </td>
+<td>4         </td>
+<td>0.000 / 17.500 / 229.000 / 17.294       </td>
+<td>0.068 / 0.072 / 0.369 / 0.015           </td>
+<td>222 / 224 / 224 / 1           </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>28        </td>
+<td>8         </td>
+<td>83.000 / 83.000 / 84.000 / 0.118        </td>
+<td>0.093 / 0.098 / 10.918 / 0.157          </td>
+<td>272 / 276 / 278 / 1           </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>28        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.500 / 0.016           </td>
+<td>0.082 / 0.083 / 10.157 / 0.143          </td>
+<td>1032 / 1067 / 1075 / 3        </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>28        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.500 / 0.007           </td>
+<td>0.082 / 0.083 / 0.617 / 0.020           </td>
+<td>1058 / 1069 / 1077 / 3        </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>28        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.500 / 0.010           </td>
+<td>0.082 / 0.085 / 11.619 / 0.165          </td>
+<td>1060 / 1071 / 1080 / 3        </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>28        </td>
+<td>24        </td>
+<td>0.000 / 0.000 / 0.500 / 0.007           </td>
+<td>0.083 / 0.085 / 0.516 / 0.022           </td>
+<td>1065 / 1083 / 1100 / 4        </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>28        </td>
+<td>28        </td>
+<td>0.000 / 0.000 / 0.500 / 0.019           </td>
+<td>0.080 / 0.087 / 11.278 / 0.160          </td>
+<td>1250 / 1267 / 1278 / 4        </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>32        </td>
+<td>4         </td>
+<td>0.000 / 18.000 / 175.500 / 17.157       </td>
+<td>0.068 / 0.078 / 0.223 / 0.019           </td>
+<td>222 / 224 / 224 / 1           </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>32        </td>
+<td>8         </td>
+<td>99.000 / 99.000 / 99.000 / 0.000        </td>
+<td>0.093 / 0.107 / 11.695 / 0.167          </td>
+<td>258 / 258 / 258 / 0           </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>32        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.500 / 0.014           </td>
+<td>0.081 / 0.084 / 8.906 / 0.126           </td>
+<td>1060 / 1071 / 1075 / 2        </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>32        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.500 / 0.010           </td>
+<td>0.082 / 0.085 / 1.458 / 0.027           </td>
+<td>1061 / 1073 / 1077 / 2        </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>32        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.500 / 0.012           </td>
+<td>0.082 / 0.084 / 10.499 / 0.148          </td>
+<td>1065 / 1074 / 1079 / 2        </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>32        </td>
+<td>24        </td>
+<td>0.000 / 0.000 / 0.500 / 0.012           </td>
+<td>0.082 / 0.085 / 0.420 / 0.020           </td>
+<td>1066 / 1075 / 1082 / 2        </td></tr>
+<tr class="b">
+<td>200       </td>
+<td>32        </td>
+<td>28        </td>
+<td>0.000 / 0.000 / 0.500 / 0.016           </td>
+<td>0.082 / 0.086 / 8.753 / 0.124           </td>
+<td>1076 / 1088 / 1102 / 4        </td></tr>
+<tr class="a">
+<td>200       </td>
+<td>32        </td>
+<td>32        </td>
+<td>0.000 / 0.000 / 0.500 / 0.010           </td>
+<td>0.080 / 0.081 / 13.248 / 0.188          </td>
+<td>1249 / 1268 / 1279 / 4        </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>4         </td>
+<td>4         </td>
+<td>0.000 / 17.500 / 175.250 / 17.530       </td>
+<td>0.136 / 0.142 / 0.591 / 0.035           </td>
+<td>218 / 220 / 220 / 0           </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>8         </td>
+<td>4         </td>
+<td>0.000 / 17.000 / 148.500 / 17.174       </td>
+<td>0.136 / 0.156 / 12.882 / 0.241          </td>
+<td>219 / 221 / 221 / 0           </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>8         </td>
+<td>8         </td>
+<td>0.000 / 3.500 / 21.250 / 3.215          </td>
+<td>0.159 / 0.186 / 10.396 / 0.151          </td>
+<td>375 / 387 / 802 / 62          </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>12        </td>
+<td>4         </td>
+<td>0.000 / 17.750 / 166.000 / 17.363       </td>
+<td>0.135 / 0.142 / 11.607 / 0.165          </td>
+<td>219 / 221 / 221 / 0           </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>12        </td>
+<td>8         </td>
+<td>0.000 / 3.250 / 23.750 / 3.133          </td>
+<td>0.166 / 0.179 / 8.713 / 0.130           </td>
+<td>376 / 387 / 401 / 3           </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>12        </td>
+<td>12        </td>
+<td>0.000 / 0.750 / 4.250 / 0.706           </td>
+<td>0.159 / 0.163 / 10.537 / 0.189          </td>
+<td>1334 / 1387 / 1434 / 14       </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>16        </td>
+<td>4         </td>
+<td>0.000 / 17.125 / 163.250 / 17.208       </td>
+<td>0.135 / 0.138 / 12.403 / 0.178          </td>
+<td>220 / 222 / 222 / 0           </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>16        </td>
+<td>8         </td>
+<td>0.000 / 3.250 / 23.500 / 3.126          </td>
+<td>0.165 / 0.179 / 9.985 / 0.147           </td>
+<td>378 / 388 / 401 / 3           </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>16        </td>
+<td>12        </td>
+<td>0.000 / 0.250 / 1.500 / 0.195           </td>
+<td>0.166 / 0.173 / 10.434 / 0.150          </td>
+<td>1414 / 1446 / 1475 / 8        </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>16        </td>
+<td>16        </td>
+<td>0.000 / 0.250 / 1.750 / 0.194           </td>
+<td>0.160 / 0.177 / 1.300 / 0.048           </td>
+<td>1761 / 1798 / 1825 / 8        </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>20        </td>
+<td>4         </td>
+<td>0.000 / 17.750 / 270.750 / 17.592       </td>
+<td>0.135 / 0.149 / 8.814 / 0.128           </td>
+<td>221 / 223 / 223 / 0           </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>20        </td>
+<td>8         </td>
+<td>0.000 / 3.500 / 23.000 / 3.103          </td>
+<td>0.165 / 0.183 / 9.271 / 0.139           </td>
+<td>379 / 389 / 398 / 3           </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>20        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.500 / 0.066           </td>
+<td>0.164 / 0.182 / 11.093 / 0.162          </td>
+<td>1617 / 1652 / 1683 / 10       </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>20        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.500 / 0.070           </td>
+<td>0.166 / 0.170 / 8.676 / 0.128           </td>
+<td>1624 / 1665 / 1706 / 10       </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>20        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.750 / 0.069           </td>
+<td>0.159 / 0.162 / 12.846 / 0.183          </td>
+<td>1840 / 1853 / 1856 / 2        </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>24        </td>
+<td>4         </td>
+<td>0.000 / 16.750 / 174.750 / 16.786       </td>
+<td>0.135 / 0.145 / 8.586 / 0.128           </td>
+<td>221 / 223 / 223 / 0           </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>24        </td>
+<td>8         </td>
+<td>0.000 / 3.500 / 24.500 / 3.090          </td>
+<td>0.165 / 0.179 / 11.344 / 0.164          </td>
+<td>377 / 389 / 399 / 3           </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>24        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.250 / 0.016           </td>
+<td>0.164 / 0.183 / 10.708 / 0.159          </td>
+<td>1828 / 1841 / 1852 / 4        </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>24        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.250 / 0.017           </td>
+<td>0.164 / 0.172 / 11.180 / 0.161          </td>
+<td>1830 / 1844 / 1855 / 4        </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>24        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.250 / 0.015           </td>
+<td>0.166 / 0.170 / 9.589 / 0.140           </td>
+<td>1843 / 1868 / 1890 / 6        </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>24        </td>
+<td>24        </td>
+<td>0.000 / 0.000 / 0.250 / 0.022           </td>
+<td>0.160 / 0.162 / 9.137 / 0.134           </td>
+<td>2213 / 2232 / 2247 / 5        </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>28        </td>
+<td>4         </td>
+<td>0.000 / 18.250 / 164.000 / 16.587       </td>
+<td>0.135 / 0.144 / 14.520 / 0.209          </td>
+<td>222 / 224 / 224 / 0           </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>28        </td>
+<td>8         </td>
+<td>33.000 / 47.750 / 56.750 / 3.588        </td>
+<td>0.164 / 0.168 / 1.305 / 0.052           </td>
+<td>355 / 375 / 394 / 5           </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>28        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.250 / 0.006           </td>
+<td>0.165 / 0.178 / 11.264 / 0.167          </td>
+<td>1837 / 1848 / 1854 / 3        </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>28        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.250 / 0.014           </td>
+<td>0.164 / 0.167 / 8.689 / 0.130           </td>
+<td>1838 / 1850 / 1857 / 3        </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>28        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.250 / 0.011           </td>
+<td>0.164 / 0.167 / 10.679 / 0.159          </td>
+<td>1839 / 1852 / 1863 / 3        </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>28        </td>
+<td>24        </td>
+<td>0.000 / 0.000 / 0.250 / 0.011           </td>
+<td>0.166 / 0.175 / 12.887 / 0.190          </td>
+<td>1857 / 1875 / 1896 / 5        </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>28        </td>
+<td>28        </td>
+<td>0.000 / 0.000 / 0.250 / 0.015           </td>
+<td>0.160 / 0.196 / 1.478 / 0.071           </td>
+<td>2217 / 2235 / 2252 / 5        </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>32        </td>
+<td>4         </td>
+<td>0.000 / 17.500 / 183.750 / 19.308       </td>
+<td>0.135 / 0.142 / 9.949 / 0.148           </td>
+<td>222 / 224 / 224 / 0           </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>32        </td>
+<td>8         </td>
+<td>46.500 / 56.250 / 63.250 / 2.194        </td>
+<td>0.164 / 0.181 / 1.209 / 0.063           </td>
+<td>349 / 370 / 390 / 5           </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>32        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.250 / 0.012           </td>
+<td>0.164 / 0.171 / 11.529 / 0.170          </td>
+<td>1837 / 1848 / 1854 / 3        </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>32        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.250 / 0.016           </td>
+<td>0.164 / 0.167 / 10.276 / 0.154          </td>
+<td>1836 / 1850 / 1856 / 3        </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>32        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.250 / 0.015           </td>
+<td>0.164 / 0.184 / 11.611 / 0.227          </td>
+<td>1839 / 1851 / 1858 / 3        </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>32        </td>
+<td>24        </td>
+<td>0.000 / 0.000 / 0.250 / 0.012           </td>
+<td>0.165 / 0.171 / 9.313 / 0.143           </td>
+<td>1842 / 1853 / 1862 / 3        </td></tr>
+<tr class="b">
+<td>400       </td>
+<td>32        </td>
+<td>28        </td>
+<td>0.000 / 0.000 / 0.250 / 0.013           </td>
+<td>0.166 / 0.180 / 10.708 / 0.162          </td>
+<td>1860 / 1877 / 1900 / 5        </td></tr>
+<tr class="a">
+<td>400       </td>
+<td>32        </td>
+<td>32        </td>
+<td>0.000 / 0.000 / 0.250 / 0.014           </td>
+<td>0.160 / 0.166 / 1.205 / 0.057           </td>
+<td>2217 / 2235 / 2249 / 5        </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>4         </td>
+<td>4         </td>
+<td>0.000 / 17.333 / 219.667 / 17.222       </td>
+<td>0.202 / 0.240 / 13.390 / 0.237          </td>
+<td>218 / 220 / 220 / 0           </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>8         </td>
+<td>4         </td>
+<td>0.000 / 17.417 / 231.333 / 17.340       </td>
+<td>0.202 / 0.224 / 9.662 / 0.150           </td>
+<td>220 / 221 / 221 / 0           </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>8         </td>
+<td>8         </td>
+<td>0.000 / 3.500 / 24.333 / 3.240          </td>
+<td>0.234 / 0.271 / 11.443 / 0.216          </td>
+<td>382 / 392 / 403 / 3           </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>12        </td>
+<td>4         </td>
+<td>0.000 / 17.167 / 188.167 / 17.324       </td>
+<td>0.202 / 0.210 / 1.320 / 0.055           </td>
+<td>219 / 221 / 221 / 0           </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>12        </td>
+<td>8         </td>
+<td>0.000 / 3.500 / 23.833 / 3.243          </td>
+<td>0.235 / 0.246 / 10.821 / 0.164          </td>
+<td>382 / 392 / 403 / 3           </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>12        </td>
+<td>12        </td>
+<td>0.000 / 0.667 / 4.500 / 0.694           </td>
+<td>0.240 / 0.259 / 12.004 / 0.183          </td>
+<td>2015 / 2101 / 2178 / 21       </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>16        </td>
+<td>4         </td>
+<td>0.000 / 17.500 / 170.000 / 17.882       </td>
+<td>0.202 / 0.218 / 9.208 / 0.140           </td>
+<td>221 / 222 / 222 / 0           </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>16        </td>
+<td>8         </td>
+<td>0.000 / 3.667 / 22.667 / 3.276          </td>
+<td>0.234 / 0.251 / 12.301 / 0.237          </td>
+<td>383 / 393 / 408 / 3           </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>16        </td>
+<td>12        </td>
+<td>0.000 / 0.167 / 1.000 / 0.170           </td>
+<td>0.250 / 0.284 / 12.000 / 0.180          </td>
+<td>2199 / 2243 / 2282 / 12       </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>16        </td>
+<td>16        </td>
+<td>0.000 / 0.167 / 1.167 / 0.171           </td>
+<td>0.239 / 0.291 / 21.163 / 0.333          </td>
+<td>2811 / 2859 / 2901 / 13       </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>20        </td>
+<td>4         </td>
+<td>0.000 / 17.667 / 176.667 / 18.162       </td>
+<td>0.202 / 0.223 / 16.405 / 0.275          </td>
+<td>221 / 223 / 223 / 0           </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>20        </td>
+<td>8         </td>
+<td>0.000 / 3.500 / 20.833 / 3.180          </td>
+<td>0.234 / 0.243 / 5.621 / 0.104           </td>
+<td>385 / 394 / 405 / 3           </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>20        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.500 / 0.066           </td>
+<td>0.246 / 0.281 / 19.279 / 0.390          </td>
+<td>2541 / 2604 / 2663 / 15       </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>20        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.667 / 0.071           </td>
+<td>0.248 / 0.258 / 24.627 / 0.387          </td>
+<td>2569 / 2627 / 2682 / 15       </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>20        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.500 / 0.069           </td>
+<td>0.239 / 0.246 / 13.606 / 0.248          </td>
+<td>2938 / 2954 / 2972 / 3        </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>24        </td>
+<td>4         </td>
+<td>0.000 / 17.333 / 210.333 / 18.157       </td>
+<td>0.202 / 0.218 / 11.991 / 0.180          </td>
+<td>222 / 223 / 223 / 0           </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>24        </td>
+<td>8         </td>
+<td>0.000 / 3.667 / 21.167 / 3.184          </td>
+<td>0.234 / 0.258 / 11.130 / 0.238          </td>
+<td>385 / 394 / 406 / 3           </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>24        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.167 / 0.017           </td>
+<td>0.246 / 0.272 / 12.032 / 0.191          </td>
+<td>2912 / 2935 / 2954 / 5        </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>24        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.167 / 0.019           </td>
+<td>0.246 / 0.268 / 19.358 / 0.282          </td>
+<td>2916 / 2940 / 2959 / 6        </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>24        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.167 / 0.015           </td>
+<td>0.248 / 0.259 / 20.293 / 0.343          </td>
+<td>2948 / 2980 / 3009 / 9        </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>24        </td>
+<td>24        </td>
+<td>0.000 / 0.000 / 0.167 / 0.017           </td>
+<td>0.239 / 0.252 / 11.721 / 0.230          </td>
+<td>3586 / 3618 / 3643 / 7        </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>28        </td>
+<td>4         </td>
+<td>0.000 / 17.667 / 182.667 / 17.309       </td>
+<td>0.202 / 0.210 / 11.957 / 0.177          </td>
+<td>223 / 224 / 224 / 0           </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>28        </td>
+<td>8         </td>
+<td>10.333 / 28.000 / 42.833 / 3.687        </td>
+<td>0.233 / 0.238 / 10.166 / 0.210          </td>
+<td>377 / 392 / 403 / 3           </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>28        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.333 / 0.013           </td>
+<td>0.246 / 0.252 / 9.907 / 0.158           </td>
+<td>3005 / 3024 / 3041 / 5        </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>28        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.167 / 0.012           </td>
+<td>0.246 / 0.256 / 18.673 / 0.273          </td>
+<td>3007 / 3026 / 3045 / 5        </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>28        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.167 / 0.011           </td>
+<td>0.246 / 0.283 / 17.762 / 0.355          </td>
+<td>3005 / 3029 / 3047 / 6        </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>28        </td>
+<td>24        </td>
+<td>0.000 / 0.000 / 0.167 / 0.011           </td>
+<td>0.248 / 0.254 / 8.821 / 0.138           </td>
+<td>3035 / 3065 / 3098 / 8        </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>28        </td>
+<td>28        </td>
+<td>0.000 / 0.000 / 0.167 / 0.011           </td>
+<td>0.239 / 0.242 / 12.171 / 0.177          </td>
+<td>3589 / 3621 / 3646 / 7        </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>32        </td>
+<td>4         </td>
+<td>0.000 / 17.333 / 200.000 / 18.294       </td>
+<td>0.202 / 0.205 / 9.036 / 0.138           </td>
+<td>223 / 224 / 224 / 0           </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>32        </td>
+<td>8         </td>
+<td>25.167 / 37.167 / 46.833 / 3.185        </td>
+<td>0.232 / 0.236 / 9.061 / 0.144           </td>
+<td>377 / 389 / 403 / 3           </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>32        </td>
+<td>12        </td>
+<td>0.000 / 0.000 / 0.167 / 0.013           </td>
+<td>0.246 / 0.256 / 11.119 / 0.169          </td>
+<td>3028 / 3043 / 3055 / 4        </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>32        </td>
+<td>16        </td>
+<td>0.000 / 0.000 / 0.167 / 0.010           </td>
+<td>0.246 / 0.257 / 11.557 / 0.213          </td>
+<td>3030 / 3045 / 3057 / 4        </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>32        </td>
+<td>20        </td>
+<td>0.000 / 0.000 / 0.167 / 0.011           </td>
+<td>0.246 / 0.266 / 20.353 / 0.295          </td>
+<td>3030 / 3047 / 3071 / 4        </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>32        </td>
+<td>24        </td>
+<td>0.000 / 0.000 / 0.167 / 0.012           </td>
+<td>0.246 / 0.270 / 11.461 / 0.183          </td>
+<td>3035 / 3050 / 3073 / 4        </td></tr>
+<tr class="b">
+<td>600       </td>
+<td>32        </td>
+<td>28        </td>
+<td>0.000 / 0.000 / 0.167 / 0.013           </td>
+<td>0.248 / 0.260 / 11.755 / 0.242          </td>
+<td>3060 / 3085 / 3108 / 7        </td></tr>
+<tr class="a">
+<td>600       </td>
+<td>32        </td>
+<td>32        </td>
+<td>0.000 / 0.000 / 0.167 / 0.012           </td>
+<td>0.239 / 0.248 / 10.380 / 0.157          </td>
+<td>3593 / 3621 / 3645 / 7        </td></tr>
+<tr class="b">
+<td>800       </td>
+<td>4         </td>
+<td>4         </td>
+<td>0.000 / 17.250 / 188.250 / 17.248       </td>
+<td>0.269 / 0.280 / 10.818 / 0.212          </td>
+<td>219 / 220 / 220 / 0           </td></tr>
+<tr class="a">
+<td>800       </td>
+<td>8         </td>
+<td>4         </td>
+<td>0.000 / 17.875 / 184.125 / 17.721       </td>
+<td>0.270 / 0.298 / 16.706 / 0.252          </td>
+<td>220 / 221 / 221 / 0           </td></tr>
+<tr class="b">
+<td>800       </td>
+<td>8         </td>
+<td>8         </td>
+<td>0.000 / 3.625 / 22.000 / 3.420          </td>
+<td>0.303 / 0.315 / 12.489 / 0.243          </td>
+<td>386 / 395 / 406 / 3           </td></tr>
+<tr class="a">
+<td>800       </td>
+<td>12        </td>
+<td>4         </td>
+<td>0.000 / 17.750 / 196.500 / 16.870       </td>
+<td>0.269 / 0.289 / 13.533 / 0.235          </td>
+<td>220 / 221 / 221 / 0           </td></tr>
+<tr class="b">
+<td>800       </td>
+<td>12        </td>
+<td>8         </td>
+<td>0.000 / 3.750 / 25.875 / 3.359          </td>
+<td>0.303 / 0.330 / 18.306 / 0.365          </td>


<TRUNCATED>

[34/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-analytics/metron-statistics/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-analytics/metron-statistics/index.html b/site/current-book/metron-analytics/metron-statistics/index.html
index 29084e8..0c3515d 100644
--- a/site/current-book/metron-analytics/metron-statistics/index.html
+++ b/site/current-book/metron-analytics/metron-statistics/index.html
@@ -1,756 +1,599 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-analytics/metron-statistics/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Statistics and Mathematical Functions</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Statistics and Mathematical Functions</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Statistics and Mathematical Functions</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                                
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-down"></i>
-        Analytics</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service">
-          <i class="none"></i>
-        Maas-service</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-profiler/index.html" title="Profiler">
-          <i class="none"></i>
-        Profiler</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client">
-          <i class="none"></i>
-        Profiler-client</a>
-            </li>
-                                                                            
-      <li class="active">
-    
-            <a href="#"><i class="icon-chevron-down"></i>Statistics</a>
-                  <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../metron-analytics/metron-statistics/HLLP.html" title="HLLP">
-          <i class="none"></i>
-        HLLP</a>
-            </li>
-              </ul>
-        </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-down"></span>Analytics</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-analytics/metron-maas-service/index.html" title="Maas-service"><span class="none"></span>Maas-service</a></li>
+    <li><a href="../../metron-analytics/metron-profiler/index.html" title="Profiler"><span class="none"></span>Profiler</a></li>
+    <li><a href="../../metron-analytics/metron-profiler-client/index.html" title="Profiler-client"><span class="none"></span>Profiler-client</a></li>
+    <li class="active"><a href="#"><span class="icon-chevron-down"></span>Statistics</a>
+    <ul class="nav nav-list">
+    <li><a href="../../metron-analytics/metron-statistics/HLLP.html" title="HLLP"><span class="none"></span>HLLP</a></li>
+    </ul>
+</li>
+    </ul>
+</li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Statistics and Mathematical Functions</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Statistics and Mathematical Functions</h1>
 <p><a name="Statistics_and_Mathematical_Functions"></a></p>
-<p>A variety of non-trivial and advanced analytics make use of statistics and advanced mathematical functions. Particular, capturing the statistical snapshots in a scalable way can open up doors for more advanced analytics such as outlier analysis. As such, this project is aimed at capturing a robust set of statistical functions and statistical-based algorithms in the form of Stellar functions. These functions can be used from everywhere where Stellar is used.</p>
+<p>A variety of non-trivial and advanced analytics make use of statistics and advanced mathematical functions.  Particular, capturing the statistical snapshots in a scalable way can open up doors for more advanced analytics such as outlier analysis.  As such, this project is aimed at capturing a robust set of statistical functions and statistical-based algorithms in the form of Stellar functions.  These functions can be used from everywhere where Stellar is used.</p>
 <div class="section">
 <h2><a name="Stellar_Functions"></a>Stellar Functions</h2>
 <div class="section">
 <h3><a name="Approximation_Statistics"></a>Approximation Statistics</h3>
 <div class="section">
 <h4><a name="HLLP_ADD"></a><tt>HLLP_ADD</tt></h4>
-
 <ul>
-  
+
 <li>Description: Add value to the HyperLogLogPlus estimator set. See <a href="HLLP.html">HLLP README</a></li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>hyperLogLogPlus - the hllp estimator to add a value to</li>
-    
 <li>value+ - value to add to the set. Takes a single item or a list.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The HyperLogLogPlus set with a new value added</li>
 </ul></div>
 <div class="section">
 <h4><a name="HLLP_CARDINALITY"></a><tt>HLLP_CARDINALITY</tt></h4>
-
 <ul>
-  
+
 <li>Description: Returns HyperLogLogPlus-estimated cardinality for this set. See <a href="HLLP.html">HLLP README</a></li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>hyperLogLogPlus - the hllp set</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: Long value representing the cardinality for this set. Cardinality of a null set is 0.</li>
 </ul></div>
 <div class="section">
 <h4><a name="HLLP_INIT"></a><tt>HLLP_INIT</tt></h4>
-
 <ul>
-  
+
 <li>Description: Initializes the HyperLogLogPlus estimator set. p must be a value between 4 and sp and sp must be less than 32 and greater than 4. See <a href="HLLP.html">HLLP README</a></li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>p - the precision value for the normal set</li>
-    
 <li>sp - the precision value for the sparse set. If p is set, but sp is 0 or not specified, the sparse set will be disabled.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: A new HyperLogLogPlus set</li>
 </ul></div>
 <div class="section">
 <h4><a name="HLLP_MERGE"></a><tt>HLLP_MERGE</tt></h4>
-
 <ul>
-  
+
 <li>Description: Merge hllp sets together. The resulting estimator is initialized with p and sp precision values from the first provided hllp estimator set. See <a href="HLLP.html">HLLP README</a></li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>hllp - List of hllp estimators to merge. Takes a single hllp set or a list.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: A new merged HyperLogLogPlus estimator set. Passing an empty list returns null.</li>
 </ul></div></div>
 <div class="section">
 <h3><a name="Mathematical_Functions"></a>Mathematical Functions</h3>
 <div class="section">
 <h4><a name="ABS"></a><tt>ABS</tt></h4>
-
 <ul>
-  
+
 <li>Description: Returns the absolute value of a number.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>number - The number to take the absolute value of</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The absolute value of the number passed in.</li>
 </ul></div>
 <div class="section">
 <h4><a name="BIN"></a><tt>BIN</tt></h4>
-
 <ul>
-  
+
 <li>Description: Computes the bin that the value is in given a set of bounds.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>value - The value to bin</li>
-    
 <li>bounds - A list of value bounds (excluding min and max) in sorted order.</li>
-  </ul></li>
-  
-<li>Returns: Which bin N the value falls in such that bound(N-1) &lt; value &lt;= bound(N). No min and max bounds are provided, so values smaller than the 0&#x2019;th bound go in the 0&#x2019;th bin, and values greater than the last bound go in the M&#x2019;th bin.</li>
+</ul>
+</li>
+<li>Returns: Which bin N the value falls in such that bound(N-1) &lt; value &lt;= bound(N).  No min and max bounds are provided, so values smaller than the 0&#x2019;th bound go in the 0&#x2019;th bin, and values greater than the last bound go in the M&#x2019;th bin.</li>
+</ul></div></div>
+<div class="section">
+<h3><a name="Sampling_Functions"></a>Sampling Functions</h3>
+<div class="section">
+<h4><a name="SAMPLE_ADD"></a><tt>SAMPLE_ADD</tt></h4>
+<ul>
+
+<li>Description: Add a value or collection of values to a sampler.</li>
+<li>Input:
+<ul>
+
+<li>sampler - Sampler to use.  If null, then a default Uniform sampler is created.</li>
+<li>o - The value to add.  If o is an Iterable, then each item is added.</li>
+</ul>
+</li>
+<li>Returns: The sampler.</li>
+</ul></div>
+<div class="section">
+<h4><a name="SAMPLE_GET"></a><tt>SAMPLE_GET</tt></h4>
+<ul>
+
+<li>Description: Return the sample.</li>
+<li>Input:
+<ul>
+
+<li>sampler - Sampler to use.</li>
+</ul>
+</li>
+<li>Returns: The resulting sample.</li>
+</ul></div>
+<div class="section">
+<h4><a name="SAMPLE_INIT"></a><tt>SAMPLE_INIT</tt></h4>
+<ul>
+
+<li>Description: Create a <a class="externalLink" href="https://en.wikipedia.org/wiki/Reservoir_sampling">reservoir sampler</a> of a specific size or, if unspecified, size 1024.  Elements sampled by the reservoir sampler will be included in the final sample with equal probability.</li>
+<li>Input:
+<ul>
+
+<li>size? - The size of the reservoir sampler.  If unspecified, the size is 1024.</li>
+</ul>
+</li>
+<li>Returns: The sampler object.</li>
+</ul></div>
+<div class="section">
+<h4><a name="SAMPLE_MERGE"></a><tt>SAMPLE_MERGE</tt></h4>
+<ul>
+
+<li>Description: Merge and resample a collection of samples.</li>
+<li>Input:
+<ul>
+
+<li>samplers - A list of samplers to merge.</li>
+</ul>
+</li>
+<li>Returns: A sampler which represents the resampled merger of the samplers.</li>
 </ul></div></div>
 <div class="section">
 <h3><a name="Distributional_Statistics"></a>Distributional Statistics</h3>
 <div class="section">
 <h4><a name="STATS_ADD"></a><tt>STATS_ADD</tt></h4>
-
 <ul>
-  
+
 <li>Description: Adds one or more input values to those that are used to calculate the summary statistics.</li>
-  
 <li>Input:
-  
 <ul>
-    
-<li>stats - The Stellar statistics object. If null, then a new one is initialized.</li>
-    
+
+<li>stats - The Stellar statistics object.  If null, then a new one is initialized.</li>
 <li>value+ - One or more items to add. Each item may be a number or a list of numbers. If an item is a list, each number in the list will be added.</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: A Stellar statistics object</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_BIN"></a><tt>STATS_BIN</tt></h4>
-
 <ul>
-  
+
 <li>Description: Computes the bin that the value is in based on the statistical distribution.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-    
 <li>value - The value to bin</li>
-    
-<li>bounds? - A list of percentile bin bounds (excluding min and max) or a string representing a known and common set of bins. For convenience, we have provided QUARTILE, QUINTILE, and DECILE which you can pass in as a string arg. If this argument is omitted, then we assume a Quartile bin split.</li>
-  </ul></li>
-  
+<li>bounds? - A list of percentile bin bounds (excluding min and max) or a string representing a known and common set of bins.  For convenience, we have provided QUARTILE, QUINTILE, and DECILE which you can pass in as a string arg. If this argument is omitted, then we assume a Quartile bin split.</li>
+</ul>
+</li>
 <li>Returns: &quot;Which bin N the value falls in such that bound(N-1) &lt; value &lt;= bound(N). No min and max bounds are provided, so values smaller than the 0&#x2019;th bound go in the 0&#x2019;th bin, and values greater than the last bound go in the M&#x2019;th bin.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_COUNT"></a><tt>STATS_COUNT</tt></h4>
-
 <ul>
-  
+
 <li>Description: Calculates the count of the values accumulated (or in the window if a window is used).</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The count of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_GEOMETRIC_MEAN"></a><tt>STATS_GEOMETRIC_MEAN</tt></h4>
-
 <ul>
-  
+
 <li>Description: Calculates the geometric mean of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The geometric mean of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_INIT"></a><tt>STATS_INIT</tt></h4>
-
 <ul>
-  
+
 <li>Description: Initializes a statistics object</li>
-  
 <li>Input:
-  
 <ul>
-    
-<li>window_size - The number of input data values to maintain in a rolling window in memory. If window_size is equal to 0, then no rolling window is maintained. Using no rolling window is less memory intensive, but cannot calculate certain statistics like percentiles and kurtosis.</li>
-  </ul></li>
-  
+
+<li>window_size - The number of input data values to maintain in a rolling window in memory.  If window_size is equal to 0, then no rolling window is maintained. Using no rolling window is less memory intensive, but cannot calculate certain statistics like percentiles and kurtosis.</li>
+</ul>
+</li>
 <li>Returns: A Stellar statistics object</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_KURTOSIS"></a><tt>STATS_KURTOSIS</tt></h4>
-
 <ul>
-  
-<li>Description: Calculates the kurtosis of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
-  
+
+<li>Description: Calculates the kurtosis of the accumulated values (or in the window if a window is used).  See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The kurtosis of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_MAX"></a><tt>STATS_MAX</tt></h4>
-
 <ul>
-  
+
 <li>Description: Calculates the maximum of the accumulated values (or in the window if a window is used).</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The maximum of the accumulated values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_MEAN"></a><tt>STATS_MEAN</tt></h4>
-
 <ul>
-  
+
 <li>Description: Calculates the mean of the accumulated values (or in the window if a window is used).</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The mean of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_MERGE"></a><tt>STATS_MERGE</tt></h4>
-
 <ul>
-  
+
 <li>Description: Merges statistics objects.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>statistics - A list of statistics objects</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: A Stellar statistics object</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_MIN"></a><tt>STATS_MIN</tt></h4>
-
 <ul>
-  
+
 <li>Description: Calculates the minimum of the accumulated values (or in the window if a window is used).</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The minimum of the accumulated values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_PERCENTILE"></a><tt>STATS_PERCENTILE</tt></h4>
-
 <ul>
-  
+
 <li>Description: Computes the p&#x2019;th percentile of the accumulated values (or in the window if a window is used).</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-    
 <li>p - a double where 0 &lt;= p &lt; 1 representing the percentile</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The p&#x2019;th percentile of the data or NaN if the statistics object is null</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_POPULATION_VARIANCE"></a><tt>STATS_POPULATION_VARIANCE</tt></h4>
-
 <ul>
-  
-<li>Description: Calculates the population variance of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
-  
+
+<li>Description: Calculates the population variance of the accumulated values (or in the window if a window is used).  See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The population variance of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_QUADRATIC_MEAN"></a><tt>STATS_QUADRATIC_MEAN</tt></h4>
-
 <ul>
-  
-<li>Description: Calculates the quadratic mean of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
-  
+
+<li>Description: Calculates the quadratic mean of the accumulated values (or in the window if a window is used).  See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The quadratic mean of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_SD"></a><tt>STATS_SD</tt></h4>
-
 <ul>
-  
-<li>Description: Calculates the standard deviation of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
-  
+
+<li>Description: Calculates the standard deviation of the accumulated values (or in the window if a window is used).  See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The standard deviation of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_SKEWNESS"></a><tt>STATS_SKEWNESS</tt></h4>
-
 <ul>
-  
-<li>Description: Calculates the skewness of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
-  
+
+<li>Description: Calculates the skewness of the accumulated values (or in the window if a window is used).  See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The skewness of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_SUM"></a><tt>STATS_SUM</tt></h4>
-
 <ul>
-  
+
 <li>Description: Calculates the sum of the accumulated values (or in the window if a window is used).</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The sum of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_SUM_LOGS"></a><tt>STATS_SUM_LOGS</tt></h4>
-
 <ul>
-  
-<li>Description: Calculates the sum of the (natural) log of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
-  
+
+<li>Description: Calculates the sum of the (natural) log of the accumulated values (or in the window if a window is used).  See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The sum of the (natural) log of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_SUM_SQUARES"></a><tt>STATS_SUM_SQUARES</tt></h4>
-
 <ul>
-  
+
 <li>Description: Calculates the sum of the squares of the accumulated values (or in the window if a window is used).</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The sum of the squares of the values in the window or NaN if the statistics object is null.</li>
 </ul></div>
 <div class="section">
 <h4><a name="STATS_VARIANCE"></a><tt>STATS_VARIANCE</tt></h4>
-
 <ul>
-  
-<li>Description: Calculates the variance of the accumulated values (or in the window if a window is used). See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
-  
+
+<li>Description: Calculates the variance of the accumulated values (or in the window if a window is used).  See <a class="externalLink" href="http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics">http://commons.apache.org/proper/commons-math/userguide/stat.html#a1.2_Descriptive_statistics</a></li>
 <li>Input:
-  
 <ul>
-    
+
 <li>stats - The Stellar statistics object</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The variance of the values in the window or NaN if the statistics object is null.</li>
 </ul></div></div>
 <div class="section">
+<h3><a name="Information_Theory_Functions"></a>Information Theory Functions</h3>
+<div class="section">
+<h4><a name="IT_ENTROPY"></a><tt>IT_ENTROPY</tt></h4>
+<ul>
+
+<li>Description: Computes the base-2 entropy of a multiset</li>
+<li>Input:
+<ul>
+
+<li>input - a multiset (a map of objects to counts).</li>
+</ul>
+</li>
+<li>Returns: The <a class="externalLink" href="https://en.wikipedia.org/wiki/Entropy_(information_theory)#Definition">base-2 entropy</a> of the count .  The unit of this is bits.</li>
+</ul></div></div>
+<div class="section">
 <h3><a name="Statistical_Outlier_Detection"></a>Statistical Outlier Detection</h3>
 <div class="section">
 <h4><a name="OUTLIER_MAD_STATE_MERGE"></a><tt>OUTLIER_MAD_STATE_MERGE</tt></h4>
-
 <ul>
-  
+
 <li>Description: Update the statistical state required to compute the Median Absolute Deviation.</li>
-  
 <li>Input:
-  
 <ul>
-    
-<li>[state] - A list of Median Absolute Deviation States to merge. Generally these are states across time.</li>
-    
+
+<li>[state] - A list of Median Absolute Deviation States to merge.  Generally these are states across time.</li>
 <li>currentState? - The current state (optional)</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The Median Absolute Deviation state</li>
 </ul></div>
 <div class="section">
 <h4><a name="OUTLIER_MAD_ADD"></a><tt>OUTLIER_MAD_ADD</tt></h4>
-
 <ul>
-  
+
 <li>Description: Add a piece of data to the state.</li>
-  
 <li>Input:
-  
 <ul>
-    
+
 <li>state - The MAD state</li>
-    
 <li>value - The numeric value to add</li>
-  </ul></li>
-  
+</ul>
+</li>
 <li>Returns: The MAD state</li>
 </ul></div>
 <div class="section">
 <h4><a name="OUTLIER_MAD_SCORE"></a><tt>OUTLIER_MAD_SCORE</tt></h4>
-
 <ul>
-  
-<li>Description: Get the modified z-score normalized by the MAD: scale * | x_i - median(X) | / MAD. See the first page of <a class="externalLink" href="http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf">http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf</a></li>
-  
+
+<li>Description: Get the modified z-score normalized by the MAD: scale * | x_i - median(X) | / MAD.  See the first page of <a class="externalLink" href="http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf">http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf</a></li>
 <li>Input:
-  
 <ul>
-    
+
 <li>state - The MAD state</li>
-    
 <li>value - The numeric value to score</li>
-    
-<li>scale? - Optionally the scale to use when computing the modified z-score. Default is <tt>0.6745</tt>, see the first page of <a class="externalLink" href="http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf">http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf</a></li>
-  </ul></li>
-  
+<li>scale? - Optionally the scale to use when computing the modified z-score.  Default is <tt>0.6745</tt>, see the first page of <a class="externalLink" href="http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf">http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf</a></li>
+</ul>
+</li>
 <li>Returns: The modified z-score</li>
 </ul>
 <p><a name="Outlier_Analysis"></a></p>
 <h1>Outlier Analysis</h1>
-<p>A common desire is to find anomalies in numerical data. To that end, we have some simple statistical anomaly detectors.</p></div></div></div>
+<p>A common desire is to find anomalies in numerical data.  To that end, we have some simple statistical anomaly detectors.</p></div></div></div>
 <div class="section">
 <h2><a name="Median_Absolute_Deviation"></a>Median Absolute Deviation</h2>
-<p>Much has been written about this robust estimator. See the first page of <a class="externalLink" href="http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf">http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf</a> for a good coverage of the good and the bad of MAD. The usage, however is fairly straightforward:</p>
-
+<p>Much has been written about this robust estimator.  See the first page of <a class="externalLink" href="http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf">http://web.ipac.caltech.edu/staff/fmasci/home/astro_refs/BetterThanMAD.pdf</a> for a good coverage of the good and the bad of MAD.  The usage, however is fairly straightforward:</p>
 <ul>
-  
+
 <li>Gather the statistical state required to compute the MAD
-  
 <ul>
-    
+
 <li>The distribution of the values of a univariate random variable over time.</li>
-    
 <li>The distribution of the absolute deviations of the values from the median.</li>
-  </ul></li>
-  
-<li>Use this statistical state to score unseen values. The higher the score, the more unlike the previously seen data the value is.</li>
 </ul>
-<p>There are a couple of issues which make MAD a bit hard to compute. First, the statistical state requires computing median, which can be computationally expensive to compute exactly. To get around this, we use the OnlineStatisticalProvider to compute a sketch rather than the exact median. Secondly, the statistical state for seasonal data should be limited to a fixed, trailing window. We do this by ensuring that the MAD state is mergeable and able to be queried from within the Profiler.</p>
+</li>
+<li>Use this statistical state to score unseen values.  The higher the score, the more unlike the previously seen data the value is.</li>
+</ul>
+<p>There are a couple of issues which make MAD a bit hard to compute. First, the statistical state requires computing median, which can be computationally expensive to compute exactly.  To get around this, we use the OnlineStatisticalProvider to compute a sketch rather than the exact median.  Secondly, the statistical state for seasonal data should be limited to a fixed, trailing window.  We do this by ensuring that the MAD state is mergeable and able to be queried from within the Profiler.</p>
 <div class="section">
 <h3><a name="Example"></a>Example</h3>
 <p>We will create a dummy data stream of gaussian noise to illustrate how to use the MAD functionality along with the profiler to tag messages as outliers or not.</p>
-<p>To do this, we will create a </p>
-
+<p>To do this, we will create a</p>
 <ul>
-  
+
 <li>data generator</li>
-  
 <li>parser</li>
-  
 <li>profiler profile</li>
-  
 <li>enrichment and threat triage</li>
 </ul>
 <div class="section">
 <h4><a name="Data_Generator"></a>Data Generator</h4>
 <p>We can create a simple python script to generate a stream of gaussian noise at the frequency of one message per second as a python script which should be saved at <tt>~/rand_gen.py</tt>:</p>
 
-<div class="source">
-<div class="source">
-<pre>#!/usr/bin/python
+<div>
+<div>
+<pre class="source">#!/usr/bin/python
 import random
 import sys
 import time
@@ -766,21 +609,19 @@ def main():
 if __name__ == '__main__':
   main()
 </pre></div></div>
-<p>This script will take the following as arguments:</p>
 
+<p>This script will take the following as arguments:</p>
 <ul>
-  
+
 <li>The mean of the data generated</li>
-  
 <li>The standard deviation of the data generated</li>
-  
 <li>The frequency (in seconds) of the data generated</li>
 </ul>
 <p>If, however, you&#x2019;d like to test a longer tailed distribution, like the student t-distribution and have numpy installed, you can use the following as <tt>~/rand_gen.py</tt>:</p>
 
-<div class="source">
-<div class="source">
-<pre>#!/usr/bin/python
+<div>
+<div>
+<pre class="source">#!/usr/bin/python
 import random
 import sys
 import time
@@ -797,12 +638,11 @@ def main():
 if __name__ == '__main__':
   main()
 </pre></div></div>
-<p>This script will take the following as arguments:</p>
 
+<p>This script will take the following as arguments:</p>
 <ul>
-  
+
 <li>The degrees of freedom for the distribution</li>
-  
 <li>The frequency (in seconds) of the data generated</li>
 </ul></div>
 <div class="section">
@@ -810,9 +650,9 @@ if __name__ == '__main__':
 <p>We will create a parser that will take the single numbers in and create a message with a field called <tt>value</tt> in them using the <tt>CSVParser</tt>.</p>
 <p>Add the following file to <tt>$METRON_HOME/config/zookeeper/parsers/mad.json</tt>:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;parserClassName&quot; : &quot;org.apache.metron.parsers.csv.CSVParser&quot;
  ,&quot;sensorTopic&quot; : &quot;mad&quot;
  ,&quot;parserConfig&quot; : {
@@ -830,15 +670,16 @@ if __name__ == '__main__':
     }
                            ]
 }
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="Enrichment_and_Threat_Intel"></a>Enrichment and Threat Intel</h4>
-<p>We will set a threat triage level of <tt>10</tt> if a message generates a outlier score of more than 3.5. This cutoff will depend on your data and should be adjusted based on the assumed underlying distribution. Note that under the assumptions of normality, MAD will act as a robust estimator of the standard deviation, so the cutoff should be considered the number of standard deviations away. For other distributions, there are other interpretations which will make sense in the context of measuring the &#x201c;degree different&#x201d;. See <a class="externalLink" href="http://eurekastatistics.com/using-the-median-absolute-deviation-to-find-outliers/">http://eurekastatistics.com/using-the-median-absolute-deviation-to-find-outliers/</a> for a brief discussion of this.</p>
+<p>We will set a threat triage level of <tt>10</tt> if a message generates a outlier score of more than 3.5. This cutoff will depend on your data and should be adjusted based on the assumed underlying distribution.  Note that under the assumptions of normality, MAD will act as a robust estimator of the standard deviation, so the cutoff should be considered the number of standard deviations away.  For other distributions, there are other interpretations which will make sense in the context of measuring the &#x201c;degree different&#x201d;.  See <a class="externalLink" href="http://eurekastatistics.com/using-the-median-absolute-deviation-to-find-outliers/">http://eurekastatistics.com/using-the-median-absolute-deviation-to-find-outliers/</a> for a brief discussion of this.</p>
 <p>Create the following in <tt>$METRON_HOME/config/zookeeper/enrichments/mad.json</tt>:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;enrichment&quot;: {
     &quot;fieldMap&quot;: {
       &quot;stellar&quot; : {
@@ -866,11 +707,12 @@ PROFILE_GET( 'sketchy_mad', 'global', PROFILE_FIXED(10, 'MINUTES')) ), value)&qu
   }
 }
 </pre></div></div>
+
 <p>We also need an indexing configuration. Create the following in <tt>$METRON_HOME/config/zookeeper/indexing/mad.json</tt>:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;hdfs&quot; : {
     &quot;index&quot;: &quot;mad&quot;,
     &quot;batchSize&quot;: 1,
@@ -882,15 +724,16 @@ PROFILE_GET( 'sketchy_mad', 'global', PROFILE_FIXED(10, 'MINUTES')) ), value)&qu
     &quot;enabled&quot; : true
   }
 }
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h4><a name="The_Profiler"></a>The Profiler</h4>
-<p>We can set up the profiler to track the MAD statistical state required to compute MAD. For the purposes of this demonstration, we will configure the profiler to capture statistics on the minute mark. We will capture a global statistical state for the <tt>value</tt> field and we will look back for a 5 minute window when computing the median.</p>
+<p>We can set up the profiler to track the MAD statistical state required to compute MAD.  For the purposes of this demonstration, we will configure the profiler to capture statistics on the minute mark.  We will capture a global statistical state for the <tt>value</tt> field and we will look back for a 5 minute window when computing the median.</p>
 <p>Create the following file at <tt>$METRON_HOME/config/zookeeper/profiler.json</tt>:</p>
 
-<div class="source">
-<div class="source">
-<pre>{
+<div>
+<div>
+<pre class="source">{
   &quot;profiles&quot;: [
     {
       &quot;profile&quot;: &quot;sketchy_mad&quot;,
@@ -908,64 +751,69 @@ PROFILE_GET( 'sketchy_mad', 'global', PROFILE_FIXED(10, 'MINUTES')) ), value)&qu
   ]
 }
 </pre></div></div>
+
 <p>Adjust <tt>$METRON_HOME/config/zookeeper/global.json</tt> to adjust the capture duration:</p>
 
-<div class="source">
-<div class="source">
-<pre> &quot;profiler.client.period.duration&quot; : &quot;1&quot;,
+<div>
+<div>
+<pre class="source"> &quot;profiler.client.period.duration&quot; : &quot;1&quot;,
  &quot;profiler.client.period.duration.units&quot; : &quot;MINUTES&quot;
 </pre></div></div>
+
 <p>Adjust <tt>$METRON_HOME/config/profiler.properties</tt> to adjust the capture duration by changing <tt>profiler.period.duration=15</tt> to <tt>profiler.period.duration=1</tt></p></div>
 <div class="section">
 <h4><a name="Execute_the_Flow"></a>Execute the Flow</h4>
-
 <ol style="list-style-type: decimal">
-  
+
 <li>
-<p>Install the elasticsearch head plugin by executing: <tt>/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head</tt></p></li>
-  
+
+<p>Install the elasticsearch head plugin by executing: <tt>/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head</tt></p>
+</li>
 <li>
-<p>Stopping all other parser topologies via monit</p></li>
-  
+
+<p>Stopping all other parser topologies via monit</p>
+</li>
 <li>
-<p>Create the <tt>mad</tt> kafka topic by executing: <tt>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper node1:2181 --create --topic mad --partitions 1 --replication-factor 1</tt></p></li>
-  
+
+<p>Create the <tt>mad</tt> kafka topic by executing: <tt>/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper node1:2181 --create --topic mad --partitions 1 --replication-factor 1</tt></p>
+</li>
 <li>
-<p>Push the modified configs by executing: <tt>$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -z node1:2181 -i $METRON_HOME/config/zookeeper/</tt></p></li>
-  
+
+<p>Push the modified configs by executing: <tt>$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -z node1:2181 -i $METRON_HOME/config/zookeeper/</tt></p>
+</li>
 <li>
-<p>Start the profiler by executing: <tt>$METRON_HOME/bin/start_profiler_topology.sh</tt></p></li>
-  
+
+<p>Start the profiler by executing: <tt>$METRON_HOME/bin/start_profiler_topology.sh</tt></p>
+</li>
 <li>
-<p>Start the parser topology by executing: <tt>$METRON_HOME/bin/start_parser_topology.sh -k node1:6667 -z node1:2181 -s mad</tt></p></li>
-  
+
+<p>Start the parser topology by executing: <tt>$METRON_HOME/bin/start_parser_topology.sh -k node1:6667 -z node1:2181 -s mad</tt></p>
+</li>
 <li>
-<p>Ensure that the enrichment and indexing topologies are started. If not, then start those via monit or by hand.</p></li>
-  
+
+<p>Ensure that the enrichment and indexing topologies are started.  If not, then start those via monit or by hand.</p>
+</li>
 <li>
-<p>Generate data into kafka by executing the following for at least 10 minutes: <tt>~/rand_gen.py 0 1 1 | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic mad</tt> Note: if you chose the use the t-distribution script above, you would adjust the parameters of the <tt>rand_gen.py</tt> script accordingly.</p></li>
-  
+
+<p>Generate data into kafka by executing the following for at least 10 minutes: <tt>~/rand_gen.py 0 1 1 | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic mad</tt> Note: if you chose the use the t-distribution script above, you would adjust the parameters of the <tt>rand_gen.py</tt> script accordingly.</p>
+</li>
 <li>
-<p>Stop the above with ctrl-c and send in an obvious outlier into kafka: <tt>echo &quot;1000&quot; | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic mad</tt></p></li>
+
+<p>Stop the above with ctrl-c and send in an obvious outlier into kafka: <tt>echo &quot;1000&quot; | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list node1:6667 --topic mad</tt></p>
+</li>
 </ol>
 <p>You should be able to find the outlier via the elasticsearch head plugin by searching for the messages where <tt>is_alert</tt> is <tt>true</tt>.</p></div></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-contrib/metron-docker/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-contrib/metron-docker/index.html b/site/current-book/metron-contrib/metron-docker/index.html
index 761c3b0..5769043 100644
--- a/site/current-book/metron-contrib/metron-docker/index.html
+++ b/site/current-book/metron-contrib/metron-docker/index.html
@@ -1,274 +1,170 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-contrib/metron-docker/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Metron Docker</title>
-    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../../css/site.css" />
     <link rel="stylesheet" href="../../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Metron Docker</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Metron Docker</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Docker</a>
-          </li>
-                                                                                                                                                                                                                                                                                                                                                                                                            
-      <li>
-    
-                          <a href="../../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-right"></i>
-        Deployment</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Docker</a></li>
+    <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Metron Docker</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Metron Docker</h1>
 <p><a name="Metron_Docker"></a></p>
-<p>Metron Docker is a <a class="externalLink" href="https://docs.docker.com/compose/overview/">Docker Compose</a> application that is intended only for development and integration testing of Metron. These images can quickly spin-up the underlying components on which Apache Metron runs.</p>
-<p>None of the core Metron components are setup or launched automatically with these Docker images. You will need to manually setup and start the Metron components that you require. You should not expect to see telemetry being parsed, enriched, or indexed. If you are looking to try-out, experiment or demo Metron capabilities on a single node, then the <a href="../../metron-deployment/vagrant/full-dev-platform/index.html">Vagrant-driven VM</a> is what you need. Use this instead of Vagrant when:</p>
-
+<p>Metron Docker is a <a class="externalLink" href="https://docs.docker.com/compose/overview/">Docker Compose</a> application that is intended only for development and integration testing of Metron.  These images can quickly spin-up the underlying components on which Apache Metron runs.</p>
+<p>None of the core Metron components are setup or launched automatically with these Docker images.  You will need to manually setup and start the Metron components that you require.  You should not expect to see telemetry being parsed, enriched, or indexed.  If you are looking to try-out, experiment or demo Metron capabilities on a single node, then the <a href="../../metron-deployment/development/centos6/index.html">Vagrant-driven VM</a> is what you need.  Use this instead of Vagrant when:</p>
 <ul>
-  
+
 <li>You want an environment that can be built and spun up quickly</li>
-  
 <li>You need to frequently rebuild and restart services</li>
-  
 <li>You only need to test, troubleshoot or develop against a subset of services</li>
 </ul>
 <p>Metron Docker includes these images that have been customized for Metron:</p>
-
 <ul>
-  
+
 <li>Kafka (with Zookeeper)</li>
-  
 <li>HBase</li>
-  
 <li>Storm</li>
-  
 <li>Elasticsearch</li>
-  
 <li>Kibana</li>
-  
 <li>HDFS</li>
 </ul>
 <div class="section">
 <h2><a name="Setup"></a>Setup</h2>
-<p>Install <a class="externalLink" href="https://docs.docker.com/docker-for-mac/">Docker for Mac</a> or <a class="externalLink" href="https://docs.docker.com/docker-for-windows/">Docker for Windows</a>. The following versions have been tested:</p>
-
+<p>Install <a class="externalLink" href="https://docs.docker.com/docker-for-mac/">Docker for Mac</a> or <a class="externalLink" href="https://docs.docker.com/docker-for-windows/">Docker for Windows</a>.  The following versions have been tested:</p>
 <ul>
-  
+
 <li>Docker version 1.12.0</li>
-  
 <li>docker-machine version 0.8.0</li>
-  
 <li>docker-compose version 1.8.0</li>
 </ul>
 <p>Build Metron from the top level directory with:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ cd $METRON_HOME
+<div>
+<div>
+<pre class="source">$ cd $METRON_HOME
 $ mvn clean install -DskipTests
 </pre></div></div>
-<p>You are welcome to use an existing Docker host but we prefer one with more resources. You can create one of those with this script:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ export METRON_DOCKER_HOME=$METRON_HOME/metron-contrib/metron-docker
-$ cd $METRON_DOCKER_HOME 
+<p>You are welcome to use an existing Docker host but we prefer one with more resources.  You can create one of those with this script:</p>
+
+<div>
+<div>
+<pre class="source">$ export METRON_DOCKER_HOME=$METRON_HOME/metron-contrib/metron-docker
+$ cd $METRON_DOCKER_HOME
 $ ./scripts/create-docker-machine.sh
 </pre></div></div>
-<p>This will create a host called &#x201c;metron-machine&#x201d;. Anytime you want to run Docker commands against this host, make sure you run this first to set the Docker environment variables:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ eval &quot;$(docker-machine env metron-machine)&quot;
+<p>This will create a host called &#x201c;metron-machine&#x201d;.  Anytime you want to run Docker commands against this host, make sure you run this first to set the Docker environment variables:</p>
+
+<div>
+<div>
+<pre class="source">$ eval &quot;$(docker-machine env metron-machine)&quot;
 </pre></div></div>
+
 <p>If you wish to use a local docker-engine install, please set an environment variable BROKER_IP_ADDR to the IP address of your host machine. This cannot be the loopback address.</p></div>
 <div class="section">
 <h2><a name="Usage"></a>Usage</h2>
 <p>Navigate to the compose application root:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ cd $METRON_DOCKER_HOME/compose/
+<div>
+<div>
+<pre class="source">$ cd $METRON_DOCKER_HOME/compose/
 </pre></div></div>
-<p>The Metron Docker environment lifecycle is controlled by the <a class="externalLink" href="https://docs.docker.com/compose/reference/overview/">docker-compose</a> command. The service names can be found in the docker-compose.yml file. For example, to build and start the environment run this command:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ eval &quot;$(docker-machine env metron-machine)&quot;
+<p>The Metron Docker environment lifecycle is controlled by the <a class="externalLink" href="https://docs.docker.com/compose/reference/overview/">docker-compose</a> command.  The service names can be found in the docker-compose.yml file.  For example, to build and start the environment run this command:</p>
+
+<div>
+<div>
+<pre class="source">$ eval &quot;$(docker-machine env metron-machine)&quot;
 $ docker-compose up -d
 </pre></div></div>
+
 <p>After all services have started list the containers and ensure their status is &#x2018;Up&#x2019;:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ docker ps --format 'table {{.Names}}\t{{.Status}}'
+<div>
+<div>
+<pre class="source">$ docker ps --format 'table {{.Names}}\t{{.Status}}'
 NAMES                    STATUS
 metron_storm_1           Up 5 minutes
 metron_hbase_1           Up 5 minutes
@@ -276,216 +172,228 @@ metron_kibana_1          Up 5 minutes
 metron_kafkazk_1         Up 5 minutes
 metron_elasticsearch_1   Up 5 minutes
 </pre></div></div>
-<p>Various services are exposed through http on the Docker host. Get the host ip from the URL property:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ docker-machine ls
+<p>Various services are exposed through http on the Docker host.  Get the host ip from the URL property:</p>
+
+<div>
+<div>
+<pre class="source">$ docker-machine ls
 NAME             ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
 metron-machine   *        virtualbox   Running   tcp://192.168.99.100:2376           v1.12.5
 </pre></div></div>
-<p>Then, assuming a host ip of <tt>192.168.99.100</tt>, the UIs and APIs are available at:</p>
 
+<p>Then, assuming a host ip of <tt>192.168.99.100</tt>, the UIs and APIs are available at:</p>
 <ul>
-  
+
 <li>Storm - <a class="externalLink" href="http://192.168.99.100:8080/">http://192.168.99.100:8080/</a></li>
-  
 <li>HBase - <a class="externalLink" href="http://192.168.99.100:16010/">http://192.168.99.100:16010/</a></li>
-  
 <li>Elasticsearch - <a class="externalLink" href="http://192.168.99.100:9200/_plugin/head/">http://192.168.99.100:9200/_plugin/head/</a></li>
-  
 <li>Kibana - <a class="externalLink" href="http://192.168.99.100:5601/">http://192.168.99.100:5601/</a></li>
-  
 <li>HDFS (Namenode) - <a class="externalLink" href="http://192.168.99.100:50070/">http://192.168.99.100:50070/</a></li>
 </ul>
-<p>The Storm logs can be useful when troubleshooting topologies. They can be found on the Storm container in <tt>/usr/share/apache-storm/logs</tt>.</p>
+<p>The Storm logs can be useful when troubleshooting topologies.  They can be found on the Storm container in <tt>/usr/share/apache-storm/logs</tt>.</p>
 <p>When done using the machine, shut it down with:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ docker-compose down
-</pre></div></div></div>
+<div>
+<div>
+<pre class="source">$ docker-compose down
+</pre></div></div>
+</div>
 <div class="section">
 <h2><a name="Examples"></a>Examples</h2>
-
 <ul>
-  
+
 <li><a href="#Deploy_a_new_parser_class">Deploy a new parser class</a></li>
-  
 <li><a href="#Connect_to_a_container">Connect to a container</a></li>
-  
 <li><a href="create-a-sensor-from-sample-data/index.html">Create a sensor from sample data</a></li>
-  
 <li><a href="upload-configs-to-zookeeper/index.html">Upload configs to Zookeeper</a></li>
-  
 <li><a href="manage-a-topology/index.html">Manage a topology</a></li>
-  
 <li><a href="run-sensor-data-end-to-end/index.html">Run sensor data end to end</a></li>
 </ul>
 <div class="section">
 <h3><a name="Deploy_a_new_parser_class"></a>Deploy a new parser class</h3>
 <p>After adding a new parser to metron-parsers, build Metron from the top level directory:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ cd $METRON_HOME
+<div>
+<div>
+<pre class="source">$ cd $METRON_HOME
 $ mvn clean install -DskipTests
 </pre></div></div>
+
 <p>Then run these commands to redeploy the parsers to the Storm image:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ cd $METRON_DOCKER_HOME/compose
+<div>
+<div>
+<pre class="source">$ cd $METRON_DOCKER_HOME/compose
 $ docker-compose down
 $ docker-compose build storm
 $ docker-compose up -d
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Connect_to_a_container"></a>Connect to a container</h3>
 <p>Suppose there is a problem with Kafka and the logs are needed for further investigation. Run this command to connect and explore the running Kafka container:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ cd $METRON_DOCKER_HOME/compose
+<div>
+<div>
+<pre class="source">$ cd $METRON_DOCKER_HOME/compose
 $ docker-compose exec kafkazk bash
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Create_a_sensor_from_sample_data"></a>Create a sensor from sample data</h3>
-<p>A tool for producing test data in Kafka is included with the Kafka/Zookeeper image. It loops through lines in a test data file and outputs them to Kafka at the desired frequency. Create a test data file in <tt>./kafkazk/data/</tt> and rebuild the Kafka/Zookeeper image:</p>
+<p>A tool for producing test data in Kafka is included with the Kafka/Zookeeper image.  It loops through lines in a test data file and outputs them to Kafka at the desired frequency.  Create a test data file in <tt>./kafkazk/data/</tt> and rebuild the Kafka/Zookeeper image:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ cd $METRON_DOCKER_HOME/compose
+<div>
+<div>
+<pre class="source">$ cd $METRON_DOCKER_HOME/compose
 $ printf 'first test data\nsecond test data\nthird test data\n' &gt; ./kafkazk/data/TestData.txt
 $ docker-compose down
 $ docker-compose build kafkazk
 $ docker-compose up -d
 </pre></div></div>
-<p>This will deploy the test data file to the Kafka/Zookeeper container. Now that data can be streamed to a Kafka topic:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ docker-compose exec kafkazk ./bin/produce-data.sh
+<p>This will deploy the test data file to the Kafka/Zookeeper container.  Now that data can be streamed to a Kafka topic:</p>
+
+<div>
+<div>
+<pre class="source">$ docker-compose exec kafkazk ./bin/produce-data.sh
 Usage:  produce-data.sh data_path topic [message_delay_in_seconds]
 
 # Stream data in TestData.txt to the 'test' Kafka topic at a frequency of 5 seconds (default is 1 second)
-$ docker-compose exec kafkazk ./bin/produce-data.sh /data/TestData.txt test 5 
+$ docker-compose exec kafkazk ./bin/produce-data.sh /data/TestData.txt test 5
 </pre></div></div>
+
 <p>The Kafka/Zookeeper image comes with sample Bro and Squid data:</p>
 
-<div class="source">
-<div class="source">
-<pre># Stream Bro test data every 1 second
+<div>
+<div>
+<pre class="source"># Stream Bro test data every 1 second
 $ docker-compose exec kafkazk ./bin/produce-data.sh /data/BroExampleOutput.txt bro
 
 # Stream Squid test data every 0.1 seconds
 $ docker-compose exec kafkazk ./bin/produce-data.sh /data/SquidExampleOutput.txt squid 0.1
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Upload_configs_to_Zookeeper"></a>Upload configs to Zookeeper</h3>
-<p>Parser configs and a global config configured for this Docker environment are included with the Kafka/Zookeeper image. Load them with:</p>
+<p>Parser configs and a global config configured for this Docker environment are included with the Kafka/Zookeeper image.  Load them with:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ docker-compose exec kafkazk bash
+<div>
+<div>
+<pre class="source">$ docker-compose exec kafkazk bash
 # $METRON_HOME/bin/zk_load_configs.sh -z localhost:2181 -m PUSH -i $METRON_HOME/config/zookeeper
 # exit
 </pre></div></div>
+
 <p>Dump out the configs with:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ docker-compose exec kafkazk bash
+<div>
+<div>
+<pre class="source">$ docker-compose exec kafkazk bash
 # $METRON_HOME/bin/zk_load_configs.sh -z localhost:2181 -m DUMP
 # exit
-</pre></div></div></div>
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Manage_a_topology"></a>Manage a topology</h3>
 <p>The Storm image comes with a script to easily start parser topologies:</p>
 
-<div class="source">
-<div class="source">
-<pre>docker-compose exec storm ./bin/start_docker_parser_topology.sh sensor_name
+<div>
+<div>
+<pre class="source">docker-compose exec storm ./bin/start_docker_parser_topology.sh sensor_name
 </pre></div></div>
+
 <p>The enrichment topology can be started with:</p>
 
-<div class="source">
-<div class="source">
-<pre>docker-compose exec storm ./bin/start_enrichment_topology.sh
+<div>
+<div>
+<pre class="source">docker-compose exec storm ./bin/start_enrichment_topology.sh
 </pre></div></div>
+
 <p>The indexing topology can be started with:</p>
 
-<div class="source">
-<div class="source">
-<pre>docker-compose exec storm ./bin/start_elasticsearch_topology.sh
+<div>
+<div>
+<pre class="source">docker-compose exec storm ./bin/start_elasticsearch_topology.sh
 </pre></div></div>
-<p>Topologies can be stopped using the Storm CLI. For example, stop the enrichment topology with:</p>
 
-<div class="source">
-<div class="source">
-<pre>docker-compose exec storm storm kill enrichments -w 0
-</pre></div></div></div>
+<p>Topologies can be stopped using the Storm CLI.  For example, stop the enrichment topology with:</p>
+
+<div>
+<div>
+<pre class="source">docker-compose exec storm storm kill enrichments -w 0
+</pre></div></div>
+</div>
 <div class="section">
 <h3><a name="Run_sensor_data_end_to_end"></a>Run sensor data end to end</h3>
 <p>First ensure configs were uploaded as described in the previous example. Then start a sensor and leave it running:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ cd $METRON_DOCKER_HOME/compose
+<div>
+<div>
+<pre class="source">$ cd $METRON_DOCKER_HOME/compose
 $ docker-compose exec kafkazk ./bin/produce-data.sh /data/BroExampleOutput.txt bro
 </pre></div></div>
+
 <p>Open a separate console session and verify the sensor is running by consuming a message from Kafka:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ export METRON_DOCKER_HOME=$METRON_HOME/metron-contrib/metron-docker
+<div>
+<div>
+<pre class="source">$ export METRON_DOCKER_HOME=$METRON_HOME/metron-contrib/metron-docker
 $ cd $METRON_DOCKER_HOME/compose
 $ docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic bro
 </pre></div></div>
+
 <p>A new message should be printed every second. Now kill the consumer and start the Bro parser topology:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ docker-compose exec storm ./bin/start_docker_parser_topology.sh bro
+<div>
+<div>
+<pre class="source">$ docker-compose exec storm ./bin/start_docker_parser_topology.sh bro
 </pre></div></div>
-<p>Bro data should be flowing through the bro parser topology and into the Kafka enrichments topic. The enrichments topic should be created automatically:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ docker-compose exec kafkazk ./bin/kafka-topics.sh --zookeeper localhost:2181 --list
+<p>Bro data should be flowing through the bro parser topology and into the Kafka enrichments topic.  The enrichments topic should be created automatically:</p>
+
+<div>
+<div>
+<pre class="source">$ docker-compose exec kafkazk ./bin/kafka-topics.sh --zookeeper localhost:2181 --list
 bro
 enrichments
 indexing
 </pre></div></div>
+
 <p>Verify parsed Bro data is in the Kafka enrichments topic:</p>
 
-<div class="source">
-<div class="source">
-<pre>docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic enrichments
+<div>
+<div>
+<pre class="source">docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic enrichments
 </pre></div></div>
+
 <p>Now start the enrichment topology:</p>
 
-<div class="source">
-<div class="source">
-<pre>docker-compose exec storm ./bin/start_enrichment_topology.sh
+<div>
+<div>
+<pre class="source">docker-compose exec storm ./bin/start_enrichment_topology.sh
 </pre></div></div>
-<p>Parsed Bro data should be flowing through the enrichment topology and into the Kafka indexing topic. Verify enriched Bro data is in the Kafka indexing topic:</p>
 
-<div class="source">
-<div class="source">
-<pre>docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic indexing
+<p>Parsed Bro data should be flowing through the enrichment topology and into the Kafka indexing topic.  Verify enriched Bro data is in the Kafka indexing topic:</p>
+
+<div>
+<div>
+<pre class="source">docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic indexing
 </pre></div></div>
+
 <p>Now start the indexing topology:</p>
 
-<div class="source">
-<div class="source">
-<pre>docker-compose exec storm ./bin/start_elasticsearch_topology.sh
+<div>
+<div>
+<pre class="source">docker-compose exec storm ./bin/start_elasticsearch_topology.sh
 </pre></div></div>
+
 <p>Enriched Bro data should now be present in the Elasticsearch container:</p>
 
-<div class="source">
-<div class="source">
-<pre>$ docker-machine ls
+<div>
+<div>
+<pre class="source">$ docker-machine ls
 NAME             ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
 metron-machine   *        virtualbox   Running   tcp://192.168.99.100:2376           v1.12.5
 
@@ -494,23 +402,17 @@ health status index                   pri rep docs.count docs.deleted store.size
 yellow open   .kibana                   1   1          1            0      3.1kb          3.1kb
 yellow open   bro_index_2016.12.19.18   5   1        180            0      475kb          475kb
 </pre></div></div></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>


[33/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

Posted by mm...@apache.org.
http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-contrib/metron-performance/index.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-contrib/metron-performance/index.html b/site/current-book/metron-contrib/metron-performance/index.html
new file mode 100644
index 0000000..f618b06
--- /dev/null
+++ b/site/current-book/metron-contrib/metron-performance/index.html
@@ -0,0 +1,317 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-contrib/metron-performance/index.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Metron &#x2013; Performance Utilities</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Performance Utilities</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
+      </div>
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li class="active"><a href="#"><span class="none"></span>Performance</a></li>
+    <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
+    <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
+            </div>
+          </div>
+        </div>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Performance Utilities</h1>
+<p><a name="Performance_Utilities"></a></p>
+<p>This project creates some useful performance monitoring and measurement utilities.</p>
+<div class="section">
+<h2><a name="load-tool.sh"></a><tt>load-tool.sh</tt></h2>
+<p>The Load tool is intended to do the following:</p>
+<ul>
+
+<li>Generate a load at a specific events per second into kafka
+<ul>
+
+<li>The messages are taken from a template file, where there is a message template per line</li>
+<li>The load can be biased (e.g. 80% of the load can be comprised of 20% of the templates)</li>
+</ul>
+</li>
+<li>Monitor the kafka offsets for a topic to determine the events per second written
+<ul>
+
+<li>This could be the topic that you are generating load on</li>
+<li>This could be another topic that represents the output of some topology (e.g. generate load on <tt>enrichments</tt> and monitor <tt>indexing</tt> to determine the throughput of the enrichment topology).</li>
+</ul>
+</li>
+</ul>
+
+<div>
+<div>
+<pre class="source">usage: Generator
+ -bs,--sample_bias &lt;BIAS_FILE&gt;         The discrete distribution to bias
+                                       the sampling. This is a CSV of 2
+                                       columns.  The first column is the %
+                                       of the templates and the 2nd column
+                                       is the probability (0-100) that
+                                       it's chosen.  For instance:
+                                       20,80
+                                       80,20
+                                       implies that 20% of the templates
+                                       will comprise 80% of the output and
+                                       the remaining 80% of the templates
+                                       will comprise 20% of the output.
+ -c,--csv &lt;CSV_FILE&gt;                   A CSV file to emit monitoring data
+                                       to.  The format is a CSV with the
+                                       following schema: timestamp, (name,
+                                       eps, historical_mean,
+                                       historical_stddev)+
+ -cg,--consumer_group &lt;GROUP_ID&gt;       Consumer Group.  The default is
+                                       load.group
+ -e,--eps &lt;EPS&gt;                        The target events per second
+ -h,--help                             Generate Help screen
+ -k,--kafka_config &lt;CONFIG_FILE&gt;       The kafka config.  This is a file
+                                       containing a JSON map with the
+                                       kafka config.
+ -l,--lookback &lt;LOOKBACK&gt;              When summarizing, how many
+                                       monitoring periods should we
+                                       summarize over?  If 0, then no
+                                       summary.  Default: 5
+ -md,--monitor_delta_ms &lt;TIME_IN_MS&gt;   The time (in ms) between monitoring
+                                       output. Default is 10000
+ -mt,--monitor_topic &lt;TOPIC&gt;           The kafka topic to monitor.
+ -ot,--output_topic &lt;TOPIC&gt;            The kafka topic to write to
+ -p,--threads &lt;NUM_THREADS&gt;            The number of threads to use when
+                                       extracting data.  The default is
+                                       the number of cores of your
+                                       machine.
+ -sd,--send_delta_ms &lt;TIME_IN_MS&gt;      The time (in ms) between sending a
+                                       batch of messages. Default is 100
+ -t,--template &lt;TEMPLATE_FILE&gt;         The template file to use for
+                                       generation.  This should be a file
+                                       with a template per line with
+                                       $METRON_TS and $METRON_GUID in the
+                                       spots for timestamp and guid, if
+                                       you so desire them.
+ -tl,--time_limit_ms &lt;MS&gt;              The total amount of time to run
+                                       this in milliseconds.  By default,
+                                       it never stops.
+ -z,--zk_quorum &lt;QUORUM&gt;               zookeeper quorum
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Templates"></a>Templates</h2>
+<p>Messages are drawn from a template file.  A template file has a message template per line.<br />
+For instance, let&#x2019;s say we want to generate JSON maps with fields: <tt>source.type</tt>, <tt>ip_src_addr</tt> and <tt>ip_dst_addr</tt>.  We can generate a template file with a template like the following per line:</p>
+
+<div>
+<div>
+<pre class="source">{ &quot;source.type&quot; : &quot;asa&quot;, &quot;ip_src_addr&quot; : &quot;127.0.0.1&quot;, &quot;ip_dst_addr&quot; : &quot;191.168.1.1&quot; }
+</pre></div></div>
+
+<p>When messages are generated, there are some special replacements that can be used: <tt>$METRON_TS</tt> and <tt>$METRON_GUID</tt>. We can adjust our previous template to use these like so:</p>
+
+<div>
+<div>
+<pre class="source">{ &quot;source.type&quot; : &quot;asa&quot;, &quot;ip_src_addr&quot; : &quot;127.0.0.1&quot;, &quot;ip_dst_addr&quot; : &quot;191.168.1.1&quot;, &quot;timestamp&quot; : $METRON_TS, &quot;guid&quot; : &quot;$METRON_GUID&quot; }
+</pre></div></div>
+
+<p>One note about GUIDs generated.  We do not generate global UUIDs, they are unique only within the context of a given generator run.</p></div>
+<div class="section">
+<h2><a name="Biased_Sampling"></a>Biased Sampling</h2>
+<p>This load tool can be configured to use biased sampling.  This is useful if, for instance, you are trying to model data which is not distributed uniformly, like many types of network data.  Generating synthetic data with similar distribution to your regular data will enable the caches to be exercised in the same way, for instance, and yield a more realistic scenario.</p>
+<p>You specify the biases in a csv file with 2 columns:</p>
+<ul>
+
+<li>The first column represents the % of the templates</li>
+<li>The second column represents the % of the generated output.</li>
+</ul>
+<p>A simple example would be to generate samples based on Pareto&#x2019;s principle:</p>
+
+<div>
+<div>
+<pre class="source">20,80
+80,20
+</pre></div></div>
+
+<p>This would yield biases that mean the first 20% of the templates in the template file would comprise 80% of the output.</p>
+<p>A more complex example might be:</p>
+
+<div>
+<div>
+<pre class="source">20,80
+20,5
+50,1
+10,14
+</pre></div></div>
+
+<p>This would would imply:</p>
+<ul>
+
+<li>The first 20% of the templates would comprise 80% of the output</li>
+<li>The next 20% of the templates would comprise 5% of the output</li>
+<li>The next 50% of the templates would comprise 1% of the output</li>
+<li>The next 10% of the templates would comprise 14% of the output.</li>
+</ul></div>
+<div class="section">
+<h2><a name="CSV_Output"></a>CSV Output</h2>
+<p>For those who would prefer a different visualization or wish to incorporate the output of this tool into an automated test, you can specify a file to emit data in CSV format to via the <tt>-c</tt> or <tt>--csv</tt> option.</p>
+<p>The CSV columns are as follows:</p>
+<ul>
+
+<li>timestamp in epoch millis</li>
+</ul>
+<p>If you are generating synthetic data, then:</p>
+<ul>
+
+<li>&#x201c;generated&#x201d;</li>
+<li>The events per second generated</li>
+<li>The mean of the events per second generated for the the last <tt>k</tt> runs, where <tt>k</tt> is the lookback (set via <tt>-l</tt> and defaulted to <tt>5</tt>)</li>
+<li>The standard deviation of the events per second generated for the last <tt>k</tt> runs, where <tt>k</tt> is the lookback (set via <tt>-l</tt> and defaulted to <tt>5</tt>)</li>
+</ul>
+<p>If you are monitoring a topic, then:</p>
+<ul>
+
+<li>&#x201c;throughput measured&#x201d;</li>
+<li>The events per second measured</li>
+<li>The mean of the events per second measured for the the last <tt>k</tt> runs, where <tt>k</tt> is the lookback (set via <tt>-l</tt> and defaulted to <tt>5</tt>)</li>
+<li>The standard deviation of the events per second measured for the last <tt>k</tt> runs, where <tt>k</tt> is the lookback (set via <tt>-l</tt> and defaulted to <tt>5</tt>)</li>
+</ul>
+<p>Obviously, if you are doing both generating and monitoring the throughput of a topic, then all of the columns are added.</p>
+<p>An example of CSV output is:</p>
+
+<div>
+<div>
+<pre class="source">1520506955047,generated,,,,throughput measured,,,
+1520506964896,generated,1045,1045,0,throughput measured,,,
+1520506974896,generated,1000,1022,31,throughput measured,1002,1002,0
+1520506984904,generated,999,1014,26,throughput measured,999,1000,2
+1520506994896,generated,1000,1011,22,throughput measured,1000,1000,1
+1520507004896,generated,1000,1008,20,throughput measured,1000,1000,1
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Use-cases_for_the_Load_Tool"></a>Use-cases for the Load Tool</h2>
+<div class="section">
+<h3><a name="Measure_Throughput_of_a_Topology"></a>Measure Throughput of a Topology</h3>
+<p>One can use the load tool to monitor performance of a kafka-to-kafka topology. For instance, we could monitor the throughput of the enrichment topology by monitoring the <tt>enrichments</tt> kafka topic:</p>
+
+<div>
+<div>
+<pre class="source">$METRON_HOME/bin/load_tool.sh -mt enrichments -z $ZOOKEEPER
+</pre></div></div>
+</div>
+<div class="section">
+<h3><a name="Generate_Synthetic_Load_and_Measure_Performance"></a>Generate Synthetic Load and Measure Performance</h3>
+<p>One can use the load tool to generate synthetic load and monitor performance of a kafka-to-kafka topology.  For instance, we could monitor the performance of the enrichment topology.  It is advised to start the enrichment topology against a new topic and write to a new topic so as to not pollute your downstream indices.  So, for instance we could create a kafka topic called <tt>enrichments_load</tt> by generating load on it.  We could also create a new  kafka topic called <tt>indexing_load</tt> and configure the enrichment topology to output to it.  We would then generate load on <tt>enrichments_load</tt> and monitor <tt>indexing_load</tt>.</p>
+
+<div>
+<div>
+<pre class="source">#Threadpool of size 5, you want somewhere between 5 and 10 depending on the throughput numbers you're trying to drive
+#Messages drawn from ~/dummy.templates, which is a message template per line
+#Generate at a rate of 9000 messages per second
+#Emit the data to a CSV file ~/measurements.csv
+$METRON_HOME/bin/load_tool.sh -p 5 -ot enrichments_load -mt indexing_load -t ~/dummy.templates -eps 9000 -z $ZOOKEEPER -c ~/measurements.csv
+</pre></div></div>
+
+<p>Now, with the help of a bash function and gnuplot we can generate a plot of the historical throughput measurements for <tt>indexing_load</tt>:</p>
+
+<div>
+<div>
+<pre class="source"># Ensure that you have installed gnuplot and the liberation font package
+# via yum install -y gnuplot liberation-sans-fonts
+# We will define a plot function that will generate a png plot.  It takes
+# one arg, the output file.  It expects to have a 2 column CSV streamed
+#  with the first dimension being the timestamp and the second dimension
+# being what you want plotted.
+plot() {
+  awk -F, '{printf &quot;%d %d\n&quot;, $1/1000, $2} END { print &quot;e&quot; }' | gnuplot -e &quot;reset;clear;set style fill solid 1.0 border -1; set nokey;set title 'Throughput Measured'; set xlabel 'Time'; set boxwidth 0.5; set xtics rotate; set ylabel 'events/sec';set xdata time; set timefmt '%s';set format x '%H:%M:%S';set term png enhanced font '/usr/share/fonts/liberation/LiberationSans-Regular.ttf' 12 size 900,400; set output '$1';plot '&lt; cat -' using 1:2 with line lt -1 lw 2;&quot;
+}
+
+# We want to transform the CSV file into a space separated file with the
+# timestamp followed by the throughput measurements.
+cat ~/measurements.csv | awk -F, '{printf &quot;%d,%d\n&quot;, $1, $8 }' | plot performance_measurement.png
+</pre></div></div>
+
+<p>This generates a plot like so to <tt>performance_measurement.png</tt>: <img src="performance_measurement.png" alt="Performance Measurement" /></p></div></div>
+        </div>
+      </div>
+    </div>
+    <hr/>
+    <footer>
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
+      </div>
+    </footer>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/Kerberos-ambari-setup.html
----------------------------------------------------------------------
diff --git a/site/current-book/metron-deployment/Kerberos-ambari-setup.html b/site/current-book/metron-deployment/Kerberos-ambari-setup.html
index 1ed07c3..ed1e96b 100644
--- a/site/current-book/metron-deployment/Kerberos-ambari-setup.html
+++ b/site/current-book/metron-deployment/Kerberos-ambari-setup.html
@@ -1,264 +1,109 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-01-03
- | Rendered using Apache Maven Fluido Skin 1.3.0
+ | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/Kerberos-ambari-setup.md at 2018-06-07
+ | Rendered using Apache Maven Fluido Skin 1.7
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180103" />
+    <meta name="Date-Revision-yyyymmdd" content="20180607" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Metron &#x2013; Setting Up Kerberos in Vagrant Full Dev</title>
-    <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" />
+    <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" />
     <link rel="stylesheet" href="../css/site.css" />
     <link rel="stylesheet" href="../css/print.css" media="print" />
-
-      
-    <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script>
-
-                          
-        
-<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script>
-          
-            </head>
-        <body class="topBarDisabled">
-          
-                
-                    
-    
-        <div class="container-fluid">
-          <div id="banner">
-        <div class="pull-left">
-                                    <a href="http://metron.apache.org/" id="bannerLeft">
-                                                                                                <img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/>
-                </a>
-                      </div>
-        <div class="pull-right">  </div>
+    <script type="text/javascript" src="../js/apache-maven-fluido-1.7.min.js"></script>
+<script type="text/javascript">
+              $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
+            </script>
+  </head>
+  <body class="topBarDisabled">
+    <div class="container-fluid">
+      <div id="banner">
+        <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../images/metron-logo.png"  alt="Apache Metron" width="148px" height="48px"/></a></div>
+        <div class="pull-right"></div>
         <div class="clear"><hr/></div>
       </div>
 
       <div id="breadcrumbs">
         <ul class="breadcrumb">
-                
-                    
-                              <li class="">
-                    <a href="http://www.apache.org" class="externalLink" title="Apache">
-        Apache</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="http://metron.apache.org/" class="externalLink" title="Metron">
-        Metron</a>
-        </li>
-      <li class="divider ">/</li>
-            <li class="">
-                    <a href="../index.html" title="Documentation">
-        Documentation</a>
-        </li>
-      <li class="divider ">/</li>
-        <li class="">Setting Up Kerberos in Vagrant Full Dev</li>
-        
-                
-                    
-                  <li id="publishDate" class="pull-right">Last Published: 2018-01-03</li> <li class="divider pull-right">|</li>
-              <li id="projectVersion" class="pull-right">Version: 0.4.2</li>
-            
-                            </ul>
+      <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
+      <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
+      <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
+    <li class="active ">Setting Up Kerberos in Vagrant Full Dev</li>
+        <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li>
+          <li id="projectVersion" class="pull-right">Version: 0.5.0</li>
+        </ul>
       </div>
-
-            
       <div class="row-fluid">
-        <div id="leftColumn" class="span3">
+        <div id="leftColumn" class="span2">
           <div class="well sidebar-nav">
-                
-                    
-                <ul class="nav nav-list">
-                    <li class="nav-header">User Documentation</li>
-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                          
-      <li>
-    
-                          <a href="../index.html" title="Metron">
-          <i class="icon-chevron-down"></i>
-        Metron</a>
-                    <ul class="nav nav-list">
-                      
-      <li>
-    
-                          <a href="../Upgrading.html" title="Upgrading">
-          <i class="none"></i>
-        Upgrading</a>
-            </li>
-                                                                                                                                                      
-      <li>
-    
-                          <a href="../metron-analytics/index.html" title="Analytics">
-          <i class="icon-chevron-right"></i>
-        Analytics</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-contrib/metron-docker/index.html" title="Docker">
-          <i class="none"></i>
-        Docker</a>
-            </li>
-                                                                                                                                                                                                                                                                                                                                                                                                                      
-      <li>
-    
-                          <a href="../metron-deployment/index.html" title="Deployment">
-          <i class="icon-chevron-down"></i>
-        Deployment</a>
-                    <ul class="nav nav-list">
-                      
-      <li class="active">
-    
-            <a href="#"><i class="none"></i>Kerberos-ambari-setup</a>
-          </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup">
-          <i class="none"></i>
-        Kerberos-manual-setup</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2">
-          <i class="none"></i>
-        Amazon-ec2</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-deployment/other-examples/index.html" title="Other-examples">
-          <i class="icon-chevron-right"></i>
-        Other-examples</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/ambari/index.html" title="Ambari">
-          <i class="none"></i>
-        Ambari</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker">
-          <i class="none"></i>
-        Ansible-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker">
-          <i class="none"></i>
-        Rpm-docker</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-deployment/packaging/packer-build/index.html" title="Packer-build">
-          <i class="none"></i>
-        Packer-build</a>
-            </li>
-                                                                                                                                                
-      <li>
-    
-                          <a href="../metron-deployment/roles/index.html" title="Roles">
-          <i class="icon-chevron-right"></i>
-        Roles</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../metron-deployment/vagrant/index.html" title="Vagrant">
-          <i class="icon-chevron-right"></i>
-        Vagrant</a>
-                  </li>
-              </ul>
-        </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-alerts/index.html" title="Alerts">
-          <i class="none"></i>
-        Alerts</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-config/index.html" title="Config">
-          <i class="none"></i>
-        Config</a>
-            </li>
-                      
-      <li>
-    
-                          <a href="../metron-interface/metron-rest/index.html" title="Rest">
-          <i class="none"></i>
-        Rest</a>
-            </li>
-                                                                                                                                                                                                                                                                                    
-      <li>
-    
-                          <a href="../metron-platform/index.html" title="Platform">
-          <i class="icon-chevron-right"></i>
-        Platform</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../metron-sensors/index.html" title="Sensors">
-          <i class="icon-chevron-right"></i>
-        Sensors</a>
-                  </li>
-                      
-      <li>
-    
-                          <a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example">
-          <i class="none"></i>
-        Stellar-3rd-party-example</a>
-            </li>
-                                                                        
-      <li>
-    
-                          <a href="../metron-stellar/stellar-common/index.html" title="Stellar-common">
-          <i class="icon-chevron-right"></i>
-        Stellar-common</a>
-                  </li>
-                                                                                          
-      <li>
-    
-                          <a href="../use-cases/index.html" title="Use-cases">
-          <i class="icon-chevron-right"></i>
-        Use-cases</a>
-                  </li>
-              </ul>
-        </li>
-            </ul>
-                
-                    
-                
-          <hr class="divider" />
-
-           <div id="poweredBy">
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                            <div class="clear"></div>
-                             <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
-        <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" />
-      </a>
-                  </div>
+    <ul class="nav nav-list">
+      <li class="nav-header">User Documentation</li>
+    <li><a href="../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
+    <ul class="nav nav-list">
+    <li><a href="../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
+    <li><a href="../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
+    <li><a href="../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
+    <li><a href="../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
+    <li><a href="../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
+    <li><a href="../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a>
+    <ul class="nav nav-list">
+    <li class="active"><a href="#"><span class="none"></span>Kerberos-ambari-setup</a></li>
+    <li><a href="../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li>
+    <li><a href="../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li>
+    <li><a href="../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-right"></span>Ansible</a></li>
+    <li><a href="../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li>
+    <li><a href="../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li>
+    <li><a href="../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li>
+    <li><a href="../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li>
+    <li><a href="../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li>
+    <li><a href="../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li>
+    <li><a href="../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li>
+    </ul>
+</li>
+    <li><a href="../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
+    <li><a href="../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
+    <li><a href="../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li>
+    <li><a href="../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
+    <li><a href="../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
+    <li><a href="../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
+    <li><a href="../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
+    <li><a href="../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
+    <li><a href="../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
+    </ul>
+</li>
+</ul>
+          <hr />
+          <div id="poweredBy">
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+            <div class="clear"></div>
+<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /></a>
+            </div>
           </div>
         </div>
-        
-                
-        <div id="bodyColumn"  class="span9" >
-                                  
-            <h1>Setting Up Kerberos in Vagrant Full Dev</h1>
+        <div id="bodyColumn"  class="span10" >
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h1>Setting Up Kerberos in Vagrant Full Dev</h1>
 <p><a name="Setting_Up_Kerberos_in_Vagrant_Full_Dev"></a></p>
 <p><b>Note:</b> These are instructions for Kerberizing Metron Storm topologies from Kafka to Kafka. This does not cover the sensor connections or MAAS. General Kerberization notes can be found in the metron-deployment <a href="../index.html">README.md</a></p>
 <div class="section">
@@ -266,52 +111,43 @@
 <p>See <a href="Kerberos-manual-setup.html#Setup_a_KDC">Setup a KDC</a> and <a href="Kerberos-manual-setup.html#Verify_KDC">Verify KDC</a></p></div>
 <div class="section">
 <h2><a name="Ambari_Setup"></a>Ambari Setup</h2>
-
 <ol style="list-style-type: decimal">
-  
-<li>
-<p>Kerberize the cluster via Ambari. More detailed documentation can be found <a class="externalLink" href="http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_security/content/_enabling_kerberos_security_in_ambari.html">here</a>.</p>
+
+<li>Kerberize the cluster via Ambari. More detailed documentation can be found <a class="externalLink" href="http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_security/content/_enabling_kerberos_security_in_ambari.html">here</a>.
 <p>a. For this exercise, choose existing MIT KDC (this is what we setup and installed in the previous steps.)</p>
 <p><img src="../images/enable-kerberos.png" alt="enable keberos" /></p>
 <p><img src="../images/enable-kerberos-started.png" alt="enable keberos get started" /></p>
-<p>b. Setup Kerberos configuration. Realm is EXAMPLE.COM. The admin principal will end up as admin/admin@EXAMPLE.COM when testing the KDC. Use the password you entered during the step for adding the admin principal.</p>
+<p>b. Setup Kerberos configuration. Realm is EXAMPLE.COM. The admin principal will end up as <a class="externalLink" href="mailto:admin/admin@EXAMPLE.COM">admin/admin@EXAMPLE.COM</a> when testing the KDC. Use the password you entered during the step for adding the admin principal.</p>
 <p><img src="../images/enable-kerberos-configure-kerberos.png" alt="enable keberos configure" /></p>
 <p>c. Click through to &#x201c;Start and Test Services.&#x201d; Let the cluster spin up.</p></li>
 </ol></div>
 <div class="section">
 <h2><a name="Push_Data"></a>Push Data</h2>
-
 <ol style="list-style-type: decimal">
-  
-<li>
-<p>Kinit with the metron user</p>
-  
-<div class="source">
-<div class="source">
-<pre>kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM
-</pre></div></div></li>
+
+<li>Kinit with the metron user
+
+<div>
+<div>
+<pre class="source">kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM
+</pre></div></div>
+</li>
 </ol>
 <p>See <a href="Kerberos-manual-setup.html#Push_Data">Push Data</a></p>
 <div class="section">
 <h3><a name="More_Information"></a>More Information</h3>
 <p>See <a href="Kerberos-manual-setup.html#More_Information">More Information</a></p></div></div>
-                  </div>
-            </div>
-          </div>
-
+        </div>
+      </div>
+    </div>
     <hr/>
-
     <footer>
-            <div class="container-fluid">
-              <div class="row span12">Copyright &copy;                    2018
-                        <a href="https://www.apache.org">The Apache Software Foundation</a>.
-            All Rights Reserved.      
-                    
+      <div class="container-fluid">
+        <div class="row-fluid">
+© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
+            and the Apache Metron project logo are trademarks of The Apache Software Foundation.
+        </div>
       </div>
-
-                          
-        
-                </div>
     </footer>
   </body>
 </html>