You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by jo...@apache.org on 2018/01/12 21:00:25 UTC
svn commit: r1821033 [30/46] - in /nifi/site/trunk/docs: nifi-docs/
nifi-docs/components/ nifi-docs/components/org.apache.nifi/
nifi-docs/components/org.apache.nifi/nifi-ambari-nar/
nifi-docs/components/org.apache.nifi/nifi-ambari-nar/1.5.0/ nifi-docs/...
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateCounter/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateCounter/index.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateCounter/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateCounter/index.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>UpdateCounter</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">UpdateCounter</h1><h2>Description: </h2><p>This processor allows users to set specific counters and key points in their flow. It is useful for debugging and basic counting functions.</p><h3>Tags: </h3><p>counter, debug, instrumentation</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties
"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Counter Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the counter you want to set the value off - supports expression language like ${counterName}<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name"><strong>Delta</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">Adjusts the counter by the specified delta for each flow file received. May be a positive or negative integer.<br/><strong>Supports Expression Language: true</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>Counter was updated/retrieved</td></tr></table><h3>Reads Attributes: </h3><table id="reads-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>counterName</td><td>The n
ame of the counter to update/get.</td></tr></table><h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1,390 @@
+<!DOCTYPE html>
+<html lang="en">
+ <!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+ <head>
+ <meta charset="utf-8" />
+ <title>UpdateRecord</title>
+
+ <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" />
+ </head>
+
+ <body>
+ <p>
+ UpdateRecord makes use of the NiFi <a href="../../../../../html/record-path-guide.html">
+ RecordPath Domain-Specific Language (DSL)</a> to allow the user
+ to indicate which field(s) in the Record should be updated. Users do this by adding a User-defined
+ Property to the Processor's configuration. The name of the User-defined Property must be the
+ RecordPath text that should be evaluated against each Record. The value of the Property specifies
+ what value should go into that selected Record field.
+ </p>
+
+ <p>
+ When specifying the replacement value (the value of the User-defined Property), the user is able
+ to specify a literal value such as the number <code>10</code>; an Expression Language Expression
+ to reference FlowFile attributes, such as <code>${filename}</code>; or another RecordPath path
+ from which to retrieve the desired value from the Record itself. Whether the value entered should
+ be interpreted as a literal or a RecordPath path is determined by the value of the <Replacement
+ Value Strategy> Property.
+ </p>
+
+ <p>
+ If a RecordPath is given and does not match any field in an input Record, that Property will be skipped
+ and all other Properties will still be evaluated. If the RecordPath matches exactly one field, that field
+ will be updated with the corresponding value. If multiple fields match the RecordPath, then all fields that
+ match will be updated. If the replacement value is itself a RecordPath that does not match, then a
+ <code>null</code> value will be set for the field. For instances where this is not the desired behavior,
+ RecordPath predicates can be used to filter the fields that match so that no fields will be selected.
+ See <a href="../../../../../html/record-path-guide.html#predicates">RecordPath Predicates</a> for more information.
+ </p>
+
+ <p>
+ Below, we lay out some examples in order to provide clarity about the Processor's behavior. For all of
+ the examples below, consider the example to operate on the following set of 2 (JSON) records:
+ </p>
+
+<code>
+<pre>
+ [{
+ "id": 17,
+ "name": "John",
+ "child": {
+ "id": "1"
+ },
+ "siblingIds": [4, 8],
+ "siblings": [
+ { "name": "Jeremy", "id": 4 },
+ { "name": "Julia", "id": 8 }
+ ]
+ },
+ {
+ "id": 98,
+ "name": "Jane",
+ "child": {
+ "id": 2
+ },
+ "gender": "F",
+ "siblingIds": [],
+ "siblings": []
+ }]
+</pre>
+</code>
+
+ <p>
+ For brevity, we will omit the corresponding schema and configuration of the RecordReader and
+ RecordWriter. Otherwise, consider the following set of Properties are configured for the Processor
+ and their associated outputs.
+ </p>
+
+ <h3>Example 1 - Replace with Literal</h3>
+
+ <p>
+ Here, we will replace the name of each Record with the name 'Jeremy' and set the gender to 'M':
+ </p>
+
+ <table>
+ <tr>
+ <th>Property Name</th>
+ <th>Property Value</th>
+ </tr>
+ <tr>
+ <td>Replacement Value Strategy</td>
+ <td>Literal Value</td>
+ </tr>
+ <tr>
+ <td>/name</td>
+ <td>Jeremy</td>
+ </tr>
+ <tr>
+ <td>/gender</td>
+ <td>M</td>
+ </tr>
+ </table>
+
+ <p>
+ This will yield the following output:
+ </p>
+
+<code>
+<pre>
+ [{
+ "id": 17,
+ "name": "Jeremy",
+ "child": {
+ "id": "1"
+ },
+ "gender": "M",
+ "siblingIds": [4, 8],
+ "siblings": [
+ { "name": "Jeremy", "id": 4 },
+ { "name": "Julia", "id": 8 }
+ ]
+ },
+ {
+ "id": 98,
+ "name": "Jeremy",
+ "child": {
+ "id": 2
+ },
+ "gender": "M",
+ "siblingIds": [],
+ "siblings": []
+ }]
+</pre>
+</code>
+
+ <p>
+ Note that even though the first record did not have a "gender" field in the input, one
+ will be added after the "child" field, as that's where the field is located in the schema.
+ </p>
+
+
+ <h3>Example 2 - Replace with RecordPath</h3>
+
+ <p>
+ This example will replace the value in one field of the Record with the value from another field.
+ For this example, consider the following set of Properties:
+ </p>
+
+ <table>
+ <tr>
+ <th>Property Name</th>
+ <th>Property Value</th>
+ </tr>
+ <tr>
+ <td>Replacement Value Strategy</td>
+ <td>Record Path Value</td>
+ </tr>
+ <tr>
+ <td>/name</td>
+ <td>/siblings[0]/name</td>
+ </tr>
+ </table>
+
+ <p>
+ This will yield the following output:
+ </p>
+
+<code>
+<pre>
+ [{
+ "id": 17,
+ "name": "Jeremy",
+ "child": {
+ "id": "1"
+ },
+ "siblingIds": [4, 8],
+ "siblings": [
+ { "name": "Jeremy", "id": 4 },
+ { "name": "Julia", "id": 8 }
+ ]
+ },
+ {
+ "id": 98,
+ "name": null,
+ "child": {
+ "id": 2
+ },
+ "gender": "F",
+ "siblingIds": [],
+ "siblings": []
+ }]
+</pre>
+</code>
+
+
+
+ <h3>Example 3 - Replace with Relative RecordPath</h3>
+
+ <p>
+ In the above example, we replaced the value of field based on another RecordPath. That RecordPath was an "absolute RecordPath,"
+ meaning that it starts with a "slash" character (<code>/</code>) and therefore it specifies the path from the "root" or "outer most" element.
+ However, sometimes we want to reference a field in such a way that we defined the RecordPath relative to the field being updated. This example
+ does just that. For each of the siblings given in the "siblings" array, we will replace the sibling's name with their id's. To do so, we will
+ configure the processor with the following properties:
+ </p>
+
+ <table>
+ <tr>
+ <th>Property Name</th>
+ <th>Property Value</th>
+ </tr>
+ <tr>
+ <td>Replacement Value Strategy</td>
+ <td>Record Path Value</td>
+ </tr>
+ <tr>
+ <td>/siblings[*]/name</td>
+ <td>../id</td>
+ </tr>
+ </table>
+
+ <p>
+ Note that the RecordPath that was given for the value starts with <code>..</code>, which is a reference to the parent. We do this because the field
+ that we are going to update is the "name" field of the sibling. To get to the associated "id" field, we need to go to the "name" field's parent and then
+ to its "id" child field. The above example results in the following output:
+ </p>
+
+<code>
+<pre>
+ [{
+ "id": 17,
+ "name": "John",
+ "child": {
+ "id": "1"
+ },
+ "siblingIds": [4, 8],
+ "siblings": [
+ { "name": "4", "id": 4 },
+ { "name": "8", "id": 8 }
+ ]
+ },
+ {
+ "id": 98,
+ "name": "Jane",
+ "child": {
+ "id": 2
+ },
+ "gender": "F",
+ "siblingIds": [],
+ "siblings": []
+ }]
+</pre>
+</code>
+
+
+
+
+ <h3>Example 4 - Replace Multiple Values</h3>
+
+ <p>
+ This example will replace the value of all fields that have the name "id", regardless of
+ where in the Record hierarchy the field is found. The value that it uses references the Expression Language,
+ so for this example, let's assume that the incoming FlowFile has an attribute named "replacement.id" that
+ has a value of "91":
+ </p>
+
+ <table>
+ <tr>
+ <th>Property Name</th>
+ <th>Property Value</th>
+ </tr>
+ <tr>
+ <td>Replacement Value Strategy</td>
+ <td>Literal Value</td>
+ </tr>
+ <tr>
+ <td>//id</td>
+ <td>${replacement.id}</td>
+ </tr>
+ </table>
+
+ <p>
+ This will yield the following output:
+ </p>
+
+<code>
+<pre>
+ [{
+ "id": 91,
+ "name": "John",
+ "child": {
+ "id": "91"
+ },
+ "siblingIds": [4, 8],
+ "siblings": [
+ { "name": "Jeremy", "id": 91 },
+ { "name": "Julia", "id": 91 }
+ ]
+ },
+ {
+ "id": 91,
+ "name": "Jane",
+ "child": {
+ "id": 91
+ },
+ "gender": "F",
+ "siblingIds": [],
+ "siblings": []
+ }]
+</pre>
+</code>
+
+ <p>
+ It is also worth noting that in this example, some of the "id" fields were of type STRING, while
+ others were of type INT. This is okay because the RecordReaders and RecordWriters should handle
+ these simple type coercions for us.
+ </p>
+
+
+
+
+ <h3>Example 5 - Use Expression Language to Modify Value</h3>
+
+ <p>
+ This example will capitalize the value of all 'name' fields, regardless of
+ where in the Record hierarchy the field is found. This is done by referencing the 'field.value' variable in the Expression Language.
+ We can also access the field.name variable and the field.type variable.
+ </p>
+
+ <table>
+ <tr>
+ <th>Property Name</th>
+ <th>Property Value</th>
+ </tr>
+ <tr>
+ <td>Replacement Value Strategy</td>
+ <td>Literal Value</td>
+ </tr>
+ <tr>
+ <td>//name</td>
+ <td>${field.value:toUpper()}</td>
+ </tr>
+ </table>
+
+ <p>
+ This will yield the following output:
+ </p>
+
+<code>
+<pre>
+ [{
+ "id": 17,
+ "name": "JOHN",
+ "child": {
+ "id": "1"
+ },
+ "siblingIds": [4, 8],
+ "siblings": [
+ { "name": "JEREMY", "id": 4 },
+ { "name": "JULIA", "id": 8 }
+ ]
+ },
+ {
+ "id": 98,
+ "name": "JANE",
+ "child": {
+ "id": 2
+ },
+ "gender": "F",
+ "siblingIds": [],
+ "siblings": []
+ }]
+</pre>
+</code>
+
+
+ </body>
+</html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateRecord/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateRecord/index.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateRecord/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateRecord/index.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>UpdateRecord</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">UpdateRecord</h1><h2>Description: </h2><p>Updates the contents of a FlowFile that contains Record-oriented data (i.e., data that can be read via a RecordReader and written by a RecordWriter). This Processor requires that at least one user-defined Property be added. The name of the Property should indicate a RecordPath that determines the field that should be updated. The value of the Property is either a replacement value (optionally making use of the Expression Language) or is itself a RecordPath that extracts a value from the Record. Whether the Property value is determined to be a RecordPath or
a literal value depends on the configuration of the <Replacement Value Strategy> Property.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>update, record, generic, schema, json, csv, avro, log, logs, freeform, text</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Record Reader</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordReaderFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.csv.CSVReader/index.html">CSVReader</a><br/><a href="../../../nifi-scripting-nar/1.5.0/org.apache.nifi.record.script
.ScriptedReader/index.html">ScriptedReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.grok.GrokReader/index.html">GrokReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.json.JsonTreeReader/index.html">JsonTreeReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.avro.AvroReader/index.html">AvroReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.json.JsonPathReader/index.html">JsonPathReader</a></td><td id="description">Specifies the Controller Service to use for reading incoming data</td></tr><tr><td id="name"><strong>Record Writer</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordSetWriterFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.text.FreeFormTextRecordSetWriter/inde
x.html">FreeFormTextRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.csv.CSVRecordSetWriter/index.html">CSVRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.avro.AvroRecordSetWriter/index.html">AvroRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.json.JsonRecordSetWriter/index.html">JsonRecordSetWriter</a><br/><a href="../../../nifi-scripting-nar/1.5.0/org.apache.nifi.record.script.ScriptedRecordSetWriter/index.html">ScriptedRecordSetWriter</a></td><td id="description">Specifies the Controller Service to use for writing out the records</td></tr><tr><td id="name"><strong>Replacement Value Strategy</strong></td><td id="default-value">literal-value</td><td id="allowable-values"><ul><li>Literal Value <img src="../../../../../html/images/iconInfo.png" alt="The value entered for a Property (after Expression Language has been evalu
ated) is the desired value to update the Record Fields with. Expression Language may reference variables 'field.name', 'field.type', and 'field.value' to access information about the field and the value of the field being evaluated." title="The value entered for a Property (after Expression Language has been evaluated) is the desired value to update the Record Fields with. Expression Language may reference variables 'field.name', 'field.type', and 'field.value' to access information about the field and the value of the field being evaluated."></img></li><li>Record Path Value <img src="../../../../../html/images/iconInfo.png" alt="The value entered for a Property (after Expression Language has been evaluated) is not the literal value to use but rather is a Record Path that should be evaluated against the Record, and the result of the RecordPath will be used to update the Record. Note that if this option is selected, and the Record Path results in multiple values for a given Record, t
he input FlowFile will be routed to the 'failure' Relationship." title="The value entered for a Property (after Expression Language has been evaluated) is not the literal value to use but rather is a Record Path that should be evaluated against the Record, and the result of the RecordPath will be used to update the Record. Note that if this option is selected, and the Record Path results in multiple values for a given Record, the input FlowFile will be routed to the 'failure' Relationship."></img></li></ul></td><td id="description">Specifies how to interpret the configured replacement values</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles that are successfully transformed will be routed to this relationship</td></tr><tr><td>failure</td><td>If a FlowFile cannot be transformed from the configured input format to the configured output format, the unchanged FlowFile will be routed to this rela
tionship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.standard.ConvertRecord/index.html">ConvertRecord</a></p></body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1,109 @@
+<!DOCTYPE html>
+<html lang="en">
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<head>
+ <meta charset="utf-8"/>
+ <title>ValidateCsv</title>
+ <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"/>
+</head>
+
+<body>
+<!-- Processor Documentation ================================================== -->
+<h2>Usage Information</h2>
+
+<p>
+ The Validate CSV processor is based on the super-csv library and the concept of
+ <a href="http://super-csv.github.io/super-csv/cell_processors.html" target="_blank">Cell Processors</a>.
+ The corresponding java documentation can be found
+ <a href="http://super-csv.github.io/super-csv/apidocs/org/supercsv/cellprocessor/ift/CellProcessor.html" target="_blank">here</a>.
+</p>
+
+<p>
+ The cell processors cannot be nested (except with Optional which gives the possibility to define a CellProcessor for values
+ that could be null) and must be defined in a comma-delimited string as the Schema property.
+</p>
+
+<p>
+ The supported cell processors are:
+ <ul>
+ <li>ParseBigDecimal</li>
+ <li>ParseBool</li>
+ <li>ParseChar</li>
+ <li>ParseDate</li>
+ <li>ParseDouble</li>
+ <li>ParseInt</li>
+ <li>Optional</li>
+ <li>DMinMax</li>
+ <li>Equals</li>
+ <li>ForbidSubStr</li>
+ <li>LMinMax</li>
+ <li>NotNull</li>
+ <li>Null</li>
+ <li>RequireHashCode</li>
+ <li>RequireSubStr</li>
+ <li>Strlen</li>
+ <li>StrMinMax</li>
+ <li>StrNotNullOrEmpty</li>
+ <li>StrRegEx</li>
+ <li>Unique</li>
+ <li>UniqueHashCode</li>
+ <li>IsIncludedIn</li>
+ </ul>
+</p>
+
+<p>
+ Here are some examples:
+
+ <ul>
+ <b>Schema property:</b> Null, ParseDate("dd/MM/yyyy"), Optional(ParseDouble())<br />
+ <b>Meaning:</b> the input CSV has three columns, the first one can be null and has no specification, the second one must be a date
+ formatted as expected, and the third one must a double or null (no value).
+ </ul>
+
+ <ul>
+ <b>Schema property:</b> ParseBigDecimal(), ParseBool(), ParseChar(), ParseInt(), ParseLong()<br />
+ <b>Meaning:</b> the input CSV has five columns, the first one must be a big decimal, the second one must be a boolean,
+ the third one must be a char, the fourth one must be an integer and the fifth one must be a long.
+ </ul>
+
+ <ul>
+ <b>Schema property:</b> Equals(), NotNull(), StrNotNullOrEmpty()<br />
+ <b>Meaning:</b> the input CSV has three columns, all the values of the first column must be equal to each other, all the values
+ of the second column must be not null, and all the values of the third column are not null/empty string values.
+ </ul>
+
+ <ul>
+ <b>Schema property:</b> Strlen(4), StrMinMax(3,5), StrRegex("[a-z0-9\\._]+@[a-z0-9\\.]+")<br />
+ <b>Meaning:</b> the input CSV has three columns, all the values of the first column must be 4-characters long, all the values
+ of the second column must be between 3 and 5 characters (inclusive), and all the values of the last column must match
+ the provided regular expression (email address).
+ </ul>
+
+ <ul>
+ <b>Schema property:</b> Unique(), UniqueHashCode()<br />
+ <b>Meaning:</b> the input CSV has two columns. All the values of the first column must be unique (all the values are stored in
+ memory and this can be consuming depending of the input). All the values of the second column must be unique (only hash
+ codes of the input values are stored to ensure uniqueness).
+ </ul>
+
+ <ul>
+ <b>Schema property:</b> ForbidSubStr("test", "tset"), RequireSubStr("test")<br />
+ <b>Meaning:</b> the input CSV has two columns. None of the values in the first column must contain one of the provided strings.
+ And all the values of the second column must contain the provided string.
+ </ul>
+</p>
+</body>
+</html>
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateCsv/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateCsv/index.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateCsv/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateCsv/index.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ValidateCsv</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ValidateCsv</h1><h2>Description: </h2><p>Validates the contents of FlowFiles against a user-specified CSV schema. Take a look at the additional documentation of this processor for some schema examples.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>csv, schema, validation</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowabl
e Values</th><th>Description</th></tr><tr><td id="name"><strong>Schema</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The schema to be used for validation. Is expected a comma-delimited string representing the cell processors to apply. The following cell processors are allowed in the schema definition: [ParseBigDecimal, ParseBool, ParseChar, ParseDate, ParseDouble, ParseInt, ParseLong, Optional, DMinMax, Equals, ForbidSubStr, LMinMax, NotNull, Null, RequireHashCode, RequireSubStr, Strlen, StrMinMax, StrNotNullOrEmpty, StrRegEx, Unique, UniqueHashCode, IsIncludedIn]. Note: cell processors cannot be nested except with Optional.</td></tr><tr><td id="name"><strong>Header</strong></td><td id="default-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">True if the incoming flow file contains a header to ignore, false otherwise.</td></tr><tr><td id="name"><strong>Delimiter character</strong></
td><td id="default-value">,</td><td id="allowable-values"></td><td id="description">Character used as 'delimiter' in the incoming data. Example: ,</td></tr><tr><td id="name"><strong>Quote character</strong></td><td id="default-value">"</td><td id="allowable-values"></td><td id="description">Character used as 'quote' in the incoming data. Example: "</td></tr><tr><td id="name"><strong>End of line symbols</strong></td><td id="default-value">\n</td><td id="allowable-values"></td><td id="description">Symbols used as 'end of line' in the incoming data. Example: \n</td></tr><tr><td id="name"><strong>Validation strategy</strong></td><td id="default-value">FlowFile validation</td><td id="allowable-values"><ul><li>Line by line validation <img src="../../../../../html/images/iconInfo.png" alt="In case an error is found, the input CSV file will be split into two FlowFiles: one routed to the 'valid' relationship containing all the correct lines and one routed to the 'invalid' relationship contai
ning all the incorrect lines. Take care if choosing this option while using Unique cell processors in schema definition:the first occurrence will be considered valid and the next ones as invalid." title="In case an error is found, the input CSV file will be split into two FlowFiles: one routed to the 'valid' relationship containing all the correct lines and one routed to the 'invalid' relationship containing all the incorrect lines. Take care if choosing this option while using Unique cell processors in schema definition:the first occurrence will be considered valid and the next ones as invalid."></img></li><li>FlowFile validation <img src="../../../../../html/images/iconInfo.png" alt="As soon as an error is found in the CSV file, the validation will stop and the whole flow file will be routed to the 'invalid' relationship. This option offers best performances." title="As soon as an error is found in the CSV file, the validation will stop and the whole flow file will be routed to th
e 'invalid' relationship. This option offers best performances."></img></li></ul></td><td id="description">Strategy to apply when routing input files to output relationships.</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>valid</td><td>FlowFiles that are successfully validated against the schema are routed to this relationship</td></tr><tr><td>invalid</td><td>FlowFiles that are not valid according to the specified schema are routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>count.valid.lines</td><td>If line by line validation, number of valid lines extracted from the source data</td></tr><tr><td>count.invalid.lines</td><td>If line by line validation, number of invalid lines extracted from the source data</td></tr><tr><td>count.total.lines</td><td>If line by line validatio
n, total number of lines in the source data</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateRecord/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateRecord/index.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateRecord/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateRecord/index.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ValidateRecord</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ValidateRecord</h1><h2>Description: </h2><p>Validates the Records of an incoming FlowFile against a given schema. All records that adhere to the schema are routed to the "valid" relationship while records that do not adhere to the schema are routed to the "invalid" relationship. It is therefore possible for a single incoming FlowFile to be split into two individual FlowFiles if some records are valid according to the schema and others are not. Any FlowFile that is routed to the "invalid" relationship will emit a ROUTE Provenance Event with the Details field populated to explain why records were i
nvalid. In addition, to gain further explanation of why records were invalid, DEBUG-level logging can be enabled for the "org.apache.nifi.processors.standard.ValidateRecord" logger.</p><h3>Tags: </h3><p>record, schema, validate</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Record Reader</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordReaderFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.csv.CSVReader/index.
html">CSVReader</a><br/><a href="../../../nifi-scripting-nar/1.5.0/org.apache.nifi.record.script.ScriptedReader/index.html">ScriptedReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.grok.GrokReader/index.html">GrokReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.json.JsonTreeReader/index.html">JsonTreeReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.avro.AvroReader/index.html">AvroReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.json.JsonPathReader/index.html">JsonPathReader</a></td><td id="description">Specifies the Controller Service to use for reading incoming data</td></tr><tr><td id="name"><strong>Record Writer</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordSetWriterFactory<br/><strong>Implementations: </strong><a href="../../../ni
fi-record-serialization-services-nar/1.5.0/org.apache.nifi.text.FreeFormTextRecordSetWriter/index.html">FreeFormTextRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.csv.CSVRecordSetWriter/index.html">CSVRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.avro.AvroRecordSetWriter/index.html">AvroRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.5.0/org.apache.nifi.json.JsonRecordSetWriter/index.html">JsonRecordSetWriter</a><br/><a href="../../../nifi-scripting-nar/1.5.0/org.apache.nifi.record.script.ScriptedRecordSetWriter/index.html">ScriptedRecordSetWriter</a></td><td id="description">Specifies the Controller Service to use for writing out the records</td></tr><tr><td id="name"><strong>Schema Access Strategy</strong></td><td id="default-value">reader-schema</td><td id="allowable-values"><ul><li>Use Reader's Schema <img src="../../../../../html/im
ages/iconInfo.png" alt="The schema to validate the data against is determined by asking the configured Record Reader for its schema" title="The schema to validate the data against is determined by asking the configured Record Reader for its schema"></img></li><li>Use Schema Name Property <img src="../../../../../html/images/iconInfo.png" alt="The schema to validate the data against is determined by looking at the 'Schema Name' Property and looking up the schema in the configured Schema Registry" title="The schema to validate the data against is determined by looking at the 'Schema Name' Property and looking up the schema in the configured Schema Registry"></img></li><li>Use Schema Text Property <img src="../../../../../html/images/iconInfo.png" alt="The schema to validate the data against is determined by looking at the 'Schema Text' Property and parsing the schema as an Avro schema" title="The schema to validate the data against is determined by looking at the 'Schema Text' Propert
y and parsing the schema as an Avro schema"></img></li></ul></td><td id="description">Specifies how to obtain the schema that should be used to validate records</td></tr><tr><td id="name">Schema Registry</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SchemaRegistry<br/><strong>Implementations: </strong><a href="../../../nifi-hwx-schema-registry-nar/1.5.0/org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry/index.html">HortonworksSchemaRegistry</a><br/><a href="../../../nifi-registry-nar/1.5.0/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html">AvroSchemaRegistry</a><br/><a href="../../../nifi-confluent-platform-nar/1.5.0/org.apache.nifi.confluent.schemaregistry.ConfluentSchemaRegistry/index.html">ConfluentSchemaRegistry</a></td><td id="description">Specifies the Controller Service to use for the Schema Registry. This is necessary only if the Schema Access Strategy is set to "Use 'Schema Name'
Property".</td></tr><tr><td id="name">Schema Name</td><td id="default-value">${schema.name}</td><td id="allowable-values"></td><td id="description">Specifies the name of the schema to lookup in the Schema Registry property<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name">Schema Text</td><td id="default-value">${avro.schema}</td><td id="allowable-values"></td><td id="description">The text of an Avro-formatted Schema<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name"><strong>Allow Extra Fields</strong></td><td id="default-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If the incoming data has fields that are not present in the schema, this property determines whether or not the Record is valid. If true, the Record is still valid. If false, the Record will be invalid due to the extra fields.</td></tr><tr><td id="name"><strong>Strict Type Checking</strong></td><td
id="default-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If the incoming data has a Record where a field is not of the correct type, this property determine whether how to handle the Record. If true, the Record will still be considered invalid. If false, the Record will be considered valid and the field will be coerced into the correct type (if possible, according to the type coercion supported by the Record Writer).</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>valid</td><td>Records that are valid according to the schema will be routed to this relationship</td></tr><tr><td>invalid</td><td>Records that are not valid according to the schema will be routed to this relationship</td></tr><tr><td>failure</td><td>If the records cannot be read, validated, or written, for any reason, the original FlowFile will be routed to this relationship</td></tr></table><h3>R
eads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>mime.type</td><td>Sets the mime.type attribute to the MIME Type specified by the Record Writer</td></tr><tr><td>record.count</td><td>The number of records in the FlowFile routed to a relationship</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateXml/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateXml/index.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateXml/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.ValidateXml/index.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ValidateXml</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ValidateXml</h1><h2>Description: </h2><p>Validates the contents of FlowFiles against a user-specified XML Schema file</p><h3>Tags: </h3><p>xml, schema, validation, xsd</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Schema File</strong></td><td id="default-value"></td><td id="allowable-values"><
/td><td id="description">The path to the Schema file that is to be used for validation</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>valid</td><td>FlowFiles that are successfully validated against the schema are routed to this relationship</td></tr><tr><td>invalid</td><td>FlowFiles that are not valid according to the specified schema are routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>validatexml.invalid.error</td><td>If the flow file is routed to the invalid relationship the attribute will contain the error message resulting from the validation failure.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body>
</html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.Wait/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.Wait/index.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.Wait/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.Wait/index.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>Wait</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">Wait</h1><h2>Description: </h2><p>Routes incoming FlowFiles to the 'wait' relationship until a matching release signal is stored in the distributed cache from a corresponding Notify processor. When a matching release signal is identified, a waiting FlowFile is routed to the 'success' relationship, with attributes copied from the FlowFile that produced the release signal from the Notify processor. The release signal entry is then removed from the cache. Waiting FlowFiles will be routed to 'expired' if they exceed the Expiration Duration. If you need to wait for more than one signal, specify the desired num
ber of signals via the 'Target Signal Count' property. This is particularly useful with processors that split a source FlowFile into multiple fragments, such as SplitText. In order to wait for all fragments to be processed, connect the 'original' relationship to a Wait processor, and the 'splits' relationship to a corresponding Notify processor. Configure the Notify and Wait processors to use the '${fragment.identifier}' as the value of 'Release Signal Identifier', and specify '${fragment.count}' as the value of 'Target Signal Count' in the Wait processor.It is recommended to use a prioritizer (for instance First In First Out) when using the 'wait' relationship as a loop.</p><h3>Tags: </h3><p>map, cache, wait, hold, distributed, signal, release</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property su
pports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Release Signal Identifier</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">A value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the release signal cache key<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name"><strong>Target Signal Count</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">A value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the target signal count. This processor checks whether the signal count has reached this number. If Signal Counter Name is specif
ied, this processor checks a particular counter, otherwise checks against total count in a signal.<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name">Signal Counter Name</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">A value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the signal counter name. If not specified, this processor checks the total count in a signal.<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name"><strong>Wait Buffer Count</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">Specify the maximum number of incoming FlowFiles that can be buffered to check whether it can move forward. The more buffer can provide the better performance, as it reduces the number of interactions with cache service by grouping FlowFiles by signal identifier. Only a signal
identifier can be processed at a processor execution.</td></tr><tr><td id="name"><strong>Releasable FlowFile Count</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">A value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the releasable FlowFile count. This specifies how many FlowFiles can be released when a target count reaches target signal count. Zero (0) has a special meaning, any number of FlowFiles can be released as long as signal count matches target.<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name"><strong>Expiration Duration</strong></td><td id="default-value">10 min</td><td id="allowable-values"></td><td id="description">Indicates the duration after which waiting FlowFiles will be routed to the 'expired' relationship</td></tr><tr><td id="name"><strong>Distributed Cache Service</strong></td><td id="default-value"></td>
<td id="allowable-values"><strong>Controller Service API: </strong><br/>AtomicDistributedMapCacheClient<br/><strong>Implementations: </strong><a href="../../../nifi-distributed-cache-services-nar/1.5.0/org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService/index.html">DistributedMapCacheClientService</a><br/><a href="../../../nifi-redis-nar/1.5.0/org.apache.nifi.redis.service.RedisDistributedMapCacheClientService/index.html">RedisDistributedMapCacheClientService</a></td><td id="description">The Controller Service that is used to check for release signals from a corresponding Notify processor</td></tr><tr><td id="name"><strong>Attribute Copy Mode</strong></td><td id="default-value">keeporiginal</td><td id="allowable-values"><ul><li>Replace if present <img src="../../../../../html/images/iconInfo.png" alt="When cached attributes are copied onto released FlowFiles, they replace any matching attributes." title="When cached attributes are copied onto released FlowFiles
, they replace any matching attributes."></img></li><li>Keep original <img src="../../../../../html/images/iconInfo.png" alt="Attributes on released FlowFiles are not overwritten by copied cached attributes." title="Attributes on released FlowFiles are not overwritten by copied cached attributes."></img></li></ul></td><td id="description">Specifies how to handle attributes copied from FlowFiles entering the Notify processor</td></tr><tr><td id="name"><strong>Wait Mode</strong></td><td id="default-value">wait</td><td id="allowable-values"><ul><li>Transfer to wait relationship <img src="../../../../../html/images/iconInfo.png" alt="Transfer a FlowFile to the 'wait' relationship when whose release signal has not been notified yet. This mode allows other incoming FlowFiles to be enqueued by moving FlowFiles into the wait relationship. It is recommended to set a prioritizer (for instance First In First Out) on the 'wait' relationship." title="Transfer a FlowFile to the 'wait' relationshi
p when whose release signal has not been notified yet. This mode allows other incoming FlowFiles to be enqueued by moving FlowFiles into the wait relationship. It is recommended to set a prioritizer (for instance First In First Out) on the 'wait' relationship."></img></li><li>Keep in the upstream connection <img src="../../../../../html/images/iconInfo.png" alt="Transfer a FlowFile to the upstream connection where it comes from when whose release signal has not been notified yet. This mode helps keeping upstream connection being full so that the upstream source processor will not be scheduled while back-pressure is active and limit incoming FlowFiles. " title="Transfer a FlowFile to the upstream connection where it comes from when whose release signal has not been notified yet. This mode helps keeping upstream connection being full so that the upstream source processor will not be scheduled while back-pressure is active and limit incoming FlowFiles. "></img></li></ul></td><td id="de
scription">Specifies how to handle a FlowFile waiting for a notify signal</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>expired</td><td>A FlowFile that has exceeded the configured Expiration Duration will be routed to this relationship</td></tr><tr><td>success</td><td>A FlowFile with a matching release signal in the cache will be routed to this relationship</td></tr><tr><td>wait</td><td>A FlowFile with no matching release signal in the cache will be routed to this relationship</td></tr><tr><td>failure</td><td>When the cache cannot be reached, or if the Release Signal Identifier evaluates to null or empty, FlowFiles will be routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>wait.start.timestamp</td><td>All FlowFiles will have an attribute 'wait.start.timestamp', which sets
the initial epoch timestamp when the file first entered this processor. This is used to determine the expiration time of the FlowFile.</td></tr><tr><td>wait.counter.<counterName></td><td>If a signal exists when the processor runs, each count value in the signal is copied.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>See Also:</h3><p><a href="../../../nifi-distributed-cache-services-nar/1.5.0/org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService/index.html">DistributedMapCacheClientService</a>, <a href="../../../nifi-distributed-cache-services-nar/1.5.0/org.apache.nifi.distributed.cache.server.map.DistributedMapCacheServer/index.html">DistributedMapCacheServer</a>, <a href="../org.apache.nifi.processors.standard.Notify/index.html">Notify</a></p></body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/additionalDetails.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/additionalDetails.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/additionalDetails.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/additionalDetails.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1,41 @@
+<!DOCTYPE html>
+<html lang="en">
+ <!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+ <head>
+ <meta charset="utf-8" />
+ <title>StandardGangliaReporter</title>
+ <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" />
+ </head>
+
+ <body>
+ <h2>Description:</h2>
+
+ <p>Reporting Task that reports metrics to a Ganglia server. The following metrics are reported:
+ </p>
+
+ <ul>
+ <li><strong>FlowFiles In (5 mins)</strong>: The number of FlowFiles received via Site-to-Site in the last 5 minutes</li>
+ <li><strong>Bytes In (5 mins)</strong>: The number of bytes received via Site-to-Site in the last 5 minutes</li>
+ <li><strong>FlowFiles Out (5 mins)</strong>: The number of FlowFiles pulled from Output Ports via Site-to-Site in the last 5 minutes</li>
+ <li><strong>Bytes Out (5 mins)</strong>: The number of bytes pulled from Output Ports via Site-to-Site in the last 5 minutes</li>
+ <li><strong>Bytes Read (5 mins)</strong>: The number of bytes read from disk by NiFi in the last 5 minutes</li>
+ <li><strong>Bytes Written (5 mins)</strong>: The number of bytes written to disk by NiFi in the last 5 minutes</li>
+ <li><strong>FlowFiles Queued</strong>: The total number of FlowFiles currently queued on the system at the point in time at which the Reporting Task is run</li>
+ <li><strong>Bytes Queued</strong>: The total number of bytes allocated by the FlowFiles that are currently queued on the system at the point in time at which the Reporting Task is run</li>
+ <li><strong>Active Threads</strong>: The number of threads actively running at the point in time at which the Reporting Task is run</li>
+ </ul>
+ </body>
+</html>
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/index.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/index.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>StandardGangliaReporter</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">StandardGangliaReporter</h1><h2>Description: </h2><p>Reports metrics to Ganglia so that Ganglia can be used for external monitoring of the application. Metrics reported include JVM Metrics (optional); the following 5-minute NiFi statistics: FlowFiles Received, Bytes Received, FlowFiles Sent, Bytes Sent, Bytes Read, Bytes Written, Total Task Duration; and the current values for FlowFiles Queued, Bytes Queued, and number of Active Threads.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>ganglia, stats</p><h3>Properties: </h3><p>In the list below, the n
ames of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Hostname</strong></td><td id="default-value">localhost</td><td id="allowable-values"></td><td id="description">The fully-qualified name of the host on which Ganglia is running</td></tr><tr><td id="name"><strong>Port</strong></td><td id="default-value">8649</td><td id="allowable-values"></td><td id="description">The Port on which Ganglia is listening for incoming connections</td></tr><tr><td id="name"><strong>Send JVM Metrics</strong></td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Specifies whether or not JVM Metrics should be gathered and sent, in addition to NiFi-specific metrics</td></tr></table><h3>Sta
te management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-stateful-analysis-nar/1.5.0/org.apache.nifi.processors.stateful.analysis.AttributeRollingWindow/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-stateful-analysis-nar/1.5.0/org.apache.nifi.processors.stateful.analysis.AttributeRollingWindow/index.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-stateful-analysis-nar/1.5.0/org.apache.nifi.processors.stateful.analysis.AttributeRollingWindow/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-stateful-analysis-nar/1.5.0/org.apache.nifi.processors.stateful.analysis.AttributeRollingWindow/index.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>AttributeRollingWindow</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">AttributeRollingWindow</h1><h2>Description: </h2><p>Track a Rolling Window based on evaluating an Expression Language expression on each FlowFile and add that value to the processor's state. Each FlowFile will be emitted with the count of FlowFiles and total aggregate value of values processed in the current time window.</p><h3>Tags: </h3><p>Attribute Expression Language, state, data science, rolling, window</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table a
lso indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Value to track</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The expression on which to evaluate each FlowFile. The result of the expression will be added to the rolling window value.<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name"><strong>Time window</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The time window on which to calculate the rolling window.</td></tr><tr><td id="name">Sub-window length</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">When set, values will be batched into sub-windows of the set length. This allows
for much larger length total windows to be set but sacrifices some precision. If this is not set (or is 0) then each value is stored in state with the timestamp of when it was received. After the length of time stated in Time window elaspes the value will be removed. If this is set, values will be batched together every X amount of time (where X is the time period set for this property) and removed all at once.</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>set state fail</td><td>When state fails to save when processing a FlowFile, the FlowFile is routed here.</td></tr><tr><td>success</td><td>All FlowFiles are successfully processed are routed here</td></tr><tr><td>failure</td><td>When a FlowFile fails for a reason other than failing to set state it is routed here.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</
th></tr><tr><td>rolling_window_value</td><td>The rolling window value (sum of all the values stored).</td></tr><tr><td>rolling_window_count</td><td>The count of the number of FlowFiles seen in the rolling window.</td></tr><tr><td>rolling_window_mean</td><td>The mean of the FlowFiles seen in the rolling window.</td></tr></table><h3>State management: </h3><table id="stateful"><tr><th>Scope</th><th>Description</th></tr><tr><td>LOCAL</td><td>Store the values backing the rolling window. This includes storing the individual values and their time-stamps or the batches of values and their counts.</td></tr></table><h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-tcp-nar/1.5.0/org.apache.nifi.processors.gettcp.GetTCP/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-tcp-nar/1.5.0/org.apache.nifi.processors.gettcp.GetTCP/index.html?rev=1821033&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-tcp-nar/1.5.0/org.apache.nifi.processors.gettcp.GetTCP/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-tcp-nar/1.5.0/org.apache.nifi.processors.gettcp.GetTCP/index.html Fri Jan 12 21:00:14 2018
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>GetTCP</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">GetTCP</h1><h2>Description: </h2><p>Connects over TCP to the provided endpoint(s). Received data will be written as content to the FlowFile</p><h3>Tags: </h3><p>get, fetch, poll, tcp, ingest, source, input</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Endpoint List</strong></td><td id="default-valu
e"></td><td id="allowable-values"></td><td id="description">A comma delimited list of the endpoints to connect to. The format should be <server_address>:<port>. Only one server will be connected to at a time, the others will be used as fail overs.</td></tr><tr><td id="name"><strong>Connection Attempt Count</strong></td><td id="default-value">3</td><td id="allowable-values"></td><td id="description">The number of times to try and establish a connection, before using a backup host if available. This same attempt count would be used for a backup host as well.</td></tr><tr><td id="name"><strong>Reconnect interval</strong></td><td id="default-value">5 sec</td><td id="allowable-values"></td><td id="description">The number of seconds to wait before attempting to reconnect to the endpoint.</td></tr><tr><td id="name">Receive Buffer Size</td><td id="default-value">16MB</td><td id="allowable-values"></td><td id="description">The size of the buffer to receive data in. Default 16384
(16MB).</td></tr><tr><td id="name"><strong>End of message delimiter byte</strong></td><td id="default-value">13</td><td id="allowable-values"></td><td id="description">Byte value which denotes end of message. Must be specified as integer within the valid byte range (-128 thru 127). For example, '13' = Carriage return and '10' = New line. Default '13'.</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>Partial</td><td>The relationship that all incomplete messages from the endpoint will be sent to. Incomplete message is the message that doesn't end with 'End of message delimiter byte'. This can happen when 'Receive Buffer Size' is smaller then the incoming message. If that happens that the subsequent message that completes the previous incomplete message will also end up in this relationship, after which subsequent 'complete' messages will go to 'success'.</td></tr><tr><td>Success</td><td>The relationship that all suces
sful messages from the endpoint will be sent to.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>source.endpoint</td><td>The address of the source endpoint the message came from</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component does not allow an incoming relationship.</body></html>
\ No newline at end of file