You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by al...@apache.org on 2019/02/22 01:03:50 UTC
svn commit: r1854109 [23/50] - in /nifi/site/trunk/docs/nifi-docs: ./
components/org.apache.nifi/nifi-ambari-nar/1.9.0/
components/org.apache.nifi/nifi-ambari-nar/1.9.0/org.apache.nifi.reporting.ambari.AmbariReportingTask/
components/org.apache.nifi/ni...
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.RunMongoAggregation/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.RunMongoAggregation/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.RunMongoAggregation/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.RunMongoAggregation/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>RunMongoAggregation</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">RunMongoAggregation</h1><h2>Description: </h2><p>A processor that runs an aggregation query whenever a flowfile is received.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>mongo, aggregation, aggregate</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id=
"properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name">Client Service</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>MongoDBClientService<br/><strong>Implementation: </strong><a href="../../../nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBControllerService/index.html">MongoDBControllerService</a></td><td id="description">If configured, this property will use the assigned client service for connection pooling.</td></tr><tr><td id="name">Mongo URI</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">MongoURI, typically of the form: mongodb://host1[:port1][,host2[:port2],...]<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Mongo Database Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="descrip
tion">The name of the database to use<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Mongo Collection Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the collection to use<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">SSL Context Service</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SSLContextService<br/><strong>Implementations: </strong><a href="../../../nifi-ssl-context-service-nar/1.9.0/org.apache.nifi.ssl.StandardRestrictedSSLContextService/index.html">StandardRestrictedSSLContextService</a><br/><a href="../../../nifi-ssl-context-service-nar/1.9.0/org.apache.nifi.ssl.StandardSSLContextService/index.html">StandardSSLContextService</a></td><td id="
description">The SSL Context Service used to provide client certificate information for TLS/SSL connections.</td></tr><tr><td id="name">Client Auth</td><td id="default-value">REQUIRED</td><td id="allowable-values"><ul><li>WANT</li><li>REQUIRED</li><li>NONE</li></ul></td><td id="description">Client authentication policy when connecting to secure (TLS/SSL) cluster. Possible values are REQUIRED, WANT, NONE. This property is only used when an SSL Context has been defined and enabled.</td></tr><tr><td id="name"><strong>Character Set</strong></td><td id="default-value">UTF-8</td><td id="allowable-values"></td><td id="description">Specifies the character set of the document data.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Query</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The aggregation query to be executed.<br/><strong>Suppor
ts Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>JSON Type</strong></td><td id="default-value">Extended</td><td id="allowable-values"><ul><li>Extended JSON <img src="../../../../../html/images/iconInfo.png" alt="Use MongoDB's "extended JSON". This is the JSON generated with toJson() on a MongoDB Document from the Java driver" title="Use MongoDB's "extended JSON". This is the JSON generated with toJson() on a MongoDB Document from the Java driver"></img></li><li>Standard JSON <img src="../../../../../html/images/iconInfo.png" alt="Generate a JSON document that conforms to typical JSON conventions instead of Mongo-specific conventions." title="Generate a JSON document that conforms to typical JSON conventions instead of Mongo-specific conventions."></img></li></ul></td><td id="description">By default, MongoDB's Java driver returns "extended JSON". Some of the features of th
is variant of JSON may cause problems for other JSON parsers that expect only standard JSON types and conventions. This configuration setting controls whether to use extended JSON or provide a clean view that conforms to standard JSON.</td></tr><tr><td id="name">Query Output Attribute</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">If set, the query will be written to a specified attribute on the output flowfiles.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Batch Size</td><td id="default-value">100</td><td id="allowable-values"></td><td id="description">The number of elements returned from the server in one batch.</td></tr><tr><td id="name">Results Per FlowFile</td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">How many results to put into a flowfile at once. The whole body will be treated as a JSON array o
f results.</td></tr><tr><td id="name">Date Format</td><td id="default-value">yyyy-MM-dd'T'HH:mm:ss'Z'</td><td id="allowable-values"></td><td id="description">The date format string to use for formatting Date fields that are returned from Mongo. It is only applied when the JSON output format is set to Standard JSON. Full documentation for format characters can be found here: https://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">SSL Context Service</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SSLContextService<br/><strong>Implementations: </strong><a href="../../../nifi-ssl-context-service-nar/1.9.0/org.apache.nifi.ssl.StandardRestrictedSSLContextService/index.html">StandardRestrictedSSLContextService</a><br/><a href="../../../nifi-ssl-context-service-nar/1.
9.0/org.apache.nifi.ssl.StandardSSLContextService/index.html">StandardSSLContextService</a></td><td id="description">The SSL Context Service used to provide client certificate information for TLS/SSL connections.</td></tr><tr><td id="name">Client Auth</td><td id="default-value">REQUIRED</td><td id="allowable-values"><ul><li>WANT</li><li>REQUIRED</li><li>NONE</li></ul></td><td id="description">Client authentication policy when connecting to secure (TLS/SSL) cluster. Possible values are REQUIRED, WANT, NONE. This property is only used when an SSL Context has been defined and enabled.</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>results</td><td>The result set of the aggregation will be sent to this relationship.</td></tr><tr><td>failure</td><td>The input flowfile gets sent to this relationship when the query fails.</td></tr><tr><td>original</td><td>The input flowfile gets sent to this relationship when the query su
cceeds.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component allows an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/additionalDetails.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/additionalDetails.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/additionalDetails.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/additionalDetails.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,32 @@
+<!DOCTYPE html>
+<html lang="en">
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<head>
+ <meta charset="utf-8" />
+ <title>DeleteGridFS</title>
+ <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" />
+</head>
+
+<body>
+<!-- Processor Documentation ================================================== -->
+<h2>Description:</h2>
+<p>
+ This processor retrieves one or more files from GridFS. The query to execute can be either provided in the query
+ configuration parameter or generated from the value pulled from the filename configuration parameter. Upon successful
+ execution, it will append the query that was executed as an attribute on the flowfile that was processed.
+</p>
+</body>
+</html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>DeleteGridFS</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">DeleteGridFS</h1><h2>Description: </h2><p>Deletes a file from GridFS using a file name or a query.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>gridfs, delete, mongodb</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Defau
lt Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Client Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>MongoDBClientService<br/><strong>Implementation: </strong><a href="../../../nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBControllerService/index.html">MongoDBControllerService</a></td><td id="description">The MongoDB client service to use for database connections.</td></tr><tr><td id="name"><strong>Mongo Database Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the database to use<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Bucket Name</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The GridFS bucket where the files will be stored. If left blank, it w
ill use the default value 'fs' that the MongoDB client driver uses.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">File Name</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the file in the bucket that is the target of this processor. GridFS file names do not include path information because GridFS does not sort files into folders within a bucket.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Query</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">A valid MongoDB query to use to find and delete one or more files from GridFS.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Query Output Attribute
</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">If set, the query will be written to a specified attribute on the output flowfiles.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>When the operation succeeds, the flowfile is sent to this relationship.</td></tr><tr><td>failure</td><td>When there is a failure processing the flowfile, it goes to this relationship.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/additionalDetails.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/additionalDetails.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/additionalDetails.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/additionalDetails.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,43 @@
+<!DOCTYPE html>
+<html lang="en">
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<head>
+ <meta charset="utf-8" />
+ <title>FetchGridFS</title>
+ <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" />
+</head>
+
+<body>
+<!-- Processor Documentation ================================================== -->
+<h2>Description:</h2>
+<p>
+ This processor retrieves one or more files from GridFS. The query can be provided in one of three ways:
+</p>
+
+<ul>
+ <li>Query configuration parameter.</li>
+ <li>Built for you by configuring the filename parameter. (Note: this is just a filename, Mongo queries cannot be
+ embedded in the field).</li>
+ <li>Retrieving the query from the flowfile contents.</li>
+</ul>
+
+<p>
+ The processor can also be configured to either commit only once at the end of a fetch operation or after each file
+ that is retrieved. Multiple commits is generally only necessary when retrieving a lot of data from GridFS as measured
+ in total data size, not file count, to ensure that the disks NiFi is using are not overloaded.
+</p>
+</body>
+</html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>FetchGridFS</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">FetchGridFS</h1><h2>Description: </h2><p>Retrieves one or more files from a GridFS bucket by file name or by a user-defined query.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>fetch, gridfs, mongo</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties
"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Client Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>MongoDBClientService<br/><strong>Implementation: </strong><a href="../../../nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBControllerService/index.html">MongoDBControllerService</a></td><td id="description">The MongoDB client service to use for database connections.</td></tr><tr><td id="name"><strong>Mongo Database Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the database to use<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Bucket Name</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The GridFS bucket where the files will be
stored. If left blank, it will use the default value 'fs' that the MongoDB client driver uses.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">File Name</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the file in the bucket that is the target of this processor.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Query</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">A valid MongoDB query to use to fetch one or more files from GridFS.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Query Output Attribute</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">If set, the
query will be written to a specified attribute on the output flowfiles.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Operation Mode</strong></td><td id="default-value">all-at-once</td><td id="allowable-values"><ul><li>Full Query Fetch <img src="../../../../../html/images/iconInfo.png" alt="Fetch the entire query result and then make it available to downstream processors." title="Fetch the entire query result and then make it available to downstream processors."></img></li><li>Stream Query Results <img src="../../../../../html/images/iconInfo.png" alt="As soon as the query start sending results to the downstream processors at regular intervals." title="As soon as the query start sending results to the downstream processors at regular intervals."></img></li></ul></td><td id="description">This option controls when results are made available to downstream processors. If Str
eam Query Results is enabled, provenance will not be tracked relative to the input flowfile if an input flowfile is received and starts the query. In Stream Query Results mode errors will be handled by sending a new flowfile with the original content and attributes of the input flowfile to the failure relationship. Streaming should only be used if there is reliable connectivity between MongoDB and NiFi.</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>When the operation succeeds, the flowfile is sent to this relationship.</td></tr><tr><td>failure</td><td>When there is a failure processing the flowfile, it goes to this relationship.</td></tr><tr><td>original</td><td>The original input flowfile goes to this relationship if the query does not cause an error</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Descriptio
n</th></tr><tr><td>gridfs.file.metadata</td><td>The custom metadata stored with a file is attached to this property if it exists.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/additionalDetails.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/additionalDetails.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/additionalDetails.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/additionalDetails.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,58 @@
+<!DOCTYPE html>
+<html lang="en">
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<head>
+ <meta charset="utf-8" />
+ <title>PutGridFS</title>
+ <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" />
+</head>
+
+<body>
+<!-- Processor Documentation ================================================== -->
+<h2>Description:</h2>
+<p>
+ This processor puts a file with one or more user-defined metadata values into GridFS in the configured bucket. It
+ allows the user to define how big each file chunk will be during ingestion and provides some ability to intelligently
+ attempt to enforce file uniqueness using filename or hash values instead of just relying on a database index.
+</p>
+<h3>GridFS File Attributes</h3>
+<p>
+ <em>PutGridFS</em> allows for flowfile attributes that start with a configured prefix to be added to the GridFS
+ document. These can be very useful later when working with GridFS for providing metadata about a file.
+</p>
+<h3>Chunk Size</h3>
+<p>
+ GridFS splits up file into chunks within Mongo documents as the file is ingested into the database. The chunk size
+ configuration parameter configures the maximum size of each chunk. This field should be left at its default value
+ unless there is a specific business case to increase or decrease it.
+</p>
+<h3>Uniqueness Enforcement</h3>
+<p>
+ There are four operating modes:
+</p>
+<ul>
+ <li>No enforcement at the application level.</li>
+ <li>Enforce by unique file name.</li>
+ <li>Enforce by unique hash value.</li>
+ <li>Use both hash and file name.</li>
+</ul>
+<p>
+ The hash value by default is taken from the attribute <em>hash.value</em> which can be generated by configuring a
+ <em>HashContent</em> processor upstream of <em>PutGridFS</em>. Both this and the name option use a query on the existing
+ data to see if a file matching that criteria exists before attempting to write the flowfile contents.
+</p>
+</body>
+</html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.9.0/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>PutGridFS</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">PutGridFS</h1><h2>Description: </h2><p>Writes a file to a GridFS bucket.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>mongo, gridfs, put, file, store</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allo
wable Values</th><th>Description</th></tr><tr><td id="name"><strong>Client Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>MongoDBClientService<br/><strong>Implementation: </strong><a href="../../../nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBControllerService/index.html">MongoDBControllerService</a></td><td id="description">The MongoDB client service to use for database connections.</td></tr><tr><td id="name"><strong>Mongo Database Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the database to use<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Bucket Name</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The GridFS bucket where the files will be stored. If left blank, it will use the default v
alue 'fs' that the MongoDB client driver uses.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>File Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the file in the bucket that is the target of this processor. GridFS file names do not include path information because GridFS does not sort files into folders within a bucket.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">File Properties Prefix</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Attributes that have this prefix will be added to the file stored in GridFS as metadata.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"
><strong>Enforce Uniqueness</strong></td><td id="default-value">none</td><td id="allowable-values"><ul><li>None <img src="../../../../../html/images/iconInfo.png" alt="No uniqueness will be enforced." title="No uniqueness will be enforced."></img></li><li>Both <img src="../../../../../html/images/iconInfo.png" alt="Both the filename and hash must be unique." title="Both the filename and hash must be unique."></img></li><li>Name <img src="../../../../../html/images/iconInfo.png" alt="Only the filename must be unique." title="Only the filename must be unique."></img></li><li>Hash <img src="../../../../../html/images/iconInfo.png" alt="Only the file hash must be unique." title="Only the file hash must be unique."></img></li></ul></td><td id="description">When enabled, this option will ensure that uniqueness is enforced on the bucket. It will do so by creating a MongoDB index that matches your selection. It should ideally be configured once when the bucket is created for the first time
because it could take a long time to build on an existing bucket wit a lot of data.</td></tr><tr><td id="name">Hash Attribute</td><td id="default-value">hash.value</td><td id="allowable-values"></td><td id="description">If uniquness enforcement is enabled and the file hash is part of the constraint, this must be set to an attribute that exists on all incoming flowfiles.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Chunk Size</strong></td><td id="default-value">256 KB</td><td id="allowable-values"></td><td id="description">Controls the maximum size of each chunk of a file uploaded into GridFS.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>When the operation succeeds, the flowfile is
sent to this relationship.</td></tr><tr><td>duplicate</td><td>Flowfiles that fail the duplicate check are sent to this relationship.</td></tr><tr><td>failure</td><td>When there is a failure processing the flowfile, it goes to this relationship.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBControllerService/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBControllerService/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBControllerService/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBControllerService/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>MongoDBControllerService</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">MongoDBControllerService</h1><h2>Description: </h2><p>Provides a controller service that configures a connection to MongoDB and provides access to that connection to other Mongo-related components.</p><h3>Tags: </h3><p>mongo, mongodb, service</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><ta
ble id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Mongo URI</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">MongoURI, typically of the form: mongodb://host1[:port1][,host2[:port2],...]<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">SSL Context Service</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SSLContextService<br/><strong>Implementations: </strong><a href="../../../nifi-ssl-context-service-nar/1.9.0/org.apache.nifi.ssl.StandardRestrictedSSLContextService/index.html">StandardRestrictedSSLContextService</a><br/><a href="../../../nifi-ssl-context-service-nar/1.9.0/org.apache.nifi.ssl.StandardSSLContextService/index.html">StandardSSLContextService</a></td><td id="description">The SSL Context Service used to provide
client certificate information for TLS/SSL connections.</td></tr><tr><td id="name">Client Auth</td><td id="default-value">REQUIRED</td><td id="allowable-values"><ul><li>WANT</li><li>REQUIRED</li><li>NONE</li></ul></td><td id="description">Client authentication policy when connecting to secure (TLS/SSL) cluster. Possible values are REQUIRED, WANT, NONE. This property is only used when an SSL Context has been defined and enabled.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBLookupService/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBLookupService/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBLookupService/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.9.0/org.apache.nifi.mongodb.MongoDBLookupService/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,4 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>MongoDBLookupService</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">MongoDBLookupService</h1><h2>Description: </h2><p>Provides a lookup service based around MongoDB. Each key that is specified
+will be added to a query as-is. For example, if you specify the two keys,
+user and email, the resulting query will be { "user": "tester", "email": "tester@test.com" }.
+The query is limited to the first result (findOne in the Mongo documentation). If no "Lookup Value Field" is specified then the entire MongoDB result document minus the _id field will be returned as a record.</p><h3>Tags: </h3><p>mongo, mongodb, lookup, record</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Schema Access Strategy</strong></td><td id="default-value">infer</td><td id="allowable-values"><ul><li>Use 'Schema Name' Property <img src="../../../../../html/images/iconInfo.png" alt="The name of the Schema to use is specified by the 'Schema Name' P
roperty. The value of this property is used to lookup the Schema in the configured Schema Registry service." title="The name of the Schema to use is specified by the 'Schema Name' Property. The value of this property is used to lookup the Schema in the configured Schema Registry service."></img></li><li>Use 'Schema Text' Property <img src="../../../../../html/images/iconInfo.png" alt="The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Text' property must be valid after substituting the expressions." title="The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Text' property must be valid after substituting the expressions."></img></li><li>Infer from Result</li></ul></td><td id="description">Specifies how to obtain the
schema that is to be used for interpreting the data.</td></tr><tr><td id="name">Schema Registry</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SchemaRegistry<br/><strong>Implementations: </strong><a href="../../../nifi-hwx-schema-registry-nar/1.9.0/org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry/index.html">HortonworksSchemaRegistry</a><br/><a href="../../../nifi-confluent-platform-nar/1.9.0/org.apache.nifi.confluent.schemaregistry.ConfluentSchemaRegistry/index.html">ConfluentSchemaRegistry</a><br/><a href="../../../nifi-registry-nar/1.9.0/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html">AvroSchemaRegistry</a></td><td id="description">Specifies the Controller Service to use for the Schema Registry</td></tr><tr><td id="name">Schema Name</td><td id="default-value">${schema.name}</td><td id="allowable-values"></td><td id="description">Specifies the name of the schema to lookup in the Sch
ema Registry property<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Schema Version</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the version of the schema to lookup in the Schema Registry. If not specified then the latest version of the schema will be retrieved.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Schema Branch</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the name of the branch to use when looking up the schema in the Schema Registry property. If the chosen Schema Registry does not support branching, this value will be ignored.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id
="name">Schema Text</td><td id="default-value">${avro.schema}</td><td id="allowable-values"></td><td id="description">The text of an Avro-formatted Schema<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Client Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>MongoDBClientService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.mongodb.MongoDBControllerService/index.html">MongoDBControllerService</a></td><td id="description">A MongoDB controller service to use with this lookup service.</td></tr><tr><td id="name"><strong>Mongo Database Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the database to use<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></t
d></tr><tr><td id="name"><strong>Mongo Collection Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the collection to use<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Lookup Value Field</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The field whose value will be returned when the lookup key(s) match a record. If not specified then the entire MongoDB result document minus the _id field will be returned as a record.</td></tr><tr><td id="name">Projection</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies a projection for limiting which fields will be returned.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>System Resource Considerations:</h3>None specifi
ed.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.9.0/org.apache.nifi.processors.mqtt.ConsumeMQTT/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.9.0/org.apache.nifi.processors.mqtt.ConsumeMQTT/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.9.0/org.apache.nifi.processors.mqtt.ConsumeMQTT/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.9.0/org.apache.nifi.processors.mqtt.ConsumeMQTT/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ConsumeMQTT</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ConsumeMQTT</h1><h2>Description: </h2><p>Subscribes to a topic and receives messages from an MQTT broker</p><h3>Tags: </h3><p>subscribe, MQTT, IOT, consume, listen</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the <strong>nifi.properties</strong> file has an entry for the property
<strong>nifi.sensitive.props.key</strong>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Broker URI</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The URI to use to connect to the MQTT broker (e.g. tcp://localhost:1883). The 'tcp', 'ssl', 'ws' and 'wss' schemes are supported. In order to use 'ssl', the SSL Context Service property must be set.</td></tr><tr><td id="name"><strong>Client ID</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">MQTT client ID to use</td></tr><tr><td id="name">Username</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Username to use when connecting to the broker</td></tr><tr><td id="name">Password</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Password to use when connecting to the broker<br/><strong>Sensit
ive Property: true</strong></td></tr><tr><td id="name">SSL Context Service</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SSLContextService<br/><strong>Implementations: </strong><a href="../../../nifi-ssl-context-service-nar/1.9.0/org.apache.nifi.ssl.StandardRestrictedSSLContextService/index.html">StandardRestrictedSSLContextService</a><br/><a href="../../../nifi-ssl-context-service-nar/1.9.0/org.apache.nifi.ssl.StandardSSLContextService/index.html">StandardSSLContextService</a></td><td id="description">The SSL Context Service used to provide client certificate information for TLS/SSL connections.</td></tr><tr><td id="name">Last Will Topic</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The topic to send the client's Last Will to. If the Last Will topic and message are not set then a Last Will will not be sent.</td></tr><tr><td id="name">Last Will Message</td><td id="default-value"></td><td
id="allowable-values"></td><td id="description">The message to send as the client's Last Will. If the Last Will topic and message are not set then a Last Will will not be sent.</td></tr><tr><td id="name">Last Will Retain</td><td id="default-value"></td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Whether to retain the client's Last Will. If the Last Will topic and message are not set then a Last Will will not be sent.</td></tr><tr><td id="name">Last Will QoS Level</td><td id="default-value"></td><td id="allowable-values"><ul><li>0 - At most once <img src="../../../../../html/images/iconInfo.png" alt="Best effort delivery. A message wonât be acknowledged by the receiver or stored and redelivered by the sender. This is often called âfire and forgetâ and provides the same guarantee as the underlying TCP protocol." title="Best effort delivery. A message wonât be acknowledged by the receiver or stored and redelivered by the s
ender. This is often called âfire and forgetâ and provides the same guarantee as the underlying TCP protocol."></img></li><li>1 - At least once <img src="../../../../../html/images/iconInfo.png" alt="Guarantees that a message will be delivered at least once to the receiver. The message can also be delivered more than once" title="Guarantees that a message will be delivered at least once to the receiver. The message can also be delivered more than once"></img></li><li>2 - Exactly once <img src="../../../../../html/images/iconInfo.png" alt="Guarantees that each message is received only once by the counterpart. It is the safest and also the slowest quality of service level. The guarantee is provided by two round-trip flows between sender and receiver." title="Guarantees that each message is received only once by the counterpart. It is the safest and also the slowest quality of service level. The guarantee is provided by two round-trip flows between sender and receiver."></img
></li></ul></td><td id="description">QoS level to be used when publishing the Last Will Message</td></tr><tr><td id="name"><strong>Session state</strong></td><td id="default-value">true</td><td id="allowable-values"><ul><li>Clean Session <img src="../../../../../html/images/iconInfo.png" alt="Client and Server discard any previous session and start a new one. This session lasts as long as the network connection. State data associated with this session is not reused in any subsequent session" title="Client and Server discard any previous session and start a new one. This session lasts as long as the network connection. State data associated with this session is not reused in any subsequent session"></img></li><li>Resume Session <img src="../../../../../html/images/iconInfo.png" alt="Server resumes communications with the client based on state from the current session (as identified by the ClientID). The client and server store the session after the client and server are disconnected.
After the disconnection of a session that was not a clean session, the server stores further QoS 1 and QoS 2 messages that match any subscriptions that the client had at the time of disconnection as part of the session state" title="Server resumes communications with the client based on state from the current session (as identified by the ClientID). The client and server store the session after the client and server are disconnected. After the disconnection of a session that was not a clean session, the server stores further QoS 1 and QoS 2 messages that match any subscriptions that the client had at the time of disconnection as part of the session state"></img></li></ul></td><td id="description">Whether to start afresh or resume previous flows. See the allowable value descriptions for more details.</td></tr><tr><td id="name"><strong>MQTT Specification Version</strong></td><td id="default-value">0</td><td id="allowable-values"><ul><li>AUTO <img src="../../../../../html/images/iconI
nfo.png" alt="Start with v3.1.1 and fallback to v3.1.0 if not supported by a broker" title="Start with v3.1.1 and fallback to v3.1.0 if not supported by a broker"></img></li><li>v3.1.1</li><li>v3.1.0</li></ul></td><td id="description">The MQTT specification version when connecting with the broker. See the allowable value descriptions for more details.</td></tr><tr><td id="name">Connection Timeout (seconds)</td><td id="default-value">30</td><td id="allowable-values"></td><td id="description">Maximum time interval the client will wait for the network connection to the MQTT server to be established. The default timeout is 30 seconds. A value of 0 disables timeout processing meaning the client will wait until the network connection is made successfully or fails.</td></tr><tr><td id="name">Keep Alive Interval (seconds)</td><td id="default-value">60</td><td id="allowable-values"></td><td id="description">Defines the maximum time interval between messages sent or received. It enables the c
lient to detect if the server is no longer available, without having to wait for the TCP/IP timeout. The client will ensure that at least one message travels across the network within each keep alive period. In the absence of a data-related message during the time period, the client sends a very small "ping" message, which the server will acknowledge. A value of 0 disables keepalive processing in the client.</td></tr><tr><td id="name"><strong>Topic Filter</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The MQTT topic filter to designate the topics to subscribe to.</td></tr><tr><td id="name"><strong>Quality of Service(QoS)</strong></td><td id="default-value">0</td><td id="allowable-values"><ul><li>0 - At most once <img src="../../../../../html/images/iconInfo.png" alt="Best effort delivery. A message wonât be acknowledged by the receiver or stored and redelivered by the sender. This is often called âfire and forgetâ and prov
ides the same guarantee as the underlying TCP protocol." title="Best effort delivery. A message wonât be acknowledged by the receiver or stored and redelivered by the sender. This is often called âfire and forgetâ and provides the same guarantee as the underlying TCP protocol."></img></li><li>1 - At least once <img src="../../../../../html/images/iconInfo.png" alt="Guarantees that a message will be delivered at least once to the receiver. The message can also be delivered more than once" title="Guarantees that a message will be delivered at least once to the receiver. The message can also be delivered more than once"></img></li><li>2 - Exactly once <img src="../../../../../html/images/iconInfo.png" alt="Guarantees that each message is received only once by the counterpart. It is the safest and also the slowest quality of service level. The guarantee is provided by two round-trip flows between sender and receiver." title="Guarantees that each message is received only o
nce by the counterpart. It is the safest and also the slowest quality of service level. The guarantee is provided by two round-trip flows between sender and receiver."></img></li></ul></td><td id="description">The Quality of Service(QoS) to receive the message with. Accepts values '0', '1' or '2'; '0' for 'at most once', '1' for 'at least once', '2' for 'exactly once'.</td></tr><tr><td id="name"><strong>Max Queue Size</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The MQTT messages are always being sent to subscribers on a topic. If the 'Run Schedule' is significantly behind the rate at which the messages are arriving to this processor then a back up can occur. This property specifies the maximum number of messages this processor will hold in memory at one time.</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>Message</td><td>The MQTT message output</td></tr></table><h3>
Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>mqtt.broker</td><td>MQTT broker that was the message source</td></tr><tr><td>mqtt.topic</td><td>MQTT topic on which message was received</td></tr><tr><td>mqtt.qos</td><td>The quality of service for this message.</td></tr><tr><td>mqtt.isDuplicate</td><td>Whether or not this message might be a duplicate of one which has already been received.</td></tr><tr><td>mqtt.isRetained</td><td>Whether or not this message was from a current publisher, or was "retained" by the server as the last message published on the topic.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component does not allow an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.m
qtt.PublishMQTT/index.html">PublishMQTT</a></p></body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.9.0/org.apache.nifi.processors.mqtt.PublishMQTT/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.9.0/org.apache.nifi.processors.mqtt.PublishMQTT/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.9.0/org.apache.nifi.processors.mqtt.PublishMQTT/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.9.0/org.apache.nifi.processors.mqtt.PublishMQTT/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>PublishMQTT</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">PublishMQTT</h1><h2>Description: </h2><p>Publishes a message to an MQTT topic</p><h3>Tags: </h3><p>publish, MQTT, IOT</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure
that the <strong>nifi.properties</strong> file has an entry for the property <strong>nifi.sensitive.props.key</strong>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Broker URI</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The URI to use to connect to the MQTT broker (e.g. tcp://localhost:1883). The 'tcp', 'ssl', 'ws' and 'wss' schemes are supported. In order to use 'ssl', the SSL Context Service property must be set.</td></tr><tr><td id="name"><strong>Client ID</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">MQTT client ID to use</td></tr><tr><td id="name">Username</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Username to use when connecting to the broker</td></tr><tr><td id="name">Password</td><td id="default-value"></td><td id="allowable-values"></td><td id="
description">Password to use when connecting to the broker<br/><strong>Sensitive Property: true</strong></td></tr><tr><td id="name">SSL Context Service</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SSLContextService<br/><strong>Implementations: </strong><a href="../../../nifi-ssl-context-service-nar/1.9.0/org.apache.nifi.ssl.StandardRestrictedSSLContextService/index.html">StandardRestrictedSSLContextService</a><br/><a href="../../../nifi-ssl-context-service-nar/1.9.0/org.apache.nifi.ssl.StandardSSLContextService/index.html">StandardSSLContextService</a></td><td id="description">The SSL Context Service used to provide client certificate information for TLS/SSL connections.</td></tr><tr><td id="name">Last Will Topic</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The topic to send the client's Last Will to. If the Last Will topic and message are not set then a Last Will will not be sent.</td>
</tr><tr><td id="name">Last Will Message</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The message to send as the client's Last Will. If the Last Will topic and message are not set then a Last Will will not be sent.</td></tr><tr><td id="name">Last Will Retain</td><td id="default-value"></td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Whether to retain the client's Last Will. If the Last Will topic and message are not set then a Last Will will not be sent.</td></tr><tr><td id="name">Last Will QoS Level</td><td id="default-value"></td><td id="allowable-values"><ul><li>0 - At most once <img src="../../../../../html/images/iconInfo.png" alt="Best effort delivery. A message wonât be acknowledged by the receiver or stored and redelivered by the sender. This is often called âfire and forgetâ and provides the same guarantee as the underlying TCP protocol." title="Best effort delivery. A message
wonât be acknowledged by the receiver or stored and redelivered by the sender. This is often called âfire and forgetâ and provides the same guarantee as the underlying TCP protocol."></img></li><li>1 - At least once <img src="../../../../../html/images/iconInfo.png" alt="Guarantees that a message will be delivered at least once to the receiver. The message can also be delivered more than once" title="Guarantees that a message will be delivered at least once to the receiver. The message can also be delivered more than once"></img></li><li>2 - Exactly once <img src="../../../../../html/images/iconInfo.png" alt="Guarantees that each message is received only once by the counterpart. It is the safest and also the slowest quality of service level. The guarantee is provided by two round-trip flows between sender and receiver." title="Guarantees that each message is received only once by the counterpart. It is the safest and also the slowest quality of service level. The guar
antee is provided by two round-trip flows between sender and receiver."></img></li></ul></td><td id="description">QoS level to be used when publishing the Last Will Message</td></tr><tr><td id="name"><strong>Session state</strong></td><td id="default-value">true</td><td id="allowable-values"><ul><li>Clean Session <img src="../../../../../html/images/iconInfo.png" alt="Client and Server discard any previous session and start a new one. This session lasts as long as the network connection. State data associated with this session is not reused in any subsequent session" title="Client and Server discard any previous session and start a new one. This session lasts as long as the network connection. State data associated with this session is not reused in any subsequent session"></img></li><li>Resume Session <img src="../../../../../html/images/iconInfo.png" alt="Server resumes communications with the client based on state from the current session (as identified by the ClientID). The clie
nt and server store the session after the client and server are disconnected. After the disconnection of a session that was not a clean session, the server stores further QoS 1 and QoS 2 messages that match any subscriptions that the client had at the time of disconnection as part of the session state" title="Server resumes communications with the client based on state from the current session (as identified by the ClientID). The client and server store the session after the client and server are disconnected. After the disconnection of a session that was not a clean session, the server stores further QoS 1 and QoS 2 messages that match any subscriptions that the client had at the time of disconnection as part of the session state"></img></li></ul></td><td id="description">Whether to start afresh or resume previous flows. See the allowable value descriptions for more details.</td></tr><tr><td id="name"><strong>MQTT Specification Version</strong></td><td id="default-value">0</td><td
id="allowable-values"><ul><li>AUTO <img src="../../../../../html/images/iconInfo.png" alt="Start with v3.1.1 and fallback to v3.1.0 if not supported by a broker" title="Start with v3.1.1 and fallback to v3.1.0 if not supported by a broker"></img></li><li>v3.1.1</li><li>v3.1.0</li></ul></td><td id="description">The MQTT specification version when connecting with the broker. See the allowable value descriptions for more details.</td></tr><tr><td id="name">Connection Timeout (seconds)</td><td id="default-value">30</td><td id="allowable-values"></td><td id="description">Maximum time interval the client will wait for the network connection to the MQTT server to be established. The default timeout is 30 seconds. A value of 0 disables timeout processing meaning the client will wait until the network connection is made successfully or fails.</td></tr><tr><td id="name">Keep Alive Interval (seconds)</td><td id="default-value">60</td><td id="allowable-values"></td><td id="description">Defines
the maximum time interval between messages sent or received. It enables the client to detect if the server is no longer available, without having to wait for the TCP/IP timeout. The client will ensure that at least one message travels across the network within each keep alive period. In the absence of a data-related message during the time period, the client sends a very small "ping" message, which the server will acknowledge. A value of 0 disables keepalive processing in the client.</td></tr><tr><td id="name"><strong>Topic</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The topic to publish the message to.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Quality of Service(QoS)</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The Quality of Service(QoS) to send the message with. Accept
s three values '0', '1' and '2'; '0' for 'at most once', '1' for 'at least once', '2' for 'exactly once'. Expression language is allowed in order to support publishing messages with different QoS but the end value of the property must be either '0', '1' or '2'. <br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Retain Message</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Whether or not the retain flag should be set on the MQTT message.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles that are sent successfully to the destination are transferred to this relationship.</td></tr><tr><td>failure</td><td>FlowFiles that
failed to send to the destination are transferred to this relationship.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3><table id="system-resource-considerations"><tr><th>Resource</th><th>Description</th></tr><tr><td>MEMORY</td><td>An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.</td></tr></table><h3>See Also:</h3><p><a href="../org.apache.nifi.processors.mqtt.ConsumeMQTT/index.html">ConsumeMQTT</a></p></body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-network-processors-nar/1.9.0/org.apache.nifi.processors.network.ParseNetflowv5/additionalDetails.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-network-processors-nar/1.9.0/org.apache.nifi.processors.network.ParseNetflowv5/additionalDetails.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-network-processors-nar/1.9.0/org.apache.nifi.processors.network.ParseNetflowv5/additionalDetails.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-network-processors-nar/1.9.0/org.apache.nifi.processors.network.ParseNetflowv5/additionalDetails.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,72 @@
+<!DOCTYPE html>
+<html lang="en">
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+<head>
+<meta charset="utf-8" />
+<title>Netflowv5Parser</title>
+<link rel="stylesheet" href="../../../../../css/component-usage.css"
+ type="text/css" />
+</head>
+
+<body>
+ <p>
+ Netflowv5Parser processor parses the ingress netflowv5 datagram format
+ and transfers it either as flowfile attributes or JSON object.
+ Netflowv5 format has predefined schema named "template" for parsing
+ the netflowv5 record. More information: <a title="RFC-netflowv5"
+ href="https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html">RFC-netflowv5</a>
+ </p>
+ <h2>Netflowv5 JSON Output Schema</h2>
+ <pre>{
+ "port": int,
+ "format": string,
+ "header": {
+ "version": int,
+ "count": int,
+ "sys_uptime": long,
+ "unix_secs": long,
+ "unix_nsecs": long,
+ "flow_sequence": long,
+ "engine_type": short,
+ "engine_id": short,
+ "sampling_interval": int
+ },
+ "record": {
+ "srcaddr": string,
+ "dstaddr": string,
+ "nexthop": string,
+ "input": int,
+ "output": int,
+ "dPkts": long,
+ "dOctets": long,
+ "first": long,
+ "last": long,
+ "srcport": int,
+ "dstport": int,
+ "pad1": short,
+ "tcp_flags": short,
+ "prot": short,
+ "tos": short,
+ "src_as": int,
+ "dst_as": int,
+ "src_mask": short,
+ "dst_mask": short,
+ "pad2": int
+ }
+}
+</pre>
+</body>
+</html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-network-processors-nar/1.9.0/org.apache.nifi.processors.network.ParseNetflowv5/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-network-processors-nar/1.9.0/org.apache.nifi.processors.network.ParseNetflowv5/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-network-processors-nar/1.9.0/org.apache.nifi.processors.network.ParseNetflowv5/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-network-processors-nar/1.9.0/org.apache.nifi.processors.network.ParseNetflowv5/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ParseNetflowv5</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ParseNetflowv5</h1><h2>Description: </h2><p>Parses netflowv5 byte ingest and add to NiFi flowfile as attributes or JSON content.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>network, netflow, attributes, datagram, v5, packet, byte</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr
><tr><td id="name"><strong>Parsed fields destination</strong></td><td id="default-value">flowfile-content</td><td id="allowable-values"><ul><li>flowfile-content <img src="../../../../../html/images/iconInfo.png" alt="Parsed data routes as flowfile JSON content" title="Parsed data routes as flowfile JSON content"></img></li><li>flowfile-attribute <img src="../../../../../html/images/iconInfo.png" alt="Parsed data routes as flowfile attributes" title="Parsed data routes as flowfile attributes"></img></li></ul></td><td id="description">Indicates whether the results of the parser are written to the FlowFile content or a FlowFile attribute; if using flowfile-attribute, fields will be populated as attributes. If set to flowfile-content, the netflowv5 field will be converted into a flat JSON object.</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>Any FlowFile that is successfully parsed as a netflowv5 data
will be transferred to this Relationship.</td></tr><tr><td>failure</td><td>Any FlowFile that could not be parsed as a netflowv5 message will be transferred to this Relationship without any attributes being added</td></tr><tr><td>original</td><td>The original raw content</td></tr></table><h3>Reads Attributes: </h3><table id="reads-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>udp.port</td><td>Optionally read if packets are received from UDP datagrams.</td></tr></table><h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>netflowv5.header.*</td><td>The key and value generated by the parsing of the header fields.</td></tr><tr><td>netflowv5.record.*</td><td>The key and value generated by the parsing of the record fields.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming r
elationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-parquet-nar/1.9.0/org.apache.nifi.processors.parquet.ConvertAvroToParquet/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-parquet-nar/1.9.0/org.apache.nifi.processors.parquet.ConvertAvroToParquet/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-parquet-nar/1.9.0/org.apache.nifi.processors.parquet.ConvertAvroToParquet/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-parquet-nar/1.9.0/org.apache.nifi.processors.parquet.ConvertAvroToParquet/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ConvertAvroToParquet</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ConvertAvroToParquet</h1><h2>Description: </h2><p>Converts Avro records into Parquet file format. The incoming FlowFile should be a valid avro file. If an incoming FlowFile does not contain any records, an empty parquet file is the output. NOTE: Many Avro datatypes (collections, primitives, and unions of primitives, e.g.) can be converted to parquet, but unions of collections and other complex datatypes may not be able to be converted to Parquet.</p><h3>Tags: </h3><p>avro, parquet, convert</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold<
/strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Compression Type</strong></td><td id="default-value">UNCOMPRESSED</td><td id="allowable-values"><ul><li>UNCOMPRESSED</li><li>SNAPPY</li><li>GZIP</li><li>LZO</li><li>BROTLI</li><li>LZ4</li><li>ZSTD</li></ul></td><td id="description">The type of compression for the file being written.</td></tr><tr><td id="name">Row Group Size</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The row group size used by the Parquet writer. The value is specified in the format of <Data Size> <Data Unit> where Data Unit is one of B, KB, MB, GB, TB.<br/><strong>Supports Expression Lang
uage: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Page Size</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The page size used by the Parquet writer. The value is specified in the format of <Data Size> <Data Unit> where Data Unit is one of B, KB, MB, GB, TB.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Dictionary Page Size</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The dictionary page size used by the Parquet writer. The value is specified in the format of <Data Size> <Data Unit> where Data Unit is one of B, KB, MB, GB, TB.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Max Padding Size</td><td id="default-valu
e"></td><td id="allowable-values"></td><td id="description">The maximum amount of padding that will be used to align row groups with blocks in the underlying filesystem. If the underlying filesystem is not a block filesystem like HDFS, this has no effect. The value is specified in the format of <Data Size> <Data Unit> where Data Unit is one of B, KB, MB, GB, TB.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Enable Dictionary Encoding</td><td id="default-value"></td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Specifies whether dictionary encoding should be enabled for the Parquet writer</td></tr><tr><td id="name">Enable Validation</td><td id="default-value"></td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Specifies whether validation should be enabled for the Parquet writer</td></t
r><tr><td id="name">Writer Version</td><td id="default-value"></td><td id="allowable-values"><ul><li>PARQUET_1_0</li><li>PARQUET_2_0</li></ul></td><td id="description">Specifies the version used by Parquet writer</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>Parquet file that was converted successfully from Avro</td></tr><tr><td>failure</td><td>Avro content that could not be processed</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename</td><td>Sets the filename to the existing filename with the extension replaced by / added to by .parquet</td></tr><tr><td>record.count</td><td>Sets the number of records in the parquet file.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement
: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file