You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by al...@apache.org on 2019/02/22 01:03:50 UTC
svn commit: r1854109 [10/50] - in /nifi/site/trunk/docs/nifi-docs: ./
components/org.apache.nifi/nifi-ambari-nar/1.9.0/
components/org.apache.nifi/nifi-ambari-nar/1.9.0/org.apache.nifi.reporting.ambari.AmbariReportingTask/
components/org.apache.nifi/ni...
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.9.0/org.apache.nifi.processors.enrich.QueryDNS/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.9.0/org.apache.nifi.processors.enrich.QueryDNS/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.9.0/org.apache.nifi.processors.enrich.QueryDNS/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.9.0/org.apache.nifi.processors.enrich.QueryDNS/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,2 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>QueryDNS</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">QueryDNS</h1><h2>Description: </h2><p>A powerful DNS query processor primary designed to enrich DataFlows with DNS based APIs (e.g. RBLs, ShadowServer's ASN lookup) but that can be also used to perform regular DNS lookups.</p><h3>Tags: </h3><p>dns, enrich, ip</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><t
able id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Lookup value</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The value that should be used to populate the query<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Results Parser</strong></td><td id="default-value">None</td><td id="allowable-values"><ul><li>Split <img src="../../../../../html/images/iconInfo.png" alt="Use a delimiter character or RegEx to split the results into attributes" title="Use a delimiter character or RegEx to split the results into attributes"></img></li><li>RegEx <img src="../../../../../html/images/iconInfo.png" alt="Use a regular expression to split the results into attributes " title="Use a regular expression to split the results into attributes "></img></li><li>None
<img src="../../../../../html/images/iconInfo.png" alt="Do not split results" title="Do not split results"></img></li></ul></td><td id="description">The method used to slice the results into attribute groups</td></tr><tr><td id="name">Parser RegEx</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Choice between a splitter and regex matcher used to parse the results of the query into attribute groups.
+NOTE: This is a multiline regular expression, therefore, the DFM should decide how to handle trailing new line characters.</td></tr><tr><td id="name"><strong>DNS Query Retries</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">The number of attempts before giving up and moving on</td></tr><tr><td id="name"><strong>DNS Query Timeout</strong></td><td id="default-value">1500 ms</td><td id="allowable-values"></td><td id="description">The amount of time to wait until considering a query as failed</td></tr><tr><td id="name">DNS Servers</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">A comma separated list of DNS servers to be used. (Defaults to system wide if none is used)</td></tr><tr><td id="name"><strong>DNS Query Type</strong></td><td id="default-value">TXT</td><td id="allowable-values"></td><td id="description">The DNS query type to be used by the processor (e.g. TXT, A)</td></tr></table><h3>Relationships
: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>not found</td><td>Where to route flow files if data enrichment query rendered no results</td></tr><tr><td>found</td><td>Where to route flow files after successfully enriching attributes with data</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>enrich.dns.record*.group*</td><td>The captured fields of the DNS query response for each of the records received</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.9.0/org.apache.nifi.processors.enrich.QueryWhois/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.9.0/org.apache.nifi.processors.enrich.QueryWhois/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.9.0/org.apache.nifi.processors.enrich.QueryWhois/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.9.0/org.apache.nifi.processors.enrich.QueryWhois/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,2 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>QueryWhois</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">QueryWhois</h1><h2>Description: </h2><p>A powerful whois query processor primary designed to enrich DataFlows with whois based APIs (e.g. ShadowServer's ASN lookup) but that can be also used to perform regular whois lookups.</p><h3>Tags: </h3><p>whois, enrich, ip</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.
</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Lookup value</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The value that should be used to populate the query<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Whois Query Type</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The Whois query type to be used by the processor (if used)</td></tr><tr><td id="name"><strong>Whois Server</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The Whois server to be used</td></tr><tr><td id="name"><strong>Whois Server Port</strong></td><td id="default-value">43</td><td id="allowable-values"></td><td id="description">The TCP port of the remote Whois server</td></tr><tr><td id="name"><
strong>Whois Query Timeout</strong></td><td id="default-value">1500 ms</td><td id="allowable-values"></td><td id="description">The amount of time to wait until considering a query as failed</td></tr><tr><td id="name"><strong>Batch Size</strong></td><td id="default-value">25</td><td id="allowable-values"></td><td id="description">The number of incoming FlowFiles to process in a single execution of this processor. </td></tr><tr><td id="name"><strong>Bulk Protocol</strong></td><td id="default-value">None</td><td id="allowable-values"><ul><li>Begin/End <img src="../../../../../html/images/iconInfo.png" alt="The evaluated input of each flowfile is enclosed within begin and end tags. Each row contains a delimited set of fields" title="The evaluated input of each flowfile is enclosed within begin and end tags. Each row contains a delimited set of fields"></img></li><li>None <img src="../../../../../html/images/iconInfo.png" alt="Queries are made without any particular dialect" title="Queri
es are made without any particular dialect"></img></li></ul></td><td id="description">The protocol used to perform the bulk query. </td></tr><tr><td id="name"><strong>Results Parser</strong></td><td id="default-value">None</td><td id="allowable-values"><ul><li>Split <img src="../../../../../html/images/iconInfo.png" alt="Use a delimiter character or RegEx to split the results into attributes" title="Use a delimiter character or RegEx to split the results into attributes"></img></li><li>RegEx <img src="../../../../../html/images/iconInfo.png" alt="Use a regular expression to split the results into attributes " title="Use a regular expression to split the results into attributes "></img></li><li>None <img src="../../../../../html/images/iconInfo.png" alt="Do not split results" title="Do not split results"></img></li></ul></td><td id="description">The method used to slice the results into attribute groups</td></tr><tr><td id="name">Parser RegEx</td><td id="default-value"></td><td id=
"allowable-values"></td><td id="description">Choice between a splitter and regex matcher used to parse the results of the query into attribute groups.
+NOTE: This is a multiline regular expression, therefore, the DFM should decide how to handle trailing new line characters.</td></tr><tr><td id="name">Key lookup group (multiline / batch)</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">When performing a batched lookup, the following RegEx numbered capture group or Column number will be used to match the whois server response with the lookup field</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>not found</td><td>Where to route flow files if data enrichment query rendered no results</td></tr><tr><td>found</td><td>Where to route flow files after successfully enriching attributes with data</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>enrich.dns.record*.group*</td><td>The captured fields of the Whois query response
for each of the records received</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.9.0/org.apache.nifi.processors.evtx.ParseEvtx/additionalDetails.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.9.0/org.apache.nifi.processors.evtx.ParseEvtx/additionalDetails.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.9.0/org.apache.nifi.processors.evtx.ParseEvtx/additionalDetails.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.9.0/org.apache.nifi.processors.evtx.ParseEvtx/additionalDetails.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,91 @@
+<!DOCTYPE html>
+<html lang="en">
+ <!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+ <head>
+ <meta charset="utf-8" />
+ <title>ParseEvtx</title>
+
+ <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" />
+ </head>
+
+ <body>
+ <!-- Processor Documentation ================================================== -->
+ <h2>Description:</h2>
+ <p>This processor is used to parse Windows event logs in the binary evtx format. The input flow files' content should be evtx files. The processor has 4 outputs:
+ <ul>
+ <li>The original unmodified FlowFile</li>
+ <li>The XML resulting from parsing at the configured granularity</li>
+ <li>Failed parsing with partial output</li>
+ <li>Malformed chunk in binary form</li>
+ </ul>
+ </p>
+ <h2>Output XML Example:</h2>
+ <p>
+ <pre>
+<?xml version="1.0"?>
+<Events>
+ <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
+ <System>
+ <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" Ev
+entSourceName="Service Control Manager"/>
+ <EventID Qualifiers="16384">7036</EventID>
+ <Version>0</Version>
+ <Level>4</Level>
+ <Task>0</Task>
+ <Opcode>0</Opcode>
+ <Keywords>0x8080000000000000</Keywords>
+ <TimeCreated SystemTime="2016-01-08 16:49:47.518"/>
+ <EventRecordID>780</EventRecordID>
+ <Correlation ActivityID="" RelatedActivityID=""/>
+ <Execution ProcessID="480" ThreadID="596"/>
+ <Channel>System</Channel>
+ <Computer>win7-pro-vm</Computer>
+ <Security UserID=""/>
+ </System>
+ <EventData>
+ <Data Name="param1">Workstation</Data>
+ <Data Name="param2">running</Data>
+ <Binary>TABhAG4AbQBhAG4AVwBvAHIAawBzAHQAYQB0AGkAbwBuAC8ANAAAAA==</Binary>
+ </EventData>
+ </Event>
+ <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
+ <System>
+ <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager"/>
+ <EventID Qualifiers="16384">7036</EventID>
+ <Version>0</Version>
+ <Level>4</Level>
+ <Task>0</Task>
+ <Opcode>0</Opcode>
+ <Keywords>0x8080000000000000</Keywords>
+ <TimeCreated SystemTime="2016-01-08 16:49:47.535"/>
+ <EventRecordID>781</EventRecordID>
+ <Correlation ActivityID="" RelatedActivityID=""/>
+ <Execution ProcessID="480" ThreadID="576"/>
+ <Channel>System</Channel>
+ <Computer>win7-pro-vm</Computer>
+ <Security UserID=""/>
+ </System>
+ <EventData>
+ <Data Name="param1">Cryptographic Services</Data>
+ <Data Name="param2">running</Data>
+ <Binary>QwByAHkAcAB0AFMAdgBjAC8ANAAAAA==</Binary>
+ </EventData>
+ </Event>
+</Events>
+ </pre>
+ </p>
+</body>
+</html>
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.9.0/org.apache.nifi.processors.evtx.ParseEvtx/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.9.0/org.apache.nifi.processors.evtx.ParseEvtx/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.9.0/org.apache.nifi.processors.evtx.ParseEvtx/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.9.0/org.apache.nifi.processors.evtx.ParseEvtx/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ParseEvtx</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ParseEvtx</h1><h2>Description: </h2><p>Parses the contents of a Windows Event Log file (evtx) and writes the resulting XML to the FlowFile</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>logs, windows, event, evtx, message, file</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td i
d="name"><strong>Granularity</strong></td><td id="default-value">Chunk</td><td id="allowable-values"><ul><li>Record</li><li>Chunk</li><li>File</li></ul></td><td id="description">Output flow file for each Record, Chunk, or File encountered in the event log</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>Any FlowFile that was successfully converted from evtx to XML</td></tr><tr><td>failure</td><td>Any FlowFile that encountered an exception during conversion will be transferred to this relationship with as much parsing as possible done</td></tr><tr><td>original</td><td>The unmodified input FlowFile will be transferred to this relationship</td></tr><tr><td>bad chunk</td><td>Any bad chunks of records will be transferred to this relationship in their original binary form</td></tr></table><h3>Reads Attributes: </h3><table id="reads-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename</td>
<td>The filename of the evtx file</td></tr></table><h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename</td><td>The output filename</td></tr><tr><td>mime.type</td><td>The output filetype (application/xml for success and failure relationships, original value for bad chunk and original relationships)</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/additionalDetails.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/additionalDetails.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/additionalDetails.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/additionalDetails.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,155 @@
+<!DOCTYPE html>
+<html lang="en">
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<head>
+ <meta charset="utf-8" />
+ <title>ExecuteFlumeSink</title>
+ <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" />
+</head>
+
+<body>
+<h2>Data Model</h2>
+<p>
+ This processor executes an Apache Flume sink. FlowFiles are wrapped in Flume's
+ Event interface. The content of the FlowFile becomes the body of the Event and
+ the attributes of the FlowFile become Event headers. The following special
+ headers are also set:
+</p>
+<table id="headers">
+ <tr>
+ <th>Flume Event Header</th>
+ <th>FlowFile Attribute</th>
+ </tr>
+ <tr>
+ <td>nifi.entry.date</td>
+ <td>FlowFile#getEntryDate()</td>
+ </tr>
+ <tr>
+ <td>nifi.id</td>
+ <td>FlowFile#getId()</td>
+ </tr>
+ <tr>
+ <td>nifi.last.queue.date</td>
+ <td>FlowFile#getLastQueueDate()</td>
+ </tr>
+ <tr>
+ <td>nifi.lineage.start.date</td>
+ <td>FlowFile#getLineageStartDate()</td>
+ </tr>
+ <tr>
+ <td>nifi.size</td>
+ <td>FlowFile#getSize()</td>
+ </tr>
+</table>
+<h2>Warning</h2>
+<p>
+ In NiFi, the contents of a FlowFile are accessed via a stream, but in Flume it is
+ stored in a byte array. This means the full content will be loaded into memory when
+ a FlowFile is processed by the ExecuteFlumeSink processor. You should consider the
+ typical size of the FlowFiles you'll process and the batch size, if any, your sink
+ is configured with when setting NiFi's heap size.
+</p>
+<h2>Configuration Details</h2>
+<p>
+ This processor is designed to execute arbitrary Flume sinks. Most of the details
+ of configuring the sink is deferred to Flume's built-in configuration system.
+ For details on the available settings for each sink type, refer to the Flume
+ <a href="http://flume.apache.org/FlumeUserGuide.html#flume-sinks">User Guide</a>.
+ Configuring the Flume sink is a four step process:
+</p>
+<ol>
+ <li>Set the Sink Type property to a valid Flume sink type.</li>
+ <li>
+ Set the Agent Name property to the name of the agent in your
+ Flume configuration. This is the prefix of the properties in the Flume
+ configuration file. Example: <code>tier1</code>
+ </li>
+ <li>
+ Set the Sink Name property to the name of the sink in your Flume
+ configuration. If Agent Name is <code>tier1</code>, then the Sink Name
+ is the value of the <code>tier1.sinks</code> property. Example: <code>sink-1</code>
+ </li>
+ <li>
+ Copy and paste the configuration for the sink from your Flume configuration
+ file into the Flume Configuration property. Assuming you're using
+ the same Agent Name and Sink Name as in the examples above, this will be all
+ of the properties that start with <code>tier1.sinks.sink-1</code>.
+ Do not copy the <code>tier1.sinks.sink-1.type</code> or
+ <code>tier1.sinks.sink-1.channel</code> properties.
+ </li>
+</ol>
+<h2>Usage Example</h2>
+<p>
+ Assuming you had the following existing Flume configuration file:
+</p>
+ <pre>
+a1.sources = r1
+a1.sinks = k1
+a1.channels = c1
+
+a1.sources.r1.type = netcat
+a1.sources.r1.bind = localhost
+a1.sources.r1.port = 44444
+
+a1.sinks.k1.type = hdfs
+a1.sinks.k1.channel = c1
+a1.sinks.k1.hdfs.path = /flume/events/%y-%m-%d/%H%M/%S
+a1.sinks.k1.hdfs.filePrefix = events-
+a1.sinks.k1.hdfs.round = true
+a1.sinks.k1.hdfs.roundValue = 10
+a1.sinks.k1.hdfs.roundUnit = minute
+
+a1.channels.c1.type = memory
+a1.channels.c1.capacity = 1000
+a1.channels.c1.transactionCapacity = 100
+
+a1.sources.r1.channels = c1
+a1.sinks.k1.channel = c1</pre>
+<p>
+ Then you'd configure the ExecuteFlumeSink as follows:
+</p>
+<table id="example">
+ <tr>
+ <th>Property</th>
+ <th>Value</th>
+ </tr>
+ <tr>
+ <td>Sink Type</td>
+ <td>hdfs</td>
+ </tr>
+ <tr>
+ <td>Agent Name</td>
+ <td>a1</td>
+ </tr>
+ <tr>
+ <td>Sink Name</td>
+ <td>k1</td>
+ </tr>
+ <tr>
+ <td>Flume Configuration</td>
+ <td>
+ <code>
+ a1.sinks.k1.hdfs.path = /flume/events/%y-%m-%d/%H%M/%S<br>
+ a1.sinks.k1.hdfs.filePrefix = events-<br>
+ a1.sinks.k1.hdfs.round = true<br>
+ a1.sinks.k1.hdfs.roundValue = 10<br>
+ a1.sinks.k1.hdfs.roundUnit = minute
+ </code>
+ </td>
+ </tr>
+</table>
+</body>
+</html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ExecuteFlumeSink</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ExecuteFlumeSink</h1><h2>Description: </h2><p>Execute a Flume sink. Each input FlowFile is converted into a Flume Event for processing by the sink.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>flume, hadoop, put, sink</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id
="name"><strong>Sink Type</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The component type name for the sink. For some sinks, this is a short, symbolic name (e.g. hdfs). For others, it's the fully-qualified name of the Sink class. See the Flume User Guide for details.</td></tr><tr><td id="name"><strong>Agent Name</strong></td><td id="default-value">tier1</td><td id="allowable-values"></td><td id="description">The name of the agent used in the Flume sink configuration</td></tr><tr><td id="name"><strong>Sink Name</strong></td><td id="default-value">sink-1</td><td id="allowable-values"></td><td id="description">The name of the sink used in the Flume sink configuration</td></tr><tr><td id="name"><strong>Flume Configuration</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The Flume configuration for the sink copied from the flume.properties file</td></tr></table><h3>Relationships: </h3><table id="r
elationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td></td></tr><tr><td>failure</td><td></td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3><table id="restrictions"><tr><th>Required Permission</th><th>Explanation</th></tr><tr><td>execute code</td><td>Provides operator the ability to execute arbitrary Flume configurations assuming all permissions that NiFi has.</td></tr></table><h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/additionalDetails.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/additionalDetails.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/additionalDetails.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/additionalDetails.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,114 @@
+<!DOCTYPE html>
+<html lang="en">
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<head>
+ <meta charset="utf-8" />
+ <title>ExecuteFlumeSource</title>
+ <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" />
+</head>
+
+<body>
+<h2>Data Model</h2>
+<p>
+ This processor executes an Apache Flume source. Each Flume Event is turned into
+ a FlowFile. The content of the FlowFile is set to the body of the Event and
+ the Event headers become FlowFile attributes.
+</p>
+<h2>Configuration Details</h2>
+<p>
+ This processor is designed to execute arbitrary Flume sources. Most of the details
+ of configuring the source is deferred to Flume's built-in configuration system.
+ For details on the available settings for each source type, refer to the Flume
+ <a href="http://flume.apache.org/FlumeUserGuide.html#flume-sources">User Guide</a>.
+ Configuring the Flume source is a four step process:
+</p>
+<ol>
+ <li>Set the Source Type property to a valid Flume source type.</li>
+ <li>
+ Set the Agent Name property to the name of the agent in your
+ Flume configuration. This is the prefix of the properties in the Flume
+ configuration file. Example: <code>tier1</code>
+ </li>
+ <li>
+ Set the Source Name property to the name of the source in your Flume
+ configuration. If Agent Name is <code>tier1</code>, then the Source Name
+ is the value of the <code>tier1.sources</code> property. Example: <code>src-1</code>
+ </li>
+ <li>
+ Copy and paste the configuration for the source from your Flume configuration
+ file into the Flume Configuration property. Assuming you're using
+ the same Agent Name and Source Name as in the examples above, this will be all
+ of the properties that start with <code>tier1.sources.src-1</code>.
+ Do not copy the <code>tier1.sources.src-1.type</code> or
+ <code>tier1.sources.src-1.channel</code> properties.
+ </li>
+</ol>
+<h2>Usage Example</h2>
+<p>
+ Assuming you had the following existing Flume configuration file:
+</p>
+ <pre>
+a1.sources = r1
+a1.sinks = k1
+a1.channels = c1
+
+a1.sources.r1.type = multiport_syslogtcp
+a1.sources.r1.channels = c1
+a1.sources.r1.host = 0.0.0.0
+a1.sources.r1.ports = 10001 10002 10003
+a1.sources.r1.portHeader = port
+
+a1.sinks.k1.type = logger
+
+a1.channels.c1.type = memory
+a1.channels.c1.capacity = 1000
+a1.channels.c1.transactionCapacity = 100
+
+a1.sources.r1.channels = c1
+a1.sinks.k1.channel = c1</pre>
+<p>
+ Then you'd configure the ExecuteFlumeSource as follows:
+</p>
+<table id="example">
+ <tr>
+ <th>Property</th>
+ <th>Value</th>
+ </tr>
+ <tr>
+ <td>Source Type</td>
+ <td>multiport_syslogtcp</td>
+ </tr>
+ <tr>
+ <td>Agent Name</td>
+ <td>a1</td>
+ </tr>
+ <tr>
+ <td>Source Name</td>
+ <td>r1</td>
+ </tr>
+ <tr>
+ <td>Flume Configuration</td>
+ <td>
+ <code>
+ a1.sources.r1.host = 0.0.0.0<br>
+ a1.sources.r1.ports = 10001 10002 10003<br>
+ a1.sources.r1.portHeader = port
+ </code>
+ </td>
+ </tr>
+</table>
+</body>
+</html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.9.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ExecuteFlumeSource</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ExecuteFlumeSource</h1><h2>Description: </h2><p>Execute a Flume source. Each Flume Event is sent to the success relationship as a FlowFile</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>flume, hadoop, get, source</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="nam
e"><strong>Source Type</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The component type name for the source. For some sources, this is a short, symbolic name (e.g. spooldir). For others, it's the fully-qualified name of the Source class. See the Flume User Guide for details.</td></tr><tr><td id="name"><strong>Agent Name</strong></td><td id="default-value">tier1</td><td id="allowable-values"></td><td id="description">The name of the agent used in the Flume source configuration</td></tr><tr><td id="name"><strong>Source Name</strong></td><td id="default-value">src-1</td><td id="allowable-values"></td><td id="description">The name of the source used in the Flume source configuration</td></tr><tr><td id="name"><strong>Flume Configuration</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The Flume configuration for the source copied from the flume.properties file</td></tr></table><h3>Relationships: <
/h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td></td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3><table id="restrictions"><tr><th>Required Permission</th><th>Explanation</th></tr><tr><td>execute code</td><td>Provides operator the ability to execute arbitrary Flume configurations assuming all permissions that NiFi has.</td></tr></table><h3>Input requirement: </h3>This component does not allow an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,4 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>PutBigQueryBatch</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">PutBigQueryBatch</h1><h2>Description: </h2><p>Batch loads flow files content to a Google BigQuery table.</p><h3>Tags: </h3><p>google, google cloud, bq, bigquery</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before e
ntering a value in a sensitive property, ensure that the <strong>nifi.properties</strong> file has an entry for the property <strong>nifi.sensitive.props.key</strong>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>GCP Credentials Provider Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>GCPCredentialsService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html">GCPCredentialsControllerService</a></td><td id="description">The Controller Service used to obtain Google Cloud Platform credentials.</td></tr><tr><td id="name">Project ID</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Google Cloud Project ID<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)<
/strong></td></tr><tr><td id="name"><strong>Number of retries</strong></td><td id="default-value">6</td><td id="allowable-values"></td><td id="description">How many retry attempts should be made before routing to the failure relationship.</td></tr><tr><td id="name">Proxy host</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">IP or hostname of the proxy to be used.
+ You might need to set the following properties in bootstrap for https proxy usage:
+-Djdk.http.auth.tunneling.disabledSchemes=
+-Djdk.http.auth.proxying.disabledSchemes=<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy port</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Proxy port number<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">HTTP Proxy Username</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">HTTP Proxy Username<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">HTTP Proxy Password</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">HTTP Proxy Password<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy Configuration
Service</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>ProxyConfigurationService<br/><strong>Implementation: </strong><a href="../../../nifi-proxy-configuration-nar/1.9.0/org.apache.nifi.proxy.StandardProxyConfigurationService/index.html">StandardProxyConfigurationService</a></td><td id="description">Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN</td></tr><tr><td id="name"><strong>Dataset</strong></td><td id="default-value">${bq.dataset}</td><td id="allowable-values"></td><td id="description">BigQuery dataset name (Note - The dataset must exist in GCP)<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Table Name</strong></td><td id="default-value">${bq.table.name}</td><td id="allowable-values"
></td><td id="description">BigQuery table name<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Table Schema</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">BigQuery schema in JSON format<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Read Timeout</strong></td><td id="default-value">5 minutes</td><td id="allowable-values"></td><td id="description">Load Job Time Out<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Load file type</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Data type of the file to be loaded. Possible values: AVRO, NEWLINE_DELIMITED_JSON, CSV.<br/><strong>Supports
Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Create Disposition</strong></td><td id="default-value">CREATE_IF_NEEDED</td><td id="allowable-values"><ul><li>CREATE_IF_NEEDED <img src="../../../../../html/images/iconInfo.png" alt="Configures the job to create the table if it does not exist." title="Configures the job to create the table if it does not exist."></img></li><li>CREATE_NEVER <img src="../../../../../html/images/iconInfo.png" alt="Configures the job to fail with a not-found error if the table does not exist." title="Configures the job to fail with a not-found error if the table does not exist."></img></li></ul></td><td id="description">Sets whether the job is allowed to create new tables</td></tr><tr><td id="name"><strong>Write Disposition</strong></td><td id="default-value">WRITE_EMPTY</td><td id="allowable-values"><ul><li>WRITE_EMPTY <img src="../../../../../html/images/iconInfo.p
ng" alt="Configures the job to fail with a duplicate error if the table already exists." title="Configures the job to fail with a duplicate error if the table already exists."></img></li><li>WRITE_APPEND <img src="../../../../../html/images/iconInfo.png" alt="Configures the job to append data to the table if it already exists." title="Configures the job to append data to the table if it already exists."></img></li><li>WRITE_TRUNCATE <img src="../../../../../html/images/iconInfo.png" alt="Configures the job to overwrite the table data if table already exists." title="Configures the job to overwrite the table data if table already exists."></img></li></ul></td><td id="description">Sets the action that should occur if the destination table already exists.</td></tr><tr><td id="name"><strong>Max Bad Records</strong></td><td id="default-value">0</td><td id="allowable-values"></td><td id="description">Sets the maximum number of bad records that BigQuery can ignore when running the job. If
the number of bad records exceeds this value, an invalid error is returned in the job result. By default no bad record is ignored.</td></tr><tr><td id="name"><strong>Ignore Unknown Values</strong></td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Sets whether BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. By default unknown values are not allowed.</td></tr><tr><td id="name"><strong>CSV Input - Allow Jagged Rows</strong></td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Set whether BigQuery should accept rows that are missing trailing optional columns. If true, BigQuery treats missing trailing columns as null values
. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. By default, rows with missing trailing columns are considered bad records.</td></tr><tr><td id="name"><strong>CSV Input - Allow Quoted New Lines</strong></td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Sets whether BigQuery should allow quoted data sections that contain newline characters in a CSV file. By default quoted newline are not allowed.</td></tr><tr><td id="name"><strong>CSV Input - Character Set</strong></td><td id="default-value">UTF-8</td><td id="allowable-values"><ul><li>UTF-8</li><li>ISO-8859-1</li></ul></td><td id="description">Sets the character encoding of the data.</td></tr><tr><td id="name"><strong>CSV Input - Field Delimiter</strong></td><td id="default-value">,</td><td id="allowable-values"></td><td id="description">Sets the
separator for fields in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. BigQuery also supports the escape sequence " " to specify a tab separator. The default value is a comma (',').<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>CSV Input - Quote</strong></td><td id="default-value">"</td><td id="allowable-values"></td><td id="description">Sets the value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote ('"'). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the Allow Quote
d New Lines property to true.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>CSV Input - Skip Leading Rows</strong></td><td id="default-value">0</td><td id="allowable-values"></td><td id="description">Sets the number of rows at the top of a CSV file that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles are routed to this relationship after a successful Google BigQuery operation.</td></tr><tr><td>failure</td><td>FlowFiles are routed to this relationship if the Google BigQuery operation fails.</td></tr></table><
h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>bq.dataset</td><td>BigQuery dataset name (Note - The dataset must exist in GCP)</td></tr><tr><td>bq.table.name</td><td>BigQuery table name</td></tr><tr><td>bq.table.schema</td><td>BigQuery schema in JSON format</td></tr><tr><td>bq.load.type</td><td>Data type of the file to be loaded. Possible values: AVRO, NEWLINE_DELIMITED_JSON, CSV.</td></tr><tr><td>bq.load.ignore_unknown</td><td>Sets whether BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. By default unknown values are not allowed.</td></tr><tr><td>bq.load.create_disposition</td><td>Sets whether the job is allowed to create new tables</td></tr><tr><td>bq.load.write
_disposition</td><td>Sets the action that should occur if the destination table already exists.</td></tr><tr><td>bq.load.max_badrecords</td><td>Sets the maximum number of bad records that BigQuery can ignore when running the job. If the number of bad records exceeds this value, an invalid error is returned in the job result. By default no bad record is ignored.</td></tr><tr><td>bq.job.stat.creation_time</td><td>Time load job creation</td></tr><tr><td>bq.job.stat.end_time</td><td>Time load job ended</td></tr><tr><td>bq.job.stat.start_time</td><td>Time load job started</td></tr><tr><td>bq.job.link</td><td>API Link to load job</td></tr><tr><td>bq.error.message</td><td>Load job error message</td></tr><tr><td>bq.error.reason</td><td>Load job error reason</td></tr><tr><td>bq.error.location</td><td>Load job error location</td></tr><tr><td>bq.records.count</td><td>Number of records successfully inserted</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Rest
ricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.gcp.storage.PutGCSObject/index.html">PutGCSObject</a>, <a href="../org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html">DeleteGCSObject</a></p></body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>GCPCredentialsControllerService</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">GCPCredentialsControllerService</h1><h2>Description: </h2><p>Defines credentials for Google Cloud Platform processors. Uses Application Default credentials without configuration. Application Default credentials support environmental variable (GOOGLE_APPLICATION_CREDENTIALS) pointing to a credential file, the config generated by `gcloud auth application-default login`, AppEngine/Compute Engine service accounts, etc.</p><h3>Tags: </h3><p>gcp, credentials, provider</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any oth
er properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the <strong>nifi.properties</strong> file has an entry for the property <strong>nifi.sensitive.props.key</strong>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name">Use Application Default Credentials</td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If true, uses Google Application Default Credentials, which checks the GOOGLE_APPLICATION_CREDENTIALS environment variable for a filepath to a service account JSON key, the config generated by the gcloud sdk, the A
pp Engine service account, and the Compute Engine service account.</td></tr><tr><td id="name">Use Compute Engine Credentials</td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If true, uses Google Compute Engine Credentials of the Compute Engine VM Instance which NiFi is running on.</td></tr><tr><td id="name">Service Account JSON File</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Path to a file containing a Service Account key file in JSON format.</td></tr><tr><td id="name">Service Account JSON</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The raw JSON containing a Service Account keyfile.<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy Configuration Service</td><td id="default-value"></td><td id="allo
wable-values"><strong>Controller Service API: </strong><br/>ProxyConfigurationService<br/><strong>Implementation: </strong><a href="../../../nifi-proxy-configuration-nar/1.9.0/org.apache.nifi.proxy.StandardProxyConfigurationService/index.html">StandardProxyConfigurationService</a></td><td id="description">Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>System Resource Considerations:</h3>None specified.</body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ConsumeGCPubSub</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ConsumeGCPubSub</h1><h2>Description: </h2><p>Consumes message from the configured Google Cloud PubSub subscription. If the 'Batch Size' is set, the configured number of messages will be pulled in a single request, else only one message will be pulled.</p><h3>Tags: </h3><p>google, google-cloud, gcp, message, pubsub, consume</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../
html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name">Project ID</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Google Cloud Project ID<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>GCP Credentials Provider Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>GCPCredentialsService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html">GCPCredentialsControllerService</a></td><td id="description">The Controller Service used to obtain Google Cloud Platform credentials.</td></tr><tr><td id="name"><strong>Subscription</strong></td><td id="default-value"></td><td
id="allowable-values"></td><td id="description">Name of the Google Cloud Pub/Sub Subscription<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Batch Size</strong></td><td id="default-value">15</td><td id="allowable-values"></td><td id="description">Indicates the number of messages the cloud service should bundle together in a batch. If not set and left empty, only one message will be used in a batch</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles are routed to this relationship after a successful Google Cloud Pub/Sub operation.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>gcp.pubsub.ackId</td><td>Acknowledgement Id of the consumed Google Cloud PubSub message</td></tr><tr><td>gcp.
pubsub.messageSize</td><td>Serialized size of the consumed Google Cloud PubSub message</td></tr><tr><td>gcp.pubsub.attributesCount</td><td>Number of attributes the consumed PubSub message has, if any</td></tr><tr><td>gcp.pubsub.publishTime</td><td>Timestamp value when the message was published</td></tr><tr><td>Dynamic Attributes</td><td>Other than the listed attributes, this processor may write zero or more attributes, if the original Google Cloud Publisher client added any attributes to the message while sending</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component does not allow an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.gcp.pubsub.PublishGCPubSub/index.html">PublishGCPubSub</a></p></body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.pubsub.PublishGCPubSub/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.pubsub.PublishGCPubSub/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.pubsub.PublishGCPubSub/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.pubsub.PublishGCPubSub/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>PublishGCPubSub</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">PublishGCPubSub</h1><h2>Description: </h2><p>Publishes the content of the incoming flowfile to the configured Google Cloud PubSub topic. The processor supports dynamic properties. If any dynamic properties are present, they will be sent along with the message in the form of 'attributes'.</p><h3>Tags: </h3><p>google, google-cloud, gcp, message, pubsub, publish</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property
supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name">Project ID</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Google Cloud Project ID<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>GCP Credentials Provider Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>GCPCredentialsService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html">GCPCredentialsControllerService</a></td><td id="description">The Controller Service used to obtain Google Cloud Platform credentials.</td></tr><tr><td id="name"><strong>Topic Name</strong><
/td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Name of the Google Cloud PubSub Topic<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Batch Size</strong></td><td id="default-value">15</td><td id="allowable-values"></td><td id="description">Indicates the number of messages the cloud service should bundle together in a batch. If not set and left empty, only one message will be used in a batch</td></tr></table><h3>Dynamic Properties: </h3><p>Dynamic Properties allow the user to specify both the name and value of a property.<table id="dynamic-properties"><tr><th>Name</th><th>Value</th><th>Description</th></tr><tr><td id="name">Attribute name</td><td id="value">Value to be set to the attribute</td><td>Attributes to be set for the outgoing Google Cloud PubSub message<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and
variable registry)</strong></td></tr></table></p><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>retry</td><td>FlowFiles are routed to this relationship if the Google Cloud Pub/Sub operation fails but attempting the operation again may succeed.</td></tr><tr><td>success</td><td>FlowFiles are routed to this relationship after a successful Google Cloud Pub/Sub operation.</td></tr><tr><td>failure</td><td>FlowFiles are routed to this relationship if the Google Cloud Pub/Sub operation fails.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>gcp.pubsub.messageId</td><td>ID of the pubsub message published to the configured Google Cloud PubSub topic</td></tr><tr><td>gcp.pubsub.topic</td><td>Name of the Google Cloud PubSub topic the message was published to</td></tr></table><h3>State management: </h3>This component does not st
ore state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub/index.html">ConsumeGCPubSub</a></p></body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,4 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>DeleteGCSObject</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">DeleteGCSObject</h1><h2>Description: </h2><p>Deletes objects from a Google Cloud Bucket. If attempting to delete a file that does not exist, FlowFile is routed to success.</p><h3>Tags: </h3><p>google cloud, gcs, google, storage, delete</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>, and whether a property is
considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the <strong>nifi.properties</strong> file has an entry for the property <strong>nifi.sensitive.props.key</strong>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>GCP Credentials Provider Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>GCPCredentialsService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html">GCPCredentialsControllerService</a></td><td id="description">The Controller Service used to obtain Google Cloud Platform credentials.</td></tr><tr><td id="name">Project ID</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Google Cloud Project ID<br/><strong>Supports E
xpression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Number of retries</strong></td><td id="default-value">6</td><td id="allowable-values"></td><td id="description">How many retry attempts should be made before routing to the failure relationship.</td></tr><tr><td id="name">Proxy host</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">IP or hostname of the proxy to be used.
+ You might need to set the following properties in bootstrap for https proxy usage:
+-Djdk.http.auth.tunneling.disabledSchemes=
+-Djdk.http.auth.proxying.disabledSchemes=<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy port</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Proxy port number<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">HTTP Proxy Username</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">HTTP Proxy Username<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">HTTP Proxy Password</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">HTTP Proxy Password<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy Configuration
Service</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>ProxyConfigurationService<br/><strong>Implementation: </strong><a href="../../../nifi-proxy-configuration-nar/1.9.0/org.apache.nifi.proxy.StandardProxyConfigurationService/index.html">StandardProxyConfigurationService</a></td><td id="description">Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN</td></tr><tr><td id="name"><strong>Bucket</strong></td><td id="default-value">${gcs.bucket}</td><td id="allowable-values"></td><td id="description">Bucket of the object.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Key</strong></td><td id="default-value">${filename}</td><td id="allowable-values"></td><td id="description">Name of the object.<br/><
strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Generation</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The generation of the object to be deleted. If null, will use latest version of the object.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles are routed to this relationship after a successful Google Cloud Storage operation.</td></tr><tr><td>failure</td><td>FlowFiles are routed to this relationship if the Google Cloud Storage operation fails.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This c
omponent is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.gcp.storage.PutGCSObject/index.html">PutGCSObject</a>, <a href="../org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html">FetchGCSObject</a>, <a href="../org.apache.nifi.processors.gcp.storage.ListGCSBucket/index.html">ListGCSBucket</a></p></body></html>
\ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html
URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html?rev=1854109&view=auto
==============================================================================
--- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html (added)
+++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.9.0/org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html Fri Feb 22 01:03:44 2019
@@ -0,0 +1,4 @@
+<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>FetchGCSObject</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">FetchGCSObject</h1><h2>Description: </h2><p>Fetches a file from a Google Cloud Bucket. Designed to be used in tandem with ListGCSBucket.</p><h3>Tags: </h3><p>google cloud, google, storage, gcs, fetch</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>, and whether a property is considered "sensitive", meaning that
its value will be encrypted. Before entering a value in a sensitive property, ensure that the <strong>nifi.properties</strong> file has an entry for the property <strong>nifi.sensitive.props.key</strong>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>GCP Credentials Provider Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>GCPCredentialsService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html">GCPCredentialsControllerService</a></td><td id="description">The Controller Service used to obtain Google Cloud Platform credentials.</td></tr><tr><td id="name">Project ID</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Google Cloud Project ID<br/><strong>Supports Expression Language: true (will be eva
luated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Number of retries</strong></td><td id="default-value">6</td><td id="allowable-values"></td><td id="description">How many retry attempts should be made before routing to the failure relationship.</td></tr><tr><td id="name">Proxy host</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">IP or hostname of the proxy to be used.
+ You might need to set the following properties in bootstrap for https proxy usage:
+-Djdk.http.auth.tunneling.disabledSchemes=
+-Djdk.http.auth.proxying.disabledSchemes=<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy port</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Proxy port number<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">HTTP Proxy Username</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">HTTP Proxy Username<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">HTTP Proxy Password</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">HTTP Proxy Password<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Proxy Configuration
Service</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>ProxyConfigurationService<br/><strong>Implementation: </strong><a href="../../../nifi-proxy-configuration-nar/1.9.0/org.apache.nifi.proxy.StandardProxyConfigurationService/index.html">StandardProxyConfigurationService</a></td><td id="description">Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN</td></tr><tr><td id="name"><strong>Bucket</strong></td><td id="default-value">${gcs.bucket}</td><td id="allowable-values"></td><td id="description">Bucket of the object.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Key</strong></td><td id="default-value">${filename}</td><td id="allowable-values"></td><td id="description">Name of the object.<br/><
strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Object Generation</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The generation of the Object to download. If null, will download latest generation.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Server Side Encryption Key</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">An AES256 Key (encoded in base64) which the object has been encrypted in.<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles are routed to
this relationship after a successful Google Cloud Storage operation.</td></tr><tr><td>failure</td><td>FlowFiles are routed to this relationship if the Google Cloud Storage operation fails.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename</td><td>The name of the file, parsed if possible from the Content-Disposition response header</td></tr><tr><td>gcs.bucket</td><td>Bucket of the object.</td></tr><tr><td>gcs.key</td><td>Name of the object.</td></tr><tr><td>gcs.size</td><td>Size of the object.</td></tr><tr><td>gcs.cache.control</td><td>Data cache control of the object.</td></tr><tr><td>gcs.component.count</td><td>The number of components which make up the object.</td></tr><tr><td>gcs.content.disposition</td><td>The data content disposition of the object.</td></tr><tr><td>gcs.content.encoding</td><td>The content encoding of the object.</td></tr><tr><td>gcs.con
tent.language</td><td>The content language of the object.</td></tr><tr><td>mime.type</td><td>The MIME/Content-Type of the object</td></tr><tr><td>gcs.crc32c</td><td>The CRC32C checksum of object's data, encoded in base64 in big-endian order.</td></tr><tr><td>gcs.create.time</td><td>The creation time of the object (milliseconds)</td></tr><tr><td>gcs.update.time</td><td>The last modification time of the object (milliseconds)</td></tr><tr><td>gcs.encryption.algorithm</td><td>The algorithm used to encrypt the object.</td></tr><tr><td>gcs.encryption.sha256</td><td>The SHA256 hash of the key used to encrypt the object</td></tr><tr><td>gcs.etag</td><td>The HTTP 1.1 Entity tag for the object.</td></tr><tr><td>gcs.generated.id</td><td>The service-generated for the object</td></tr><tr><td>gcs.generation</td><td>The data generation of the object.</td></tr><tr><td>gcs.md5</td><td>The MD5 hash of the object's data encoded in base64.</td></tr><tr><td>gcs.media.link</td><td>The media download link
to the object.</td></tr><tr><td>gcs.metageneration</td><td>The metageneration of the object.</td></tr><tr><td>gcs.owner</td><td>The owner (uploader) of the object.</td></tr><tr><td>gcs.owner.type</td><td>The ACL entity type of the uploader of the object.</td></tr><tr><td>gcs.uri</td><td>The URI of the object as a string.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.gcp.storage.ListGCSBucket/index.html">ListGCSBucket</a>, <a href="../org.apache.nifi.processors.gcp.storage.PutGCSObject/index.html">PutGCSObject</a>, <a href="../org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html">DeleteGCSObject</a></p></body></html>
\ No newline at end of file