You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by sz...@apache.org on 2022/05/25 11:37:25 UTC
[nifi-minifi-cpp] 02/02: MINIFICPP-1837 added Windows only processors to PROCESSORS.md
This is an automated email from the ASF dual-hosted git repository.
szaszm pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 71ce94eb378a13c9a0567104bb2624a4d06b8b07
Author: Martin Zink <ma...@apache.org>
AuthorDate: Wed May 25 13:12:39 2022 +0200
MINIFICPP-1837 added Windows only processors to PROCESSORS.md
Closes #1338
Signed-off-by: Marton Szasz <sz...@apache.org>
---
PROCESSORS.md | 2003 ++++++++++++++++++++++++++++++---------------------------
README.md | 70 +-
2 files changed, 1080 insertions(+), 993 deletions(-)
diff --git a/PROCESSORS.md b/PROCESSORS.md
index cbd1ac266..f1b4bd7ba 100644
--- a/PROCESSORS.md
+++ b/PROCESSORS.md
@@ -9,10 +9,12 @@
- [BinFiles](#binfiles)
- [CapturePacket](#capturepacket)
- [CaptureRTSPFrame](#capturertspframe)
+- [CollectorInitiatedSubscription](#collectorinitiatedsubscription)
- [CompressContent](#compresscontent)
- [ConsumeJournald](#consumejournald)
- [ConsumeKafka](#consumekafka)
- [ConsumeMQTT](#consumemqtt)
+- [ConsumeWindowsEventLog](#consumewindowseventlog)
- [DefragmentText](#defragmenttext)
- [DeleteAzureBlobStorage](#deleteazureblobstorage)
- [DeleteAzureDataLakeStorage](#deleteazuredatalakestorage)
@@ -70,6 +72,7 @@
- [RetryFlowFile](#retryflowfile)
- [RouteOnAttribute](#routeonattribute)
- [RouteText](#routetext)
+- [TailEventLog](#taileventlog)
- [TailFile](#tailfile)
- [UnfocusArchiveEntry](#unfocusarchiveentry)
- [UpdateAttribute](#updateattribute)
@@ -82,17 +85,17 @@ Appends host information such as IP address and hostname as an attribute to inco
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Hostname Attribute|source.hostname||Flowfile attribute used to record the agent's hostname|
-|IP Attribute|source.ipv4||Flowfile attribute used to record the agent's IP address|
-|Network Interface Filter|||A regular expression to filter ip addresses based on the name of the network interface|
-|Refresh Policy|On schedule|On schedule<br>On every trigger|When to recalculate the host info|
+| Name | Default Value | Allowable Values | Description |
+|--------------------------|-----------------|---------------------------------|----------------------------------------------------------------------------------------|
+| Hostname Attribute | source.hostname | | Flowfile attribute used to record the agent's hostname |
+| IP Attribute | source.ipv4 | | Flowfile attribute used to record the agent's IP address |
+| Network Interface Filter | | | A regular expression to filter ip addresses based on the name of the network interface |
+| Refresh Policy | On schedule | On schedule<br>On every trigger | When to recalculate the host info |
### Relationships
-| Name | Description |
-| - | - |
-|success|success operational on the flow record|
+| Name | Description |
+|---------|----------------------------------------|
+| success | success operational on the flow record |
## ApplyTemplate
@@ -104,14 +107,14 @@ Applies the mustache template specified by the "Template" property and writes th
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Template|||Path to the input mustache template file|
+| Name | Default Value | Allowable Values | Description |
+|----------|---------------|------------------|------------------------------------------|
+| Template | | | Path to the input mustache template file |
### Relationships
-| Name | Description |
-| - | - |
-|success|success operational on the flow record|
+| Name | Description |
+|---------|----------------------------------------|
+| success | success operational on the flow record |
## AttributesToJSON
@@ -122,18 +125,18 @@ Generates a JSON representation of the input FlowFile Attributes. The resulting
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Attributes List|||Comma separated list of attributes to be included in the resulting JSON. If this value is left empty then all existing Attributes will be included. This list of attributes is case sensitive. If an attribute specified in the list is not found it will be be emitted to the resulting JSON with an empty string or NULL value.|
-|Attributes Regular Expression|||Regular expression that will be evaluated against the flow file attributes to select the matching attributes. Both the matching attributes and the selected attributes from the Attributes List property will be written in the resulting JSON.|
-|**Destination**|flowfile-attribute|flowfile-attribute<br>flowfile-content<br>|Control if JSON value is written as a new flowfile attribute 'JSONAttributes' or written in the flowfile content. Writing to flowfile content will overwrite any existing flowfile content.|
-|**Include Core Attributes**|true||Determines if the FlowFile core attributes which are contained in every FlowFile should be included in the final JSON value generated.|
-|**Null Value**|false||If true a non existing selected attribute will be NULL in the resulting JSON. If false an empty string will be placed in the JSON.|
+| Name | Default Value | Allowable Values | Description |
+|-------------------------------|--------------------|--------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Attributes List | | | Comma separated list of attributes to be included in the resulting JSON. If this value is left empty then all existing Attributes will be included. This list of attributes is case sensitive. If an attribute specified in the list is not found it will be be emitted to the resulting JSON with an empty string or NULL value. |
+| Attributes Regular Expression | | | Regular expression that will be evaluated against the flow file attributes to select the matching attributes. Both the matching attributes and the selected attributes from the Attributes List property will be written in the resulting JSON. |
+| **Destination** | flowfile-attribute | flowfile-attribute<br>flowfile-content<br> | Control if JSON value is written as a new flowfile attribute 'JSONAttributes' or written in the flowfile content. Writing to flowfile content will overwrite any existing flowfile content. |
+| **Include Core Attributes** | true | | Determines if the FlowFile core attributes which are contained in every FlowFile should be included in the final JSON value generated. |
+| **Null Value** | false | | If true a non existing selected attribute will be NULL in the resulting JSON. If false an empty string will be placed in the JSON. |
### Relationships
-| Name | Description |
-| - | - |
-|success|All FlowFiles received are routed to success|
+| Name | Description |
+|---------|----------------------------------------------|
+| success | All FlowFiles received are routed to success |
## BinFiles
@@ -144,20 +147,20 @@ Bins flow files into buckets based on the number of entries or size of entries
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Max Bin Age|||The maximum age of a Bin that will trigger a Bin to be complete. Expected format is <duration> <time unit>|
-|Maximum Group Size|||The maximum size for the bundle. If not specified, there is no maximum.|
-|Maximum Number of Entries|||The maximum number of files to include in a bundle. If not specified, there is no maximum.|
-|Maximum number of Bins|100||Specifies the maximum number of bins that can be held in memory at any one time|
-|Minimum Group Size|0||The minimum size of for the bundle|
-|Minimum Number of Entries|1||The minimum number of files to include in a bundle|
+| Name | Default Value | Allowable Values | Description |
+|---------------------------|---------------|------------------|------------------------------------------------------------------------------------------------------------|
+| Max Bin Age | | | The maximum age of a Bin that will trigger a Bin to be complete. Expected format is <duration> <time unit> |
+| Maximum Group Size | | | The maximum size for the bundle. If not specified, there is no maximum. |
+| Maximum Number of Entries | | | The maximum number of files to include in a bundle. If not specified, there is no maximum. |
+| Maximum number of Bins | 100 | | Specifies the maximum number of bins that can be held in memory at any one time |
+| Minimum Group Size | 0 | | The minimum size of for the bundle |
+| Minimum Number of Entries | 1 | | The minimum number of files to include in a bundle |
### Relationships
-| Name | Description |
-| - | - |
-|failure|If the bundle cannot be created, all FlowFiles that would have been used to create the bundle will be transferred to failure|
-|original|The FlowFiles that were used to create the bundle|
+| Name | Description |
+|----------|------------------------------------------------------------------------------------------------------------------------------|
+| failure | If the bundle cannot be created, all FlowFiles that would have been used to create the bundle will be transferred to failure |
+| original | The FlowFiles that were used to create the bundle |
## CapturePacket
@@ -169,17 +172,17 @@ CapturePacket captures and writes one or more packets into a PCAP file that will
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Base Directory|/tmp/||Scratch directory for PCAP files|
-|Batch Size|50||The number of packets to combine within a given PCAP|
-|Capture Bluetooth|false||True indicates that we support bluetooth interfaces|
-|Network Controllers|.*||Regular expression of the network controller(s) to which we will attach|
+| Name | Default Value | Allowable Values | Description |
+|---------------------|---------------|------------------|-------------------------------------------------------------------------|
+| Base Directory | /tmp/ | | Scratch directory for PCAP files |
+| Batch Size | 50 | | The number of packets to combine within a given PCAP |
+| Capture Bluetooth | false | | True indicates that we support bluetooth interfaces |
+| Network Controllers | .* | | Regular expression of the network controller(s) to which we will attach |
### Relationships
-| Name | Description |
-| - | - |
-|success|All files are routed to success|
+| Name | Description |
+|---------|---------------------------------|
+| success | All files are routed to success |
## CaptureRTSPFrame
@@ -191,20 +194,50 @@ Captures a frame from the RTSP stream at specified intervals.
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Image Encoding|.jpg||The encoding that should be applied the the frame images captured from the RTSP stream|
-|RTSP Hostname|||Hostname of the RTSP stream we are trying to connect to|
-|RTSP Password|||Password used to connect to the RTSP stream|
-|RTSP Port|||Port that should be connected to to receive RTSP Frames|
-|RTSP URI|||URI that should be appended to the RTSP stream hostname|
-|RTSP Username|||The username for connecting to the RTSP stream|
+| Name | Default Value | Allowable Values | Description |
+|----------------|---------------|------------------|----------------------------------------------------------------------------------------|
+| Image Encoding | .jpg | | The encoding that should be applied the the frame images captured from the RTSP stream |
+| RTSP Hostname | | | Hostname of the RTSP stream we are trying to connect to |
+| RTSP Password | | | Password used to connect to the RTSP stream |
+| RTSP Port | | | Port that should be connected to to receive RTSP Frames |
+| RTSP URI | | | URI that should be appended to the RTSP stream hostname |
+| RTSP Username | | | The username for connecting to the RTSP stream |
+### Relationships
+
+| Name | Description |
+|---------|----------------------------------|
+| failure | Failures to capture RTSP frame |
+| success | Successful capture of RTSP frame |
+
+## CollectorInitiatedSubscription
+
+### Description
+
+Windows Event Log Subscribe Callback to receive FlowFiles from Events on Windows.
+### Properties
+
+In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
+
+| Name | Default Value | Allowable Values | Description |
+|------------------------------------|-----------------|------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Subscription Name** | | | The name of the subscription. The value provided for this parameter should be unique within the computer's scope.<br/>**Supports Expression Language: true** |
+| **Subscription Description** | | | A description of the subscription.<br/>**Supports Expression Language: true** |
+| **Source Address** | | | The IP address or fully qualified domain name (FQDN) of the local or remote computer (event source) from which the events are collected.<br/>**Supports Expression Language: true** |
+| **Source User Name** | | | The user name, which is used by the remote computer (event source) to authenticate the user.<br/>**Supports Expression Language: true** |
+| **Source Password** | | | The password, which is used by the remote computer (event source) to authenticate the user.<br/>**Supports Expression Language: true** |
+| **Source Channels** | | | The Windows Event Log Channels (on domain computer(s)) from which events are transferred.<br/>**Supports Expression Language: true** |
+| **Max Delivery Items** | 1000 | | Determines the maximum number of items that will forwarded from an event source for each request. |
+| **Delivery MaxLatency Time** | 10 min | | How long, in milliseconds, the event source should wait before sending events. |
+| **Heartbeat Interval** | 10 min | | Time interval, in milliseconds, which is observed between the sent heartbeat messages. The event collector uses this property to determine the interval between queries to the event source. |
+| **Channel** | ForwardedEvents | | The Windows Event Log Channel (on local machine) to which events are transferred.<br/>**Supports Expression Language: true** |
+| **Query** | * | | XPath Query to filter events. (See https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx for examples.)<br/>**Supports Expression Language: true** |
+| **Max Buffer Size** | 1 MB | | The individual Event Log XMLs are rendered to a buffer. This specifies the maximum size in bytes that the buffer will be allowed to grow to. (Limiting the maximum size of an individual Event XML.) |
+| **Inactive Duration To Reconnect** | 10 min | | If no new event logs are processed for the specified time period, this processor will try reconnecting to recover from a state where any further messages cannot be consumed. Such situation can happen if Windows Event Log service is restarted, or ERROR_EVT_QUERY_RESULT_STALE (15011) is returned. Setting no duration, e.g. '0 ms' disables auto-reconnection. |
### Relationships
-| Name | Description |
-| - | - |
-|failure|Failures to capture RTSP frame|
-|success|Successful capture of RTSP frame|
+| Name | Description |
+|---------|------------------------------------------------|
+| success | Relationship for successfully consumed events. |
## CompressContent
@@ -216,18 +249,18 @@ Compresses or decompresses the contents of FlowFiles using a user-specified comp
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Compression Format|use mime.type attribute||The compression format to use.|
-|Compression Level|1||The compression level to use; this is valid only when using GZIP compression.|
-|Mode|compress||Indicates whether the processor should compress content or decompress content.|
-|Update Filename|false||Determines if filename extension need to be updated|
+| Name | Default Value | Allowable Values | Description |
+|--------------------|-------------------------|------------------|--------------------------------------------------------------------------------|
+| Compression Format | use mime.type attribute | | The compression format to use. |
+| Compression Level | 1 | | The compression level to use; this is valid only when using GZIP compression. |
+| Mode | compress | | Indicates whether the processor should compress content or decompress content. |
+| Update Filename | false | | Determines if filename extension need to be updated |
### Relationships
-| Name | Description |
-| - | - |
-|failure|FlowFiles will be transferred to the failure relationship if they fail to compress/decompress|
-|success|FlowFiles will be transferred to the success relationship after successfully being compressed or decompressed|
+| Name | Description |
+|---------|---------------------------------------------------------------------------------------------------------------|
+| failure | FlowFiles will be transferred to the failure relationship if they fail to compress/decompress |
+| success | FlowFiles will be transferred to the success relationship after successfully being compressed or decompressed |
## ConsumeJournald
@@ -237,19 +270,19 @@ Consume systemd-journald journal messages. Available on Linux only.
### Properties
All properties are required with a default value, making them effectively optional. None of the properties support the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| -------------------- | --------------- | ---------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Batch Size | 1000 | Positive numbers | The maximum number of entries processed in a single execution. |
-| Payload Format | Syslog | Raw<br>Syslog | Configures flow file content formatting.<br>Raw: only the message.<br>Syslog: similar to syslog or journalctl output. |
-| Include Timestamp | true | true<br>false | Include message timestamp in the 'timestamp' attribute. |
-| Journal Type | System | User<br>System<br>Both | Type of journal to consume. |
-| Process Old Messages | false | true<br>false | Process events created before the first usage (schedule) of the processor instance. |
-| Timestamp Format | %x %X %Z | [date format](https://howardhinnant.github.io/date/date.html#to_stream_formatting) | Format string to use when creating the timestamp attribute or writing messages in the syslog format. ISO/ISO 8601/ISO8601 are equivalent to "%FT%T%Ez". |
+| Name | Default Value | Allowable Values | Description |
+|----------------------|---------------|------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Batch Size | 1000 | Positive numbers | The maximum number of entries processed in a single execution. |
+| Payload Format | Syslog | Raw<br>Syslog | Configures flow file content formatting.<br>Raw: only the message.<br>Syslog: similar to syslog or journalctl output. |
+| Include Timestamp | true | true<br>false | Include message timestamp in the 'timestamp' attribute. |
+| Journal Type | System | User<br>System<br>Both | Type of journal to consume. |
+| Process Old Messages | false | true<br>false | Process events created before the first usage (schedule) of the processor instance. |
+| Timestamp Format | %x %X %Z | [date format](https://howardhinnant.github.io/date/date.html#to_stream_formatting) | Format string to use when creating the timestamp attribute or writing messages in the syslog format. ISO/ISO 8601/ISO8601 are equivalent to "%FT%T%Ez". |
### Relationships
-| Name | Description |
-| ------- | ------------------------------ |
+| Name | Description |
+|---------|--------------------------------|
| success | Journal messages as flow files |
@@ -262,35 +295,35 @@ Consumes messages from Apache Kafka and transform them into MiNiFi FlowFiles. Th
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Duplicate Header Handling|Keep Latest|Comma-separated Merge<br>Keep First<br>Keep Latest<br>|For headers to be added as attributes, this option specifies how to handle cases where multiple headers are present with the same key. For example in case of receiving these two headers: "Accept: text/html" and "Accept: application/xml" and we want to attach the value of "Accept" as a FlowFile attribute:<br/> - "Keep First" attaches: "Accept -> text/html"<br/> - "Keep Latest" attaches: "Accept - [...]
-|**Group ID**|||A Group ID is used to identify consumers that are within the same consumer group. Corresponds to Kafka's 'group.id' property.<br/>**Supports Expression Language: true**|
-|Headers To Add As Attributes|||A comma separated list to match against all message headers. Any message header whose name matches an item from the list will be added to the FlowFile as an Attribute. If not specified, no Header values will be added as FlowFile attributes. The behaviour on when multiple headers of the same name are present is set using the DuplicateHeaderHandling attribute.|
-|**Honor Transactions**|true||Specifies whether or not MiNiFi should honor transactional guarantees when communicating with Kafka. If false, the Processor will use an "isolation level" of read_uncomitted. This means that messages will be received as soon as they are written to Kafka but will be pulled, even if the producer cancels the transactions. If this value is true, MiNiFi will not receive any messages for which the producer's transaction was canceled, but this can result in some la [...]
-|**Kafka Brokers**|localhost:9092||A comma-separated list of known Kafka Brokers in the format <host>:<port>.<br/>**Supports Expression Language: true**|
-|Kerberos Keytab Path|||The path to the location on the local filesystem where the kerberos keytab is located. Read permission on the file is required.|
-|Kerberos Principal|||Keberos Principal|
-|Kerberos Service Name|||Kerberos Service Name|
-|**Key Attribute Encoding**|UTF-8|Hex<br>UTF-8<br>|FlowFiles that are emitted have an attribute named 'kafka.key'. This property dictates how the value of the attribute should be encoded.|
-|Max Poll Records|10000||Specifies the maximum number of records Kafka should return when polling each time the processor is triggered.|
-|**Max Poll Time**|4 seconds||Specifies the maximum amount of time the consumer can use for polling data from the brokers. Polling is a blocking operation, so the upper limit of this value is specified in 4 seconds.|
-|Message Demarcator|||Since KafkaConsumer receives messages in batches, you have an option to output FlowFiles which contains all Kafka messages in a single batch for a given topic and partition and this property allows you to provide a string (interpreted as UTF-8) to use for demarcating apart multiple Kafka messages. This is an optional property and if not provided each Kafka message received will result in a single FlowFile which time it is triggered. <br/>**Supports Expression Langua [...]
-|Message Header Encoding|UTF-8|Hex<br>UTF-8<br>|Any message header that is found on a Kafka message will be added to the outbound FlowFile as an attribute. This property indicates the Character Encoding to use for deserializing the headers.|
-|**Offset Reset**|latest|earliest<br>latest<br>none<br>|Allows you to manage the condition when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted). Corresponds to Kafka's 'auto.offset.reset' property.|
-|Password|||The password for the given username when the SASL Mechanism is sasl_plaintext|
-|SASL Mechanism|GSSAPI|GSSAPI<br/>PLAIN|The SASL mechanism to use for authentication. Corresponds to Kafka's 'sasl.mechanism' property.|
-|**Security Protocol**|plaintext|plaintext<br/>ssl<br/>sasl_plaintext<br/>sasl_ssl|Protocol used to communicate with brokers. Corresponds to Kafka's 'security.protocol' property.|
-|Session Timeout|60 seconds||Client group session and failure detection timeout. The consumer sends periodic heartbeats to indicate its liveness to the broker. If no hearts are received by the broker for a group member within the session timeout, the broker will remove the consumer from the group and trigger a rebalance. The allowed range is configured with the broker configuration properties group.min.session.timeout.ms and group.max.session.timeout.ms.|
-|SSL Context Service|||SSL Context Service Name|
-|**Topic Name Format**|Names|Names<br>Patterns<br>|Specifies whether the Topic(s) provided are a comma separated list of names or a single regular expression. Using regular expressions does not automatically discover Kafka topics created after the processor started.|
-|**Topic Names**|||The name of the Kafka Topic(s) to pull from. Multiple topic names are supported as a comma separated list.<br/>**Supports Expression Language: true**|
-|Username|||The username when the SASL Mechanism is sasl_plaintext|
-### Properties
-
-| Name | Description |
-| - | - |
-|success|Incoming Kafka messages as flowfiles. Depending on the demarcation strategy, this can be one or multiple flowfiles per message.|
+| Name | Default Value | Allowable Values | Description [...]
+|------------------------------|----------------|--------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| Duplicate Header Handling | Keep Latest | Comma-separated Merge<br>Keep First<br>Keep Latest<br> | For headers to be added as attributes, this option specifies how to handle cases where multiple headers are present with the same key. For example in case of receiving these two headers: "Accept: text/html" and "Accept: application/xml" and we want to attach the value of "Accept" as a FlowFile attribute:<br/> - "Keep First" attaches: "Accept -> text/html"<br/> - "Keep Latest" attach [...]
+| **Group ID** | | | A Group ID is used to identify consumers that are within the same consumer group. Corresponds to Kafka's 'group.id' property.<br/>**Supports Expression Language: true** [...]
+| Headers To Add As Attributes | | | A comma separated list to match against all message headers. Any message header whose name matches an item from the list will be added to the FlowFile as an Attribute. If not specified, no Header values will be added as FlowFile attributes. The behaviour on when multiple headers of the same name are present is set using the DuplicateHeaderHandling attribute. [...]
+| **Honor Transactions** | true | | Specifies whether or not MiNiFi should honor transactional guarantees when communicating with Kafka. If false, the Processor will use an "isolation level" of read_uncomitted. This means that messages will be received as soon as they are written to Kafka but will be pulled, even if the producer cancels the transactions. If this value is true, MiNiFi will not receive any messages for [...]
+| **Kafka Brokers** | localhost:9092 | | A comma-separated list of known Kafka Brokers in the format <host>:<port>.<br/>**Supports Expression Language: true** [...]
+| Kerberos Keytab Path | | | The path to the location on the local filesystem where the kerberos keytab is located. Read permission on the file is required. [...]
+| Kerberos Principal | | | Keberos Principal [...]
+| Kerberos Service Name | | | Kerberos Service Name [...]
+| **Key Attribute Encoding** | UTF-8 | Hex<br>UTF-8<br> | FlowFiles that are emitted have an attribute named 'kafka.key'. This property dictates how the value of the attribute should be encoded. [...]
+| Max Poll Records | 10000 | | Specifies the maximum number of records Kafka should return when polling each time the processor is triggered. [...]
+| **Max Poll Time** | 4 seconds | | Specifies the maximum amount of time the consumer can use for polling data from the brokers. Polling is a blocking operation, so the upper limit of this value is specified in 4 seconds. [...]
+| Message Demarcator | | | Since KafkaConsumer receives messages in batches, you have an option to output FlowFiles which contains all Kafka messages in a single batch for a given topic and partition and this property allows you to provide a string (interpreted as UTF-8) to use for demarcating apart multiple Kafka messages. This is an optional property and if not provided each Kafka message received will resu [...]
+| Message Header Encoding | UTF-8 | Hex<br>UTF-8<br> | Any message header that is found on a Kafka message will be added to the outbound FlowFile as an attribute. This property indicates the Character Encoding to use for deserializing the headers. [...]
+| **Offset Reset** | latest | earliest<br>latest<br>none<br> | Allows you to manage the condition when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted). Corresponds to Kafka's 'auto.offset.reset' property. [...]
+| Password | | | The password for the given username when the SASL Mechanism is sasl_plaintext [...]
+| SASL Mechanism | GSSAPI | GSSAPI<br/>PLAIN | The SASL mechanism to use for authentication. Corresponds to Kafka's 'sasl.mechanism' property. [...]
+| **Security Protocol** | plaintext | plaintext<br/>ssl<br/>sasl_plaintext<br/>sasl_ssl | Protocol used to communicate with brokers. Corresponds to Kafka's 'security.protocol' property. [...]
+| Session Timeout | 60 seconds | | Client group session and failure detection timeout. The consumer sends periodic heartbeats to indicate its liveness to the broker. If no hearts are received by the broker for a group member within the session timeout, the broker will remove the consumer from the group and trigger a rebalance. The allowed range is configured with the broker configuration properties group.min.session. [...]
+| SSL Context Service | | | SSL Context Service Name [...]
+| **Topic Name Format** | Names | Names<br>Patterns<br> | Specifies whether the Topic(s) provided are a comma separated list of names or a single regular expression. Using regular expressions does not automatically discover Kafka topics created after the processor started. [...]
+| **Topic Names** | | | The name of the Kafka Topic(s) to pull from. Multiple topic names are supported as a comma separated list.<br/>**Supports Expression Language: true** [...]
+| Username | | | The username when the SASL Mechanism is sasl_plaintext [...]
+### Properties
+
+| Name | Description |
+|---------|---------------------------------------------------------------------------------------------------------------------------------|
+| success | Incoming Kafka messages as flowfiles. Depending on the demarcation strategy, this can be one or multiple flowfiles per message. |
## ConsumeMQTT
@@ -301,24 +334,58 @@ This Processor gets the contents of a FlowFile from a MQTT broker for a specifie
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Broker URI|||The URI to use to connect to the MQTT broker|
-|Client ID|||MQTT client ID to use|
-|Connection Timeout|30 sec||Maximum time interval the client will wait for the network connection to the MQTT server|
-|Keep Alive Interval|60 sec||Defines the maximum time interval between messages sent or received|
-|Max Flow Segment Size|||Maximum flow content payload segment size for the MQTT record|
-|Password|||Password to use when connecting to the broker|
-|Quality of Service|MQTT_QOS_0||The Quality of Service(QoS) to send the message with. Accepts three values '0', '1' and '2'|
-|Queue Max Message|||Maximum number of messages allowed on the received MQTT queue|
-|Session state|true||Whether to start afresh or resume previous flows. See the allowable value descriptions for more details|
-|Topic|||The topic to publish the message to|
-|Username|||Username to use when connecting to the broker|
-### Relationships
-
-| Name | Description |
-| - | - |
-|success|FlowFiles that are sent successfully to the destination are transferred to this relationship|
+| Name | Default Value | Allowable Values | Description |
+|-----------------------|---------------|------------------|---------------------------------------------------------------------------------------------------------|
+| Broker URI | | | The URI to use to connect to the MQTT broker |
+| Client ID | | | MQTT client ID to use |
+| Connection Timeout | 30 sec | | Maximum time interval the client will wait for the network connection to the MQTT server |
+| Keep Alive Interval | 60 sec | | Defines the maximum time interval between messages sent or received |
+| Max Flow Segment Size | | | Maximum flow content payload segment size for the MQTT record |
+| Password | | | Password to use when connecting to the broker |
+| Quality of Service | MQTT_QOS_0 | | The Quality of Service(QoS) to send the message with. Accepts three values '0', '1' and '2' |
+| Queue Max Message | | | Maximum number of messages allowed on the received MQTT queue |
+| Session state | true | | Whether to start afresh or resume previous flows. See the allowable value descriptions for more details |
+| Topic | | | The topic to publish the message to |
+| Username | | | Username to use when connecting to the broker |
+### Relationships
+
+| Name | Description |
+|---------|----------------------------------------------------------------------------------------------|
+| success | FlowFiles that are sent successfully to the destination are transferred to this relationship |
+
+
+## ConsumeWindowsEventLog
+
+### Description
+
+Windows Event Log Subscribe Callback to receive FlowFiles from Events on Windows.
+### Properties
+
+In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
+
+| Name | Default Value | Allowable Values | Description [...]
+|------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| **Channel** | System | | The Windows Event Log Channel to listen to.<br/>**Supports Expression Language: true** [...]
+| **Query** | * | | XPath Query to filter events. (See https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx for examples.)<br/>**Supports Expression Language: true** [...]
+| **Max Buffer Size** | 1 MB | | The individual Event Log XMLs are rendered to a buffer. This specifies the maximum size in bytes that the buffer will be allowed to grow to. (Limiting the maximum size of an individual Event XML.) [...]
+| **Inactive Duration To Reconnect** | 10 min | | If no new event logs are processed for the specified time period, this processor will try reconnecting to recover from a state where any further messages cannot be consumed. Such situation can hap [...]
+| Identifier Match Regex | .*Sid | | Regular Expression to match Subject Identifier Fields. These will be placed into the attributes of the FlowFile [...]
+| Apply Identifier Function | true | true<br>false | If true it will resolve SIDs matched in the 'Identifier Match Regex' to the DOMAIN\USERNAME associated with that ID [...]
+| Resolve Metadata in Attributes | true | true<br>false | If true, any metadata that is resolved ( such as IDs or keyword metadata ) will be placed into attributes, otherwise it will be replaced in the XML or text output [...]
+| Event Header Delimiter | | | If set, the chosen delimiter will be used in the Event output header. Otherwise, a colon followed by spaces will be used. [...]
+| Event Header | LOG_NAME=Log Name, SOURCE = Source, TIME_CREATED = Date,EVENT_RECORDID=Record ID,EVENTID = Event ID,TASK_CATEGORY = Task Category,LEVEL = Level,KEYWORDS = Keywords,USER = User,COMPUTER = Computer, EVENT_TYPE = EventType | | Comma seperated list of key/value pairs with the following keys LOG_NAME, SOURCE, TIME_CREATED,EVENT_RECORDID,EVENTID,TASK_CATEGORY,LEVEL,KEYWORDS,USER,COMPUTER, and EVENT_TYPE. Eliminating fields [...]
+| **Output Format** | Both | XML<br>Plaintext<br>JSON<br>Both | Set the output format type. In case 'Both' is selected the processor generates two flow files for every event captured in format XML and Plaintext [...]
+| **JSON Format** | Simple | Simple<br>Raw<br>Flattened | Set the json format type. Only applicable if Output Format is set to 'JSON' [...]
+| Batch Commit Size | 1000 | | Maximum number of Events to consume and create to Flow Files from before committing. [...]
+| **Process Old Events** | false | true<br>false | This property defines if old events (which are created before first time server is started) should be processed. [...]
+| State Directory | CWELState | | DEPRECATED. Only use it for state migration from the state file, supplying the legacy state directory. [...]
+
+### Relationships
+
+| Name | Description |
+|---------|------------------------------------------------|
+| success | Relationship for successfully consumed events. |
+
## DefragmentText
@@ -329,19 +396,19 @@ DefragmentText splits and merges incoming flowfiles so cohesive messages are not
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Pattern**|||A regular expression to match at the start or end of messages.|
-|Pattern Location|Start of Message|Start of Message<br>End of Message|Whether the pattern is located at the start or at the end of the messages.|
-|Max Buffer Age|10 min|<duration> <time unit>|The maximum age of the buffer after which it will be transferred to success when matching Start of Message patterns or to failure when matching End of Message patterns.|
-|Max Buffer Size||<size> <size unit>|The maximum buffer size, if the buffer exceed this, it will be transferred to failure.|
+| Name | Default Value | Allowable Values | Description |
+|------------------|------------------|------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Pattern** | | | A regular expression to match at the start or end of messages. |
+| Pattern Location | Start of Message | Start of Message<br>End of Message | Whether the pattern is located at the start or at the end of the messages. |
+| Max Buffer Age | 10 min | <duration> <time unit> | The maximum age of the buffer after which it will be transferred to success when matching Start of Message patterns or to failure when matching End of Message patterns. |
+| Max Buffer Size | | <size> <size unit> | The maximum buffer size, if the buffer exceed this, it will be transferred to failure. |
### Relationships
-| Name | Description |
-| - | - |
-|success|Flowfiles that have been successfully defragmented|
-|failure|Flowfiles that failed the defragmentation process|
+| Name | Description |
+|---------|----------------------------------------------------|
+| success | Flowfiles that have been successfully defragmented |
+| failure | Flowfiles that failed the defragmentation process |
## DeleteAzureBlobStorage
@@ -352,25 +419,25 @@ Deletes the provided blob from Azure Storage
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Azure Storage Credentials Service|||Name of the Azure Storage Credentials Service used to retrieve the connection string from.|
-|**Blob**|||The filename of the blob. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true**|
-|Common Storage Account Endpoint Suffix|||Storage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).<br/>**Supports Expression Language: true**|
-|Connection String|||Connection string used to connect to Azure Storage service. This overrides all other set credential properties if Managed Identity is not used.<br/>**Supports Expression Language: true**|
-|**Container Name**|||Name of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.<br/>**Supports Expression Language: true**|
-|**Delete Snapshots Option**|None|None<br/>Include Snapshots<br/>Delete Snapshots Only|Specifies the snapshot deletion options to be used when deleting a blob.<br/>None: Deletes the blob only.<br/>Include Snapshots: Delete the blob and its snapshots.<br/>Delete Snapshots Only: Delete only the blob's snapshots.|
-|SAS Token|||Shared Access Signature token. Specify either SAS Token (recommended) or Storage Account Key together with Storage Account Name if Managed Identity is not used.<br/>**Supports Expression Language: true**|
-|Storage Account Key|||The storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token instead for fine-grained control with policies.<br/>**Supports Expression Language: true**|
-|Storage Account Name|||The storage account name.<br/>**Supports Expression Language: true**|
-|**Use Managed Identity Credentials**|false||If true Managed Identity credentials will be used together with the Storage Account Name for authentication.|
+| Name | Default Value | Allowable Values | Description |
+|----------------------------------------|---------------|------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Azure Storage Credentials Service | | | Name of the Azure Storage Credentials Service used to retrieve the connection string from. |
+| **Blob** | | | The filename of the blob. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true** |
+| Common Storage Account Endpoint Suffix | | | Storage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).<br/>**Supports Expression Language: true** |
+| Connection String | | | Connection string used to connect to Azure Storage service. This overrides all other set credential properties if Managed Identity is not used.<br/>**Supports Expression Language: true** |
+| **Container Name** | | | Name of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.<br/>**Supports Expression Language: true** |
+| **Delete Snapshots Option** | None | None<br/>Include Snapshots<br/>Delete Snapshots Only | Specifies the snapshot deletion options to be used when deleting a blob.<br/>None: Deletes the blob only.<br/>Include Snapshots: Delete the blob and its snapshots.<br/>Delete Snapshots Only: Delete only the blob's snapshots. |
+| SAS Token | | | Shared Access Signature token. Specify either SAS Token (recommended) or Storage Account Key together with Storage Account Name if Managed Identity is not used.<br/>**Supports Expression Language: true** |
+| Storage Account Key | | | The storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token instead for fine-grained control with policies.<br/>**Supports Expression Language: true** |
+| Storage Account Name | | | The storage account name.<br/>**Supports Expression Language: true** |
+| **Use Managed Identity Credentials** | false | | If true Managed Identity credentials will be used together with the Storage Account Name for authentication. |
### Relationships
-| Name | Description |
-| - | - |
-|failure|Unsuccessful operations will be transferred to the failure relationship|
-|success|All successfully processed FlowFiles are routed to this relationship|
+| Name | Description |
+|---------|-------------------------------------------------------------------------|
+| failure | Unsuccessful operations will be transferred to the failure relationship |
+| success | All successfully processed FlowFiles are routed to this relationship |
## DeleteAzureDataLakeStorage
@@ -382,19 +449,19 @@ Deletes the provided file from Azure Data Lake Storage Gen 2
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Azure Storage Credentials Service**|||Name of the Azure Storage Credentials Service used to retrieve the connection string from.|
-|File Name|||The filename in Azure Storage. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true**|
-|**Filesystem Name**|||Name of the Azure Storage File System. It is assumed to be already existing.<br/>**Supports Expression Language: true**|
-|Directory Name|||Name of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. If left empty it designates the root directory. The directory will be created if not already existing.<br/>**Supports Expression Language: true**|
+| Name | Default Value | Allowable Values | Description |
+|---------------------------------------|---------------|------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Azure Storage Credentials Service** | | | Name of the Azure Storage Credentials Service used to retrieve the connection string from. |
+| File Name | | | The filename in Azure Storage. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true** |
+| **Filesystem Name** | | | Name of the Azure Storage File System. It is assumed to be already existing.<br/>**Supports Expression Language: true** |
+| Directory Name | | | Name of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. If left empty it designates the root directory. The directory will be created if not already existing.<br/>**Supports Expression Language: true** |
### Relationships
-| Name | Description |
-| - | - |
-|failure|If file deletion from Azure Data Lake storage fails the flowfile is transferred to this relationship|
-|success|If file deletion from Azure Data Lake storage succeeds the flowfile is transferred to this relationship|
+| Name | Description |
+|---------|---------------------------------------------------------------------------------------------------------|
+| failure | If file deletion from Azure Data Lake storage fails the flowfile is transferred to this relationship |
+| success | If file deletion from Azure Data Lake storage succeeds the flowfile is transferred to this relationship |
## DeleteGCSObject
@@ -442,28 +509,28 @@ Deletes FlowFiles on an Amazon S3 Bucket. If attempting to delete a file that do
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | <div style="width:7em">Default Value</div> | <div style="width:8em">Allowable Values</div> | Description |
-| - | - | - | - |
-|Object Key|||The key of the S3 object. If none is given the filename attribute will be used by default.<br/>**Supports Expression Language: true**|
-|**Bucket**|||The S3 bucket<br/>**Supports Expression Language: true**|
-|Access Key|||AWS account access key<br/>**Supports Expression Language: true**|
-|Secret Key|||AWS account secret key<br/>**Supports Expression Language: true**|
-|Credentials File|||Path to a file containing AWS access key and secret key in properties file format. Properties used: accessKey and secretKey|
-|AWS Credentials Provider service|||The name of the AWS Credentials Provider controller service that is used to obtain AWS credentials.|
-|**Region**|us-west-2|af-south-1<br/>ap-east-1<br/>ap-northeast-1<br/>ap-northeast-2<br/>ap-northeast-3<br/>ap-south-1<br/>ap-southeast-1<br/>ap-southeast-2<br/>ca-central-1<br/>cn-north-1<br/>cn-northwest-1<br/>eu-central-1<br/>eu-north-1<br/>eu-south-1<br/>eu-west-1<br/>eu-west-2<br/>eu-west-3<br/>me-south-1<br/>sa-east-1<br/>us-east-1<br/>us-east-2<br/>us-gov-east-1<br/>us-gov-west-1<br/>us-west-1<br/>us-west-2|AWS Region|
-|**Communications Timeout**|30 sec||Sets the timeout of the communication between the AWS server and the client|
-|Endpoint Override URL|||Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints.<br/>**Supports Expression Language: true**|
-|Proxy Host|||Proxy host name or IP<br/>**Supports Expression Language: true**|
-|Proxy Port|||The port number of the proxy host<br/>**Supports Expression Language: true**|
-|Proxy Username|||Username to set when authenticating against proxy<br/>**Supports Expression Language: true**|
-|Proxy Password|||Password to set when authenticating against proxy<br/>**Supports Expression Language: true**|
-|Version|||The Version of the Object to delete<br/>**Supports Expression Language: true**|
+| Name | <div style="width:7em">Default Value</div> | <div style="width:8em">Allowable Values</div> | Description [...]
+|----------------------------------|--------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------- [...]
+| Object Key | | | The key of t [...]
+| **Bucket** | | | The S3 bucke [...]
+| Access Key | | | AWS account [...]
+| Secret Key | | | AWS account [...]
+| Credentials File | | | Path to a fi [...]
+| AWS Credentials Provider service | | | The name of [...]
+| **Region** | us-west-2 | af-south-1<br/>ap-east-1<br/>ap-northeast-1<br/>ap-northeast-2<br/>ap-northeast-3<br/>ap-south-1<br/>ap-southeast-1<br/>ap-southeast-2<br/>ca-central-1<br/>cn-north-1<br/>cn-northwest-1<br/>eu-central-1<br/>eu-north-1<br/>eu-south-1<br/>eu-west-1<br/>eu-west-2<br/>eu-west-3<br/>me-south-1<br/>sa-east-1<br/>us-east-1<br/>us-east-2<br/>us-gov-east-1<br/>us-gov-west-1<br/>us-west-1<br/>us-west-2 | AWS Region [...]
+| **Communications Timeout** | 30 sec | | Sets the tim [...]
+| Endpoint Override URL | | | Endpoint URL [...]
+| Proxy Host | | | Proxy host n [...]
+| Proxy Port | | | The port num [...]
+| Proxy Username | | | Username to [...]
+| Proxy Password | | | Password to [...]
+| Version | | | The Version [...]
### Relationships
-| Name | Description |
-| - | - |
-|failure|FlowFiles are routed to failure relationship|
-|success|FlowFiles are routed to success relationship|
+| Name | Description |
+|---------|----------------------------------------------|
+| failure | FlowFiles are routed to failure relationship |
+| success | FlowFiles are routed to success relationship |
## ExecuteProcess
@@ -475,18 +542,18 @@ Runs an operating system command specified by the user and writes the output of
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Batch Duration|0 sec||If the process is expected to be long-running and produce textual output, a batch duration can be specified.|
-|Command|||Specifies the command to be executed; if just the name of an executable is provided, it must be in the user's environment PATH.<br/>**Supports Expression Language: true**|
-|Command Arguments|||The arguments to supply to the executable delimited by white space. White space can be escaped by enclosing it in double-quotes.<br/>**Supports Expression Language: true**|
-|Redirect Error Stream|false||If true will redirect any error stream output of the process to the output stream.|
-|Working Directory|||The directory to use as the current working directory when executing the command<br/>**Supports Expression Language: true**|
+| Name | Default Value | Allowable Values | Description |
+|-----------------------|---------------|------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Batch Duration | 0 sec | | If the process is expected to be long-running and produce textual output, a batch duration can be specified. |
+| Command | | | Specifies the command to be executed; if just the name of an executable is provided, it must be in the user's environment PATH.<br/>**Supports Expression Language: true** |
+| Command Arguments | | | The arguments to supply to the executable delimited by white space. White space can be escaped by enclosing it in double-quotes.<br/>**Supports Expression Language: true** |
+| Redirect Error Stream | false | | If true will redirect any error stream output of the process to the output stream. |
+| Working Directory | | | The directory to use as the current working directory when executing the command<br/>**Supports Expression Language: true** |
### Relationships
-| Name | Description |
-| - | - |
-|success|All created FlowFiles are routed to this relationship.|
+| Name | Description |
+|---------|--------------------------------------------------------|
+| success | All created FlowFiles are routed to this relationship. |
## ExecutePythonProcessor
@@ -499,18 +566,18 @@ Executes a script given the flow file and a process session. The script is respo
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Module Directory|||Comma-separated list of paths to files and/or directories which contain modules required by the script|
-|**Reload on Script Change**|true||If true and Script File property is used, then script file will be reloaded if it has changed, otherwise the first loaded version will be used at all times.|
-|Script Body|||Script to execute|
-|Script File|||Path to script file to execute. Only one of Script File or Script Body may be used|
+| Name | Default Value | Allowable Values | Description |
+|-----------------------------|---------------|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Module Directory | | | Comma-separated list of paths to files and/or directories which contain modules required by the script |
+| **Reload on Script Change** | true | | If true and Script File property is used, then script file will be reloaded if it has changed, otherwise the first loaded version will be used at all times. |
+| Script Body | | | Script to execute |
+| Script File | | | Path to script file to execute. Only one of Script File or Script Body may be used |
### Relationships
-| Name | Description |
-| - | - |
-|failure|Script failures|
-|success|Script successes|
+| Name | Description |
+|---------|------------------|
+| failure | Script failures |
+| success | Script successes |
## ExecuteSQL
@@ -522,17 +589,17 @@ Execute provided SQL query. Query result rows will be outputted as new flow file
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**DB Controller Service**|||Database Controller Service.|
-|**Max Rows Per Flow File**|0||The maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile.|
-|**Output Format**|JSON-Pretty|JSON<br/>JSON-Pretty|Set the output format type.|
-|SQL select query|||The SQL select query to execute. The query can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming flowfiles. If this property is empty, the content of the incoming flow file is expected to contain a valid SQL select query, to be issued by the processor to the database.|
+| Name | Default Value | Allowable Values | Description |
+|----------------------------|---------------|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **DB Controller Service** | | | Database Controller Service. |
+| **Max Rows Per Flow File** | 0 | | The maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile. |
+| **Output Format** | JSON-Pretty | JSON<br/>JSON-Pretty | Set the output format type. |
+| SQL select query | | | The SQL select query to execute. The query can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming flowfiles. If this property is empty, the content of the incoming flow file is expected to contain a valid SQL select query, to be issued by the processor to the database. |
### Relationships
-| Name | Description |
-| - | - |
-|success|Successfully created FlowFile from SQL query result set.|
+| Name | Description |
+|---------|----------------------------------------------------------|
+| success | Successfully created FlowFile from SQL query result set. |
## ExecuteScript
@@ -568,20 +635,20 @@ Extracts the content of a FlowFile and places it into an attribute.
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Attribute|||Attribute to set from content|
-|Enable Case-insensitive Matching|false||Indicates that two characters match even if they are in a different case. |
-|Enable repeating capture group|false||f set to true, every string matching the capture groups will be extracted. Otherwise, if the Regular Expression matches more than once, only the first match will be extracted.|
-|Include Capture Group 0|true||Indicates that Capture Group 0 should be included as an attribute. Capture Group 0 represents the entirety of the regular expression match, is typically not used, and could have considerable length.|
-|Maximum Capture Group Length|1024||Specifies the maximum number of characters a given capture group value can have. Any characters beyond the max will be truncated.|
-|Regex Mode|false||Set this to extract parts of flowfile content using regular experssions in dynamic properties|
-|Size Limit|2097152||Maximum number of bytes to read into the attribute. 0 for no limit. Default is 2MB.|
+| Name | Default Value | Allowable Values | Description |
+|----------------------------------|---------------|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Attribute | | | Attribute to set from content |
+| Enable Case-insensitive Matching | false | | Indicates that two characters match even if they are in a different case. |
+| Enable repeating capture group | false | | f set to true, every string matching the capture groups will be extracted. Otherwise, if the Regular Expression matches more than once, only the first match will be extracted. |
+| Include Capture Group 0 | true | | Indicates that Capture Group 0 should be included as an attribute. Capture Group 0 represents the entirety of the regular expression match, is typically not used, and could have considerable length. |
+| Maximum Capture Group Length | 1024 | | Specifies the maximum number of characters a given capture group value can have. Any characters beyond the max will be truncated. |
+| Regex Mode | false | | Set this to extract parts of flowfile content using regular experssions in dynamic properties |
+| Size Limit | 2097152 | | Maximum number of bytes to read into the attribute. 0 for no limit. Default is 2MB. |
### Relationships
-| Name | Description |
-| - | - |
-|success|success operational on the flow record|
+| Name | Description |
+|---------|----------------------------------------|
+| success | success operational on the flow record |
## FetchAzureBlobStorage
@@ -593,26 +660,26 @@ Retrieves contents of an Azure Storage Blob, writing the contents to the content
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Azure Storage Credentials Service|||Name of the Azure Storage Credentials Service used to retrieve the connection string from.|
-|**Blob**|||The filename of the blob. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true**|
-|Common Storage Account Endpoint Suffix|||Storage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).<br/>**Supports Expression Language: true**|
-|Connection String|||Connection string used to connect to Azure Storage service. This overrides all other set credential properties if Managed Identity is not used.<br/>**Supports Expression Language: true**|
-|**Container Name**|||Name of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.<br/>**Supports Expression Language: true**|
-|Range Length|||The number of bytes to download from the blob, starting from the Range Start. An empty value or a value that extends beyond the end of the blob will read to the end of the blob.<br/>**Supports Expression Language: true**|
-|Range Start|||The byte position at which to start reading from the blob. An empty value or a value of zero will start reading at the beginning of the blob.<br/>**Supports Expression Language: true**|
-|SAS Token|||Shared Access Signature token. Specify either SAS Token (recommended) or Storage Account Key together with Storage Account Name if Managed Identity is not used.<br/>**Supports Expression Language: true**|
-|Storage Account Key|||The storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token instead for fine-grained control with policies.<br/>**Supports Expression Language: true**|
-|Storage Account Name|||The storage account name.<br/>**Supports Expression Language: true**|
-|**Use Managed Identity Credentials**|false||If true Managed Identity credentials will be used together with the Storage Account Name for authentication.|
+| Name | Default Value | Allowable Values | Description |
+|----------------------------------------|---------------|------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Azure Storage Credentials Service | | | Name of the Azure Storage Credentials Service used to retrieve the connection string from. |
+| **Blob** | | | The filename of the blob. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true** |
+| Common Storage Account Endpoint Suffix | | | Storage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).<br/>**Supports Expression Language: true** |
+| Connection String | | | Connection string used to connect to Azure Storage service. This overrides all other set credential properties if Managed Identity is not used.<br/>**Supports Expression Language: true** |
+| **Container Name** | | | Name of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.<br/>**Supports Expression Language: true** |
+| Range Length | | | The number of bytes to download from the blob, starting from the Range Start. An empty value or a value that extends beyond the end of the blob will read to the end of the blob.<br/>**Supports Expression Language: true** |
+| Range Start | | | The byte position at which to start reading from the blob. An empty value or a value of zero will start reading at the beginning of the blob.<br/>**Supports Expression Language: true** |
+| SAS Token | | | Shared Access Signature token. Specify either SAS Token (recommended) or Storage Account Key together with Storage Account Name if Managed Identity is not used.<br/>**Supports Expression Language: true** |
+| Storage Account Key | | | The storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token instead for fine-grained control with policies.<br/>**Supports Expression Language: true** |
+| Storage Account Name | | | The storage account name.<br/>**Supports Expression Language: true** |
+| **Use Managed Identity Credentials** | false | | If true Managed Identity credentials will be used together with the Storage Account Name for authentication. |
### Relationships
-| Name | Description |
-| - | - |
-|failure|Unsuccessful operations will be transferred to the failure relationship|
-|success|All successfully processed FlowFiles are routed to this relationship|
+| Name | Description |
+|---------|-------------------------------------------------------------------------|
+| failure | Unsuccessful operations will be transferred to the failure relationship |
+| success | All successfully processed FlowFiles are routed to this relationship |
## FetchAzureDataLakeStorage
@@ -624,22 +691,22 @@ Fetch the provided file from Azure Data Lake Storage Gen 2
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Azure Storage Credentials Service**|||Name of the Azure Storage Credentials Service used to retrieve the connection string from.|
-|File Name|||The filename in Azure Storage. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true**|
-|**Filesystem Name**|||Name of the Azure Storage File System. It is assumed to be already existing.<br/>**Supports Expression Language: true**|
-|Directory Name|||Name of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. If left empty it designates the root directory. The directory will be created if not already existing.<br/>**Supports Expression Language: true**|
-|Range Start|||The byte position at which to start reading from the object. An empty value or a value of zero will start reading at the beginning of the object.<br/>**Supports Expression Language: true**|
-|Range Length|||The number of bytes to download from the object, starting from the Range Start. An empty value or a value that extends beyond the end of the object will read to the end of the object.<br/>**Supports Expression Language: true**|
-|Number of Retries|0||The number of automatic retries to perform if the download fails.<br/>**Supports Expression Language: true**|
+| Name | Default Value | Allowable Values | Description |
+|---------------------------------------|---------------|------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Azure Storage Credentials Service** | | | Name of the Azure Storage Credentials Service used to retrieve the connection string from. |
+| File Name | | | The filename in Azure Storage. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true** |
+| **Filesystem Name** | | | Name of the Azure Storage File System. It is assumed to be already existing.<br/>**Supports Expression Language: true** |
+| Directory Name | | | Name of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. If left empty it designates the root directory. The directory will be created if not already existing.<br/>**Supports Expression Language: true** |
+| Range Start | | | The byte position at which to start reading from the object. An empty value or a value of zero will start reading at the beginning of the object.<br/>**Supports Expression Language: true** |
+| Range Length | | | The number of bytes to download from the object, starting from the Range Start. An empty value or a value that extends beyond the end of the object will read to the end of the object.<br/>**Supports Expression Language: true** |
+| Number of Retries | 0 | | The number of automatic retries to perform if the download fails.<br/>**Supports Expression Language: true** |
### Relationships
-| Name | Description |
-| - | - |
-|failure|In case of fetch failure flowfiles are transferred to this relationship|
-|success|Files that have been successfully fetched from Azure storage are transferred to this relationship|
+| Name | Description |
+|---------|---------------------------------------------------------------------------------------------------|
+| failure | In case of fetch failure flowfiles are transferred to this relationship |
+| success | Files that have been successfully fetched from Azure storage are transferred to this relationship |
## FetchGCSObject
@@ -688,23 +755,23 @@ Reads the contents of a file from disk and streams it into the contents of an in
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|File to Fetch|||The fully-qualified filename of the file to fetch from the file system. If not defined the default ${absolute.path}/${filename} path is used.<br/>**Supports Expression Language: true**|
-|**Completion Strategy**|None|None<br/>Move File<br/>Delete File|Specifies what to do with the original file on the file system once it has been pulled into MiNiFi|
-|Move Destination Directory|||The directory to move the original file to once it has been fetched from the file system. This property is ignored unless the Completion Strategy is set to "Move File". If the directory does not exist, it will be created.<br/>**Supports Expression Language: true**|
-|**Move Conflict Strategy**|Rename|Rename<br/>Replace File<br/>Keep Existing<br/>Fail|If Completion Strategy is set to Move File and a file already exists in the destination directory with the same name, this property specifies how that naming conflict should be resolved|
-|**Log level when file not found**|ERROR|TRACE<br/>DEBUG<br/>INFO<br/>WARN<br/>ERROR<br/>OFF|Log level to use in case the file does not exist when the processor is triggered|
-|**Log level when permission denied**|ERROR|TRACE<br/>DEBUG<br/>INFO<br/>WARN<br/>ERROR<br/>OFF|Log level to use in case agent does not have sufficient permissions to read the file|
+| Name | Default Value | Allowable Values | Description |
+|--------------------------------------|---------------|-----------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| File to Fetch | | | The fully-qualified filename of the file to fetch from the file system. If not defined the default ${absolute.path}/${filename} path is used.<br/>**Supports Expression Language: true** |
+| **Completion Strategy** | None | None<br/>Move File<br/>Delete File | Specifies what to do with the original file on the file system once it has been pulled into MiNiFi |
+| Move Destination Directory | | | The directory to move the original file to once it has been fetched from the file system. This property is ignored unless the Completion Strategy is set to "Move File". If the directory does not exist, it will be created.<br/>**Supports Expression Language: true** |
+| **Move Conflict Strategy** | Rename | Rename<br/>Replace File<br/>Keep Existing<br/>Fail | If Completion Strategy is set to Move File and a file already exists in the destination directory with the same name, this property specifies how that naming conflict should be resolved |
+| **Log level when file not found** | ERROR | TRACE<br/>DEBUG<br/>INFO<br/>WARN<br/>ERROR<br/>OFF | Log level to use in case the file does not exist when the processor is triggered |
+| **Log level when permission denied** | ERROR | TRACE<br/>DEBUG<br/>INFO<br/>WARN<br/>ERROR<br/>OFF | Log level to use in case agent does not have sufficient permissions to read the file |
### Relationships
-| Name | Description |
-| - | - |
-|success|Any FlowFile that is successfully fetched from the file system will be transferred to this Relationship.|
-|not.found|Any FlowFile that could not be fetched from the file system because the file could not be found will be transferred to this Relationship.|
-|permission.denied|Any FlowFile that could not be fetched from the file system due to the user running MiNiFi not having sufficient permissions will be transferred to this Relationship.|
-|failure|Any FlowFile that could not be fetched from the file system for any reason other than insufficient permissions or the file not existing will be transferred to this Relationship.|
+| Name | Description |
+|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| success | Any FlowFile that is successfully fetched from the file system will be transferred to this Relationship. |
+| not.found | Any FlowFile that could not be fetched from the file system because the file could not be found will be transferred to this Relationship. |
+| permission.denied | Any FlowFile that could not be fetched from the file system due to the user running MiNiFi not having sufficient permissions will be transferred to this Relationship. |
+| failure | Any FlowFile that could not be fetched from the file system for any reason other than insufficient permissions or the file not existing will be transferred to this Relationship. |
## FetchOPCProcessor
@@ -716,26 +783,26 @@ Fetches OPC-UA node
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Application URI|||Application URI of the client in the format 'urn:unconfigured:application'. Mandatory, if using Secure Channel and must match the URI included in the certificate's Subject Alternative Names.|
-|Certificate path|||Path to the DER-encoded cert file|
-|Key path|||Path to the DER-encoded key file|
-|**Lazy mode**|Off|Off<br>On<br>|Only creates flowfiles from nodes with new timestamp from the server.|
-|Max depth|0||Specifiec the max depth of browsing. 0 means unlimited.|
-|Namespace index|0||The index of the namespace. Used only if node ID type is not path.|
-|**Node ID**|||Specifies the ID of the root node to traverse|
-|**Node ID type**||Int<br>Path<br>String<br>|Specifies the type of the provided node ID|
-|**OPC server endpoint**|||Specifies the address, port and relative path of an OPC endpoint|
-|Password|||Password to log in with.|
-|Trusted server certificate path|||Path to the DER-encoded trusted server certificate|
-|Username|||Username to log in with.|
-### Relationships
-
-| Name | Description |
-| - | - |
-|failure|Retrieved OPC-UA nodes where value cannot be extracted (only if enabled)|
-|success|Successfully retrieved OPC-UA nodes|
+| Name | Default Value | Allowable Values | Description |
+|---------------------------------|---------------|---------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Application URI | | | Application URI of the client in the format 'urn:unconfigured:application'. Mandatory, if using Secure Channel and must match the URI included in the certificate's Subject Alternative Names. |
+| Certificate path | | | Path to the DER-encoded cert file |
+| Key path | | | Path to the DER-encoded key file |
+| **Lazy mode** | Off | Off<br>On<br> | Only creates flowfiles from nodes with new timestamp from the server. |
+| Max depth | 0 | | Specifiec the max depth of browsing. 0 means unlimited. |
+| Namespace index | 0 | | The index of the namespace. Used only if node ID type is not path. |
+| **Node ID** | | | Specifies the ID of the root node to traverse |
+| **Node ID type** | | Int<br>Path<br>String<br> | Specifies the type of the provided node ID |
+| **OPC server endpoint** | | | Specifies the address, port and relative path of an OPC endpoint |
+| Password | | | Password to log in with. |
+| Trusted server certificate path | | | Path to the DER-encoded trusted server certificate |
+| Username | | | Username to log in with. |
+### Relationships
+
+| Name | Description |
+|---------|--------------------------------------------------------------------------|
+| failure | Retrieved OPC-UA nodes where value cannot be extracted (only if enabled) |
+| success | Successfully retrieved OPC-UA nodes |
## FetchS3Object
@@ -747,29 +814,29 @@ Retrieves the contents of an S3 Object and writes it to the content of a FlowFil
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | <div style="width:7em">Default Value</div> | <div style="width:8em">Allowable Values</div> | Description |
-| - | - | - | - |
-|Object Key|||The key of the S3 object. If none is given the filename attribute will be used by default.<br/>**Supports Expression Language: true**|
-|**Bucket**|||The S3 bucket<br/>**Supports Expression Language: true**|
-|Access Key|||AWS account access key<br/>**Supports Expression Language: true**|
-|Secret Key|||AWS account secret key<br/>**Supports Expression Language: true**|
-|Credentials File|||Path to a file containing AWS access key and secret key in properties file format. Properties used: accessKey and secretKey|
-|AWS Credentials Provider service|||The name of the AWS Credentials Provider controller service that is used to obtain AWS credentials.|
-|**Region**|us-west-2|af-south-1<br/>ap-east-1<br/>ap-northeast-1<br/>ap-northeast-2<br/>ap-northeast-3<br/>ap-south-1<br/>ap-southeast-1<br/>ap-southeast-2<br/>ca-central-1<br/>cn-north-1<br/>cn-northwest-1<br/>eu-central-1<br/>eu-north-1<br/>eu-south-1<br/>eu-west-1<br/>eu-west-2<br/>eu-west-3<br/>me-south-1<br/>sa-east-1<br/>us-east-1<br/>us-east-2<br/>us-gov-east-1<br/>us-gov-west-1<br/>us-west-1<br/>us-west-2|AWS Region|
-|**Communications Timeout**|30 sec||Sets the timeout of the communication between the AWS server and the client|
-|Endpoint Override URL|||Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints.<br/>**Supports Expression Language: true**|
-|Proxy Host|||Proxy host name or IP<br/>**Supports Expression Language: true**|
-|Proxy Port|||The port number of the proxy host<br/>**Supports Expression Language: true**|
-|Proxy Username|||Username to set when authenticating against proxy<br/>**Supports Expression Language: true**|
-|Proxy Password|||Password to set when authenticating against proxy<br/>**Supports Expression Language: true**|
-|Version|||The Version of the Object to download<br/>**Supports Expression Language: true**|
-|**Requester Pays**|false||If true, indicates that the requester consents to pay any charges associated with retrieving objects from the S3 bucket. This sets the 'x-amz-request-payer' header to 'requester'.|
+| Name | <div style="width:7em">Default Value</div> | <div style="width:8em">Allowable Values</div> | Description [...]
+|----------------------------------|--------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------- [...]
+| Object Key | | | The key of t [...]
+| **Bucket** | | | The S3 bucke [...]
+| Access Key | | | AWS account [...]
+| Secret Key | | | AWS account [...]
+| Credentials File | | | Path to a fi [...]
+| AWS Credentials Provider service | | | The name of [...]
+| **Region** | us-west-2 | af-south-1<br/>ap-east-1<br/>ap-northeast-1<br/>ap-northeast-2<br/>ap-northeast-3<br/>ap-south-1<br/>ap-southeast-1<br/>ap-southeast-2<br/>ca-central-1<br/>cn-north-1<br/>cn-northwest-1<br/>eu-central-1<br/>eu-north-1<br/>eu-south-1<br/>eu-west-1<br/>eu-west-2<br/>eu-west-3<br/>me-south-1<br/>sa-east-1<br/>us-east-1<br/>us-east-2<br/>us-gov-east-1<br/>us-gov-west-1<br/>us-west-1<br/>us-west-2 | AWS Region [...]
+| **Communications Timeout** | 30 sec | | Sets the tim [...]
+| Endpoint Override URL | | | Endpoint URL [...]
+| Proxy Host | | | Proxy host n [...]
+| Proxy Port | | | The port num [...]
+| Proxy Username | | | Username to [...]
+| Proxy Password | | | Password to [...]
+| Version | | | The Version [...]
+| **Requester Pays** | false | | If true, ind [...]
### Relationships
-| Name | Description |
-| - | - |
-|failure|FlowFiles are routed to failure relationship|
-|success|FlowFiles are routed to success relationship|
+| Name | Description |
+|---------|----------------------------------------------|
+| failure | FlowFiles are routed to failure relationship |
+| success | FlowFiles are routed to success relationship |
## FetchSFTP
@@ -781,38 +848,38 @@ Fetches the content of a file from a remote SFTP server and overwrites the conte
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Completion Strategy**|None|Delete File<br>Move File<br>None<br>|Specifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.|
-|**Connection Timeout**|30 sec||Amount of time to wait before timing out while creating a connection|
-|**Create Directory**|false||Specifies whether or not the remote directory should be created if it does not exist.|
-|**Data Timeout**|30 sec||When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems|
-|Disable Directory Listing|false||Control how 'Move Destination Directory' is created when 'Completion Strategy' is 'Move File' and 'Create Directory' is enabled. If set to 'true', directory listing is not performed prior to create missing directories. By default, this processor executes a directory listing command to see target directory existence before creating missing directories. However, there are situations that you might need to disable the directory listing such as the following [...]
-|Host Key File|||If supplied, the given file will be used as the Host Key; otherwise, no use host key file will be used|
-|**Hostname**|||The fully qualified hostname or IP address of the remote system<br/>**Supports Expression Language: true**|
-|Http Proxy Password|||Http Proxy Password<br/>**Supports Expression Language: true**|
-|Http Proxy Username|||Http Proxy Username<br/>**Supports Expression Language: true**|
-|Move Destination Directory|||The directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.<br/>**Supports Expression Language: true**|
-|Password|||Password for the user account<br/>**Supports Expression Language: true**|
-|**Port**|||The port that the remote system is listening on for file transfers<br/>**Supports Expression Language: true**|
-|Private Key Passphrase|||Password for the private key<br/>**Supports Expression Language: true**|
-|Private Key Path|||The fully qualified path to the Private Key file<br/>**Supports Expression Language: true**|
-|Proxy Host|||The fully qualified hostname or IP address of the proxy server<br/>**Supports Expression Language: true**|
-|Proxy Port|||The port of the proxy server<br/>**Supports Expression Language: true**|
-|Proxy Type|DIRECT|DIRECT<br>HTTP<br>SOCKS<br>|Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthN|
-|**Remote File**|||The fully qualified filename on the remote system<br/>**Supports Expression Language: true**|
-|**Send Keep Alive On Timeout**|true||Indicates whether or not to send a single Keep Alive message when SSH socket times out|
-|**Strict Host Key Checking**|false||Indicates whether or not strict enforcement of hosts keys should be applied|
-|**Use Compression**|false||Indicates whether or not ZLIB compression should be used when transferring files|
-|**Username**|||Username<br/>**Supports Expression Language: true**|
-### Relationships
-
-| Name | Description |
-| - | - |
-|comms.failure|Any FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.|
-|not.found|Any FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.|
-|permission.denied|Any FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship.|
-|success|All FlowFiles that are received are routed to success|
+| Name | Default Value | Allowable Values | Description [...]
+|--------------------------------|---------------|--------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| **Completion Strategy** | None | Delete File<br>Move File<br>None<br> | Specifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred. [...]
+| **Connection Timeout** | 30 sec | | Amount of time to wait before timing out while creating a connection [...]
+| **Create Directory** | false | | Specifies whether or not the remote directory should be created if it does not exist. [...]
+| **Data Timeout** | 30 sec | | When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems [...]
+| Disable Directory Listing | false | | Control how 'Move Destination Directory' is created when 'Completion Strategy' is 'Move File' and 'Create Directory' is enabled. If set to 'true', directory listing is not performed prior to create missing directories. By default, this processor executes a directory listing command to see target directory existence before creating missing directories. However, there are situations that you might nee [...]
+| Host Key File | | | If supplied, the given file will be used as the Host Key; otherwise, no use host key file will be used [...]
+| **Hostname** | | | The fully qualified hostname or IP address of the remote system<br/>**Supports Expression Language: true** [...]
+| Http Proxy Password | | | Http Proxy Password<br/>**Supports Expression Language: true** [...]
+| Http Proxy Username | | | Http Proxy Username<br/>**Supports Expression Language: true** [...]
+| Move Destination Directory | | | The directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.<br/>**Supports Expression Language: true** [...]
+| Password | | | Password for the user account<br/>**Supports Expression Language: true** [...]
+| **Port** | | | The port that the remote system is listening on for file transfers<br/>**Supports Expression Language: true** [...]
+| Private Key Passphrase | | | Password for the private key<br/>**Supports Expression Language: true** [...]
+| Private Key Path | | | The fully qualified path to the Private Key file<br/>**Supports Expression Language: true** [...]
+| Proxy Host | | | The fully qualified hostname or IP address of the proxy server<br/>**Supports Expression Language: true** [...]
+| Proxy Port | | | The port of the proxy server<br/>**Supports Expression Language: true** [...]
+| Proxy Type | DIRECT | DIRECT<br>HTTP<br>SOCKS<br> | Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthN [...]
+| **Remote File** | | | The fully qualified filename on the remote system<br/>**Supports Expression Language: true** [...]
+| **Send Keep Alive On Timeout** | true | | Indicates whether or not to send a single Keep Alive message when SSH socket times out [...]
+| **Strict Host Key Checking** | false | | Indicates whether or not strict enforcement of hosts keys should be applied [...]
+| **Use Compression** | false | | Indicates whether or not ZLIB compression should be used when transferring files [...]
+| **Username** | | | Username<br/>**Supports Expression Language: true** [...]
+### Relationships
+
+| Name | Description |
+|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------|
+| comms.failure | Any FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship. |
+| not.found | Any FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship. |
+| permission.denied | Any FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship. |
+| success | All FlowFiles that are received are routed to success |
## FocusArchiveEntry
@@ -824,14 +891,14 @@ Allows manipulation of entries within an archive (e.g. TAR) by focusing on one e
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Path|||The path within the archive to focus ("/" to focus the total archive)|
+| Name | Default Value | Allowable Values | Description |
+|------|---------------|------------------|-----------------------------------------------------------------------|
+| Path | | | The path within the archive to focus ("/" to focus the total archive) |
### Relationships
-| Name | Description |
-| - | - |
-|success|success operational on the flow record|
+| Name | Description |
+|---------|----------------------------------------|
+| success | success operational on the flow record |
## GenerateFlowFile
@@ -843,18 +910,18 @@ This processor creates FlowFiles with random data or custom content. GenerateFlo
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Batch Size|1||The number of FlowFiles to be transferred in each invocation|
-|Data Format|Binary|Text<br>Binary<br>|Specifies whether the data should be Text or Binary|
-|File Size|1 kB||The size of the file that will be used|
-|Unique FlowFiles|true||If true, each FlowFile that is generated will be unique. If false, a random value will be generated and all FlowFiles|
-|Custom Text|||If Data Format is text and if Unique FlowFiles is false, then this custom text will be used as content of the generated FlowFiles and the File Size will be ignored. Finally, if Expression Language is used, evaluation will be performed only once per batch of generated FlowFiles<br/>**Supports Expression Language: true**|
+| Name | Default Value | Allowable Values | Description |
+|------------------|---------------|--------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Batch Size | 1 | | The number of FlowFiles to be transferred in each invocation |
+| Data Format | Binary | Text<br>Binary<br> | Specifies whether the data should be Text or Binary |
+| File Size | 1 kB | | The size of the file that will be used |
+| Unique FlowFiles | true | | If true, each FlowFile that is generated will be unique. If false, a random value will be generated and all FlowFiles |
+| Custom Text | | | If Data Format is text and if Unique FlowFiles is false, then this custom text will be used as content of the generated FlowFiles and the File Size will be ignored. Finally, if Expression Language is used, evaluation will be performed only once per batch of generated FlowFiles<br/>**Supports Expression Language: true** |
### Relationships
-| Name | Description |
-| - | - |
-|success|success operational on the flow record|
+| Name | Description |
+|---------|----------------------------------------|
+| success | success operational on the flow record |
## GetFile
@@ -866,24 +933,24 @@ Creates FlowFiles from files in a directory. MiNiFi will ignore files for which
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Batch Size|10||The maximum number of files to pull in each iteration|
-|File Filter|[^\.].*||Only files whose names match the given regular expression will be picked up|
-|Ignore Hidden Files|true||Indicates whether or not hidden files should be ignored|
-|**Input Directory**|||The input directory from which to pull files<br/>**Supports Expression Language: true**|
-|Keep Source File|false||If true, the file is not deleted after it has been copied to the Content Repository|
-|Maximum File Age|0 sec||The maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignored|
-|Maximum File Size|0 B||The maximum size that a file can be in order to be pulled|
-|Minimum File Age|0 sec||The minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored|
-|Minimum File Size|0 B||The minimum size that a file can be in order to be pulled|
-|Polling Interval|0 sec||Indicates how long to wait before performing a directory listing|
-|Recurse Subdirectories|true||Indicates whether or not to pull files from subdirectories|
+| Name | Default Value | Allowable Values | Description |
+|------------------------|---------------|------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Batch Size | 10 | | The maximum number of files to pull in each iteration |
+| File Filter | [^\.].* | | Only files whose names match the given regular expression will be picked up |
+| Ignore Hidden Files | true | | Indicates whether or not hidden files should be ignored |
+| **Input Directory** | | | The input directory from which to pull files<br/>**Supports Expression Language: true** |
+| Keep Source File | false | | If true, the file is not deleted after it has been copied to the Content Repository |
+| Maximum File Age | 0 sec | | The maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignored |
+| Maximum File Size | 0 B | | The maximum size that a file can be in order to be pulled |
+| Minimum File Age | 0 sec | | The minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored |
+| Minimum File Size | 0 B | | The minimum size that a file can be in order to be pulled |
+| Polling Interval | 0 sec | | Indicates how long to wait before performing a directory listing |
+| Recurse Subdirectories | true | | Indicates whether or not to pull files from subdirectories |
### Relationships
-| Name | Description |
-| - | - |
-|success|All files are routed to success|
+| Name | Description |
+|---------|---------------------------------|
+| success | All files are routed to success |
## GetGPS
@@ -895,16 +962,16 @@ Obtains GPS coordinates from the GPSDHost and port.
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|GPSD Host|localhost||The host running the GPSD daemon|
-|GPSD Port|2947||The GPSD daemon port|
-|GPSD Wait Time|50000000||Timeout value for waiting for data from the GPSD instance|
+| Name | Default Value | Allowable Values | Description |
+|----------------|---------------|------------------|-----------------------------------------------------------|
+| GPSD Host | localhost | | The host running the GPSD daemon |
+| GPSD Port | 2947 | | The GPSD daemon port |
+| GPSD Wait Time | 50000000 | | Timeout value for waiting for data from the GPSD instance |
### Relationships
-| Name | Description |
-| - | - |
-|success|All files are routed to success|
+| Name | Description |
+|---------|---------------------------------|
+| success | All files are routed to success |
## GetTCP
@@ -916,21 +983,21 @@ Establishes a TCP Server that defines and retrieves one or more byte messages fr
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|SSL Context Service|||SSL Context Service Name|
-|Stay Connected|true||Determines if we keep the same socket despite having no data|
-|concurrent-handler-count|1||Number of concurrent handlers for this session|
-|connection-attempt-timeout|3||Maximum number of connection attempts before attempting backup hosts, if configured|
-|end-of-message-byte|13||Byte value which denotes end of message. Must be specified as integer within the valid byte range (-128 thru 127). For example, '13' = Carriage return and '10' = New line. Default '13'.|
-|**endpoint-list**|||A comma delimited list of the endpoints to connect to. The format should be <server_address>:<port>.|
-|receive-buffer-size|16 MB||The size of the buffer to receive data in. Default 16384 (16MB).|
+| Name | Default Value | Allowable Values | Description |
+|----------------------------|---------------|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| SSL Context Service | | | SSL Context Service Name |
+| Stay Connected | true | | Determines if we keep the same socket despite having no data |
+| concurrent-handler-count | 1 | | Number of concurrent handlers for this session |
+| connection-attempt-timeout | 3 | | Maximum number of connection attempts before attempting backup hosts, if configured |
+| end-of-message-byte | 13 | | Byte value which denotes end of message. Must be specified as integer within the valid byte range (-128 thru 127). For example, '13' = Carriage return and '10' = New line. Default '13'. |
+| **endpoint-list** | | | A comma delimited list of the endpoints to connect to. The format should be <server_address>:<port>. |
+| receive-buffer-size | 16 MB | | The size of the buffer to receive data in. Default 16384 (16MB). |
### Relationships
-| Name | Description |
-| - | - |
-|partial|Indicates an incomplete message as a result of encountering the end of message byte trigger|
-|success|All files are routed to success|
+| Name | Description |
+|---------|---------------------------------------------------------------------------------------------|
+| partial | Indicates an incomplete message as a result of encountering the end of message byte trigger |
+| success | All files are routed to success |
## GetUSBCamera
@@ -942,21 +1009,21 @@ Gets images from USB Video Class (UVC)-compatible devices. Outputs one flow file
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|FPS|1||Frames per second to capture from USB camera|
-|Format|PNG||Frame format (currently only PNG and RAW are supported; RAW is a binary pixel buffer of RGB values)|
-|Height|||Target height of image to capture from USB camera|
-|USB Product ID|0x0||USB Product ID of camera device, in hexadecimal format|
-|USB Serial No.|||USB Serial No. of camera device|
-|USB Vendor ID|0x0||USB Vendor ID of camera device, in hexadecimal format|
-|Width|||Target width of image to capture from USB camera|
+| Name | Default Value | Allowable Values | Description |
+|----------------|---------------|------------------|-----------------------------------------------------------------------------------------------------|
+| FPS | 1 | | Frames per second to capture from USB camera |
+| Format | PNG | | Frame format (currently only PNG and RAW are supported; RAW is a binary pixel buffer of RGB values) |
+| Height | | | Target height of image to capture from USB camera |
+| USB Product ID | 0x0 | | USB Product ID of camera device, in hexadecimal format |
+| USB Serial No. | | | USB Serial No. of camera device |
+| USB Vendor ID | 0x0 | | USB Vendor ID of camera device, in hexadecimal format |
+| Width | | | Target width of image to capture from USB camera |
### Relationships
-| Name | Description |
-| - | - |
-|failure|Failures sent here|
-|success|Sucessfully captured images sent here|
+| Name | Description |
+|---------|---------------------------------------|
+| failure | Failures sent here |
+| success | Sucessfully captured images sent here |
## HashContent
@@ -968,17 +1035,17 @@ HashContent calculates the checksum of the content of the flowfile and adds it a
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Hash Algorithm|SHA256||Name of the algorithm used to generate checksum|
-|Hash Attribute|Checksum||Attribute to store checksum to|
-|Fail on empty|false||Route to failure relationship in case of empty content|
+| Name | Default Value | Allowable Values | Description |
+|----------------|---------------|------------------|--------------------------------------------------------|
+| Hash Algorithm | SHA256 | | Name of the algorithm used to generate checksum |
+| Hash Attribute | Checksum | | Attribute to store checksum to |
+| Fail on empty | false | | Route to failure relationship in case of empty content |
### Properties
-| Name | Description |
-| - | - |
-|failure|failure operational on the flow record|
-|success|success operational on the flow record|
+| Name | Description |
+|---------|----------------------------------------|
+| failure | failure operational on the flow record |
+| success | success operational on the flow record |
## InvokeHTTP
@@ -990,38 +1057,38 @@ An HTTP client processor which can interact with a configurable HTTP Endpoint. T
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Always Output Response|false||Will force a response FlowFile to be generated and routed to the 'Response' relationship regardless of what the server status code received is |
-|Attributes to Send|||Regular expression that defines which attributes to send as HTTP headers in the request. If not defined, no attributes are sent as headers.|
-|Connection Timeout|5 secs||Max wait time for connection to remote service.|
-|Content-type|application/octet-stream||The Content-Type to specify for when content is being transmitted through a PUT, POST or PATCH. In the case of an empty value after evaluating an expression language expression, Content-Type defaults to|
-|Disable Peer Verification|false||Disables peer verification for the SSL session|
-|Follow Redirects|true||Follow HTTP redirects issued by remote server.|
-|HTTP Method|GET||HTTP request method (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS). Arbitrary methods are also supported. Methods other than POST, PUT and PATCH will be sent without a message body.|
-|Include Date Header|true||Include an RFC-2616 Date header in the request.|
-|**Invalid HTTP Header Field Handling Strategy**|transform|transform<br/>fail<br/>drop|Indicates what should happen when an attribute's name is not a valid HTTP header field name.<br/>Options:<br/>transform - invalid characters are replaced<br/>fail - flow file is transferred to failure<br/>drop - drops invalid attributes from HTTP message|
-|invokehttp-proxy-password|||Password to set when authenticating against proxy|
-|invokehttp-proxy-username|||Username to set when authenticating against proxy|
-|Penalize on "No Retry"|false||Enabling this property will penalize FlowFiles that are routed to the "No Retry" relationship.|
-|Proxy Host|||The fully qualified hostname or IP address of the proxy server|
-|Proxy Port|||The port of the proxy server|
-|Put Response Body in Attribute|||If set, the response body received back will be put into an attribute of the original FlowFile instead of a separate FlowFile. The attribute key to put to is determined by evaluating value of this property.|
-|Read Timeout|15 secs||Max wait time for response from remote service.|
-|Remote URL|||Remote URL which will be connected to, including scheme, host, port, path.<br/>**Supports Expression Language: true**|
-|send-message-body|true||DEPRECATED. Only kept for backwards compatibility, no functionality is included.|
-|Send Message Body|true||If true, sends the HTTP message body on POST/PUT/PATCH requests (default). If false, suppresses the message body and content-type header for these requests.|
-|SSL Context Service|||The SSL Context Service used to provide client certificate information for TLS/SSL (https) connections.|
-|Use Chunked Encoding|false||When POST'ing, PUT'ing or PATCH'ing content set this property to true in order to not pass the 'Content-length' header and instead send 'Transfer-Encoding' with a value of 'chunked'. This will enable the data transfer mechanism which was introduced in HTTP 1.1 to pass data of unknown lengths in chunks.|
-### Relationships
-
-| Name | Description |
-| - | - |
-|success|The original FlowFile will be routed upon success (2xx status codes). It will have new attributes detailing the success of the request.|
-|response|A Response FlowFile will be routed upon success (2xx status codes). If the 'Always Output Response' property is true then the response will be sent to this relationship regardless of the status code received.|
-|retry|The original FlowFile will be routed on any status code that can be retried (5xx status codes). It will have new attributes detailing the request.|
-|no retry|The original FlowFile will be routed on any status code that should NOT be retried (1xx, 3xx, 4xx status codes). It will have new attributes detailing the request.|
-|failure|The original FlowFile will be routed on any type of connection failure, timeout or general exception. It will have new attributes detailing the request.|
+| Name | Default Value | Allowable Values | Description |
+|-------------------------------------------------|--------------------------|-----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Always Output Response | false | | Will force a response FlowFile to be generated and routed to the 'Response' relationship regardless of what the server status code received is |
+| Attributes to Send | | | Regular expression that defines which attributes to send as HTTP headers in the request. If not defined, no attributes are sent as headers. |
+| Connection Timeout | 5 secs | | Max wait time for connection to remote service. |
+| Content-type | application/octet-stream | | The Content-Type to specify for when content is being transmitted through a PUT, POST or PATCH. In the case of an empty value after evaluating an expression language expression, Content-Type defaults to |
+| Disable Peer Verification | false | | Disables peer verification for the SSL session |
+| Follow Redirects | true | | Follow HTTP redirects issued by remote server. |
+| HTTP Method | GET | | HTTP request method (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS). Arbitrary methods are also supported. Methods other than POST, PUT and PATCH will be sent without a message body. |
+| Include Date Header | true | | Include an RFC-2616 Date header in the request. |
+| **Invalid HTTP Header Field Handling Strategy** | transform | transform<br/>fail<br/>drop | Indicates what should happen when an attribute's name is not a valid HTTP header field name.<br/>Options:<br/>transform - invalid characters are replaced<br/>fail - flow file is transferred to failure<br/>drop - drops invalid attributes from HTTP message |
+| invokehttp-proxy-password | | | Password to set when authenticating against proxy |
+| invokehttp-proxy-username | | | Username to set when authenticating against proxy |
+| Penalize on "No Retry" | false | | Enabling this property will penalize FlowFiles that are routed to the "No Retry" relationship. |
+| Proxy Host | | | The fully qualified hostname or IP address of the proxy server |
+| Proxy Port | | | The port of the proxy server |
+| Put Response Body in Attribute | | | If set, the response body received back will be put into an attribute of the original FlowFile instead of a separate FlowFile. The attribute key to put to is determined by evaluating value of this property. |
+| Read Timeout | 15 secs | | Max wait time for response from remote service. |
+| Remote URL | | | Remote URL which will be connected to, including scheme, host, port, path.<br/>**Supports Expression Language: true** |
+| send-message-body | true | | DEPRECATED. Only kept for backwards compatibility, no functionality is included. |
+| Send Message Body | true | | If true, sends the HTTP message body on POST/PUT/PATCH requests (default). If false, suppresses the message body and content-type header for these requests. |
+| SSL Context Service | | | The SSL Context Service used to provide client certificate information for TLS/SSL (https) connections. |
+| Use Chunked Encoding | false | | When POST'ing, PUT'ing or PATCH'ing content set this property to true in order to not pass the 'Content-length' header and instead send 'Transfer-Encoding' with a value of 'chunked'. This will enable the data transfer mechanism which was introduced in HTTP 1.1 to pass data of unknown lengths in chunks. |
+### Relationships
+
+| Name | Description |
+|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| success | The original FlowFile will be routed upon success (2xx status codes). It will have new attributes detailing the success of the request. |
+| response | A Response FlowFile will be routed upon success (2xx status codes). If the 'Always Output Response' property is true then the response will be sent to this relationship regardless of the status code received. |
+| retry | The original FlowFile will be routed on any status code that can be retried (5xx status codes). It will have new attributes detailing the request. |
+| no retry | The original FlowFile will be routed on any status code that should NOT be retried (1xx, 3xx, 4xx status codes). It will have new attributes detailing the request. |
+| failure | The original FlowFile will be routed on any type of connection failure, timeout or general exception. It will have new attributes detailing the request. |
## ListAzureBlobStorage
@@ -1033,24 +1100,24 @@ Lists blobs in an Azure Storage container. Listing details are attached to an em
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Azure Storage Credentials Service|||Name of the Azure Storage Credentials Service used to retrieve the connection string from.|
-|Common Storage Account Endpoint Suffix|||Storage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).<br/>**Supports Expression Language: true**|
-|Connection String|||Connection string used to connect to Azure Storage service. This overrides all other set credential properties if Managed Identity is not used.<br/>**Supports Expression Language: true**|
-|**Container Name**|||Name of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.<br/>**Supports Expression Language: true**|
-|Listing Strategy|timestamps|none<br/>timestamps|Specify how to determine new/updated entities. If 'timestamps' is selected it tracks the latest timestamp of listed entity to determine new/updated entities. If 'none' is selected it lists an entity without any tracking, the same entity will be listed each time on executing this processor.|
-|Prefix|||Search prefix for listing<br/>**Supports Expression Language: true**|
-|SAS Token|||Shared Access Signature token. Specify either SAS Token (recommended) or Storage Account Key together with Storage Account Name if Managed Identity is not used.<br/>**Supports Expression Language: true**|
-|Storage Account Key|||The storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token instead for fine-grained control with policies.<br/>**Supports Expression Language: true**|
-|Storage Account Name|||The storage account name.<br/>**Supports Expression Language: true**|
-|**Use Managed Identity Credentials**|false||If true Managed Identity credentials will be used together with the Storage Account Name for authentication.|
+| Name | Default Value | Allowable Values | Description |
+|----------------------------------------|---------------|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Azure Storage Credentials Service | | | Name of the Azure Storage Credentials Service used to retrieve the connection string from. |
+| Common Storage Account Endpoint Suffix | | | Storage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).<br/>**Supports Expression Language: true** |
+| Connection String | | | Connection string used to connect to Azure Storage service. This overrides all other set credential properties if Managed Identity is not used.<br/>**Supports Expression Language: true** |
+| **Container Name** | | | Name of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.<br/>**Supports Expression Language: true** |
+| Listing Strategy | timestamps | none<br/>timestamps | Specify how to determine new/updated entities. If 'timestamps' is selected it tracks the latest timestamp of listed entity to determine new/updated entities. If 'none' is selected it lists an entity without any tracking, the same entity will be listed each time on executing this processor. |
+| Prefix | | | Search prefix for listing<br/>**Supports Expression Language: true** |
+| SAS Token | | | Shared Access Signature token. Specify either SAS Token (recommended) or Storage Account Key together with Storage Account Name if Managed Identity is not used.<br/>**Supports Expression Language: true** |
+| Storage Account Key | | | The storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token instead for fine-grained control with policies.<br/>**Supports Expression Language: true** |
+| Storage Account Name | | | The storage account name.<br/>**Supports Expression Language: true** |
+| **Use Managed Identity Credentials** | false | | If true Managed Identity credentials will be used together with the Storage Account Name for authentication. |
### Relationships
-| Name | Description |
-| - | - |
-|success|All FlowFiles that are received are routed to success|
+| Name | Description |
+|---------|-------------------------------------------------------|
+| success | All FlowFiles that are received are routed to success |
## ListAzureDataLakeStorage
@@ -1062,21 +1129,21 @@ Lists directory in an Azure Data Lake Storage Gen 2 filesystem
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Azure Storage Credentials Service**|||Name of the Azure Storage Credentials Service used to retrieve the connection string from.|
-|**Filesystem Name**|||Name of the Azure Storage File System. It is assumed to be already existing.<br/>**Supports Expression Language: true**|
-|Directory Name|||Name of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. If left empty it designates the root directory. The directory will be created if not already existing.<br/>**Supports Expression Language: true**|
-|**Recurse Subdirectories**|true||Indicates whether to list files from subdirectories of the directory|
-|File Filter|||Only files whose names match the given regular expression will be listed|
-|Path Filter|||When 'Recurse Subdirectories' is true, then only subdirectories whose paths match the given regular expression will be scanned|
-|Listing Strategy|timestamps|none<br/>timestamps|Specify how to determine new/updated entities. If 'timestamps' is selected it tracks the latest timestamp of listed entity to determine new/updated entities. If 'none' is selected it lists an entity without any tracking, the same entity will be listed each time on executing this processor.|
+| Name | Default Value | Allowable Values | Description |
+|---------------------------------------|---------------|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Azure Storage Credentials Service** | | | Name of the Azure Storage Credentials Service used to retrieve the connection string from. |
+| **Filesystem Name** | | | Name of the Azure Storage File System. It is assumed to be already existing.<br/>**Supports Expression Language: true** |
+| Directory Name | | | Name of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. If left empty it designates the root directory. The directory will be created if not already existing.<br/>**Supports Expression Language: true** |
+| **Recurse Subdirectories** | true | | Indicates whether to list files from subdirectories of the directory |
+| File Filter | | | Only files whose names match the given regular expression will be listed |
+| Path Filter | | | When 'Recurse Subdirectories' is true, then only subdirectories whose paths match the given regular expression will be scanned |
+| Listing Strategy | timestamps | none<br/>timestamps | Specify how to determine new/updated entities. If 'timestamps' is selected it tracks the latest timestamp of listed entity to determine new/updated entities. If 'none' is selected it lists an entity without any tracking, the same entity will be listed each time on executing this processor. |
### Relationships
-| Name | Description |
-| - | - |
-|success|All FlowFiles that are received are routed to success|
+| Name | Description |
+|---------|-------------------------------------------------------|
+| success | All FlowFiles that are received are routed to success |
## ListGCSBucket
@@ -1141,23 +1208,23 @@ Starts an HTTP Server and listens on a given base path to transform incoming req
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Authorized DN Pattern|.*||A Regular Expression to apply against the Distinguished Name of incoming connections. If the Pattern does not match the DN, the connection will be refused.|
-|Base Path|contentListener||Base path for incoming connections|
-|Batch Size|0||Maximum number of buffered requests to be processed in a single batch. If set to zero all buffered requests are processed.|
-|Buffer Size|0||Maximum number of HTTP Requests allowed to be buffered before processing them when the processor is triggered. If the buffer full, the request is refused. If set to zero the buffer is unlimited.|
-|HTTP Headers to receive as Attributes (Regex)|||Specifies the Regular Expression that determines the names of HTTP Headers that should be passed along as FlowFile attributes|
-|**Listening Port**|80||The Port to listen on for incoming connections. 0 means port is going to be selected randomly.|
-|SSL Certificate|||File containing PEM-formatted file including TLS/SSL certificate and key|
-|SSL Certificate Authority|||File containing trusted PEM-formatted certificates|
-|SSL Minimum Version|TLS1.2|TLS1.2<br>|Minimum TLS/SSL version allowed (TLS1.2)|
-|SSL Verify Peer|no|no<br>yes<br>|Whether or not to verify the client's certificate (yes/no)|
+| Name | Default Value | Allowable Values | Description |
+|-----------------------------------------------|-----------------|------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Authorized DN Pattern | .* | | A Regular Expression to apply against the Distinguished Name of incoming connections. If the Pattern does not match the DN, the connection will be refused. |
+| Base Path | contentListener | | Base path for incoming connections |
+| Batch Size | 0 | | Maximum number of buffered requests to be processed in a single batch. If set to zero all buffered requests are processed. |
+| Buffer Size | 0 | | Maximum number of HTTP Requests allowed to be buffered before processing them when the processor is triggered. If the buffer full, the request is refused. If set to zero the buffer is unlimited. |
+| HTTP Headers to receive as Attributes (Regex) | | | Specifies the Regular Expression that determines the names of HTTP Headers that should be passed along as FlowFile attributes |
+| **Listening Port** | 80 | | The Port to listen on for incoming connections. 0 means port is going to be selected randomly. |
+| SSL Certificate | | | File containing PEM-formatted file including TLS/SSL certificate and key |
+| SSL Certificate Authority | | | File containing trusted PEM-formatted certificates |
+| SSL Minimum Version | TLS1.2 | TLS1.2<br> | Minimum TLS/SSL version allowed (TLS1.2) |
+| SSL Verify Peer | no | no<br>yes<br> | Whether or not to verify the client's certificate (yes/no) |
### Relationships
-| Name | Description |
-| - | - |
-|success|All files are routed to success|
+| Name | Description |
+|---------|---------------------------------|
+| success | All files are routed to success |
## ListenSyslog
@@ -1221,35 +1288,35 @@ Retrieves a listing of files from the local filesystem. For each file that is li
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Input Directory**|||The input directory from which files to pull files|
-|**Recurse Subdirectories**|true||Indicates whether to list files from subdirectories of the directory|
-|File Filter|||Only files whose names match the given regular expression will be picked up|
-|Path Filter|||When Recurse Subdirectories is true, then only subdirectories whose path matches the given regular expression will be scanned|
-|**Minimum File Age**|0 sec||The minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored|
-|Maximum File Age|||The maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignored|
-|**Minimum File Size**|0 B||The minimum size that a file must be in order to be pulled|
-|Maximum File Size|||The maximum size that a file can be in order to be pulled|
-|**Ignore Hidden Files**|true||Indicates whether or not hidden files should be ignored|
-### Relationships
-
-| Name | Description |
-| - | - |
-|success|All FlowFiles that are received are routed to success|
+| Name | Default Value | Allowable Values | Description |
+|----------------------------|---------------|------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Input Directory** | | | The input directory from which files to pull files |
+| **Recurse Subdirectories** | true | | Indicates whether to list files from subdirectories of the directory |
+| File Filter | | | Only files whose names match the given regular expression will be picked up |
+| Path Filter | | | When Recurse Subdirectories is true, then only subdirectories whose path matches the given regular expression will be scanned |
+| **Minimum File Age** | 0 sec | | The minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored |
+| Maximum File Age | | | The maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignored |
+| **Minimum File Size** | 0 B | | The minimum size that a file must be in order to be pulled |
+| Maximum File Size | | | The maximum size that a file can be in order to be pulled |
+| **Ignore Hidden Files** | true | | Indicates whether or not hidden files should be ignored |
+### Relationships
+
+| Name | Description |
+|---------|-------------------------------------------------------|
+| success | All FlowFiles that are received are routed to success |
### Output Attributes
-| Attribute | Relationship | Description |
-|----------------------------|--------------|--------------------------------------------------------------------|
-| _filename_ | success | The name of the file that was read from filesystem. |
-| _path_ | success | The path is set to the relative path of the file's directory on filesystem compared to the Input Directory property. For example, if Input Directory is set to /tmp, then files picked up from /tmp will have the path attribute set to "./". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "abc/1/2/3/". |
-| _absolute.path_ | success | The absolute.path is set to the absolute path of the file's directory on filesystem. For example, if the Input Directory property is set to /tmp, then files picked up from /tmp will have the path attribute set to "/tmp/". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "/tmp/abc/1/2/3/". |
-| _file.owner_ | success | The user that owns the file in filesystem |
-| _file.group_ | success | The group that owns the file in filesystem |
-| _file.size_ | success | The number of bytes in the file in filesystem |
-| _file.permissions_ | success | The permissions for the file in filesystem. This is formatted as 3 characters for the owner, 3 for the group, and 3 for other users. For example rw-rw-r-- |
-| _file.lastModifiedTime_ | success | The timestamp of when the file in filesystem was last modified as 'yyyy-MM-dd'T'HH:mm:ssZ' |
+| Attribute | Relationship | Description |
+|-------------------------|--------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| _filename_ | success | The name of the file that was read from filesystem. |
+| _path_ | success | The path is set to the relative path of the file's directory on filesystem compared to the Input Directory property. For example, if Input Directory is set to /tmp, then files picked up from /tmp will have the path attribute set to "./". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "abc/1/2/3/". |
+| _absolute.path_ | success | The absolute.path is set to the absolute path of the file's directory on filesystem. For example, if the Input Directory property is set to /tmp, then files picked up from /tmp will have the path attribute set to "/tmp/". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "/tmp/abc/1/2/3/". |
+| _file.owner_ | success | The user that owns the file in filesystem |
+| _file.group_ | success | The group that owns the file in filesystem |
+| _file.size_ | success | The number of bytes in the file in filesystem |
+| _file.permissions_ | success | The permissions for the file in filesystem. This is formatted as 3 characters for the owner, 3 for the group, and 3 for other users. For example rw-rw-r-- |
+| _file.lastModifiedTime_ | success | The timestamp of when the file in filesystem was last modified as 'yyyy-MM-dd'T'HH:mm:ssZ' |
## ListS3
@@ -1261,27 +1328,27 @@ Retrieves a listing of objects from an S3 bucket. For each object that is listed
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Bucket**|||The S3 bucket<br/>**Supports Expression Language: true**|
-|Access Key|||AWS account access key<br/>**Supports Expression Language: true**|
-|Secret Key|||AWS account secret key<br/>**Supports Expression Language: true**|
-|Credentials File|||Path to a file containing AWS access key and secret key in properties file format. Properties used: accessKey and secretKey|
-|AWS Credentials Provider service|||The name of the AWS Credentials Provider controller service that is used to obtain AWS credentials.|
-|**Region**|us-west-2|af-south-1<br/>ap-east-1<br/>ap-northeast-1<br/>ap-northeast-2<br/>ap-northeast-3<br/>ap-south-1<br/>ap-southeast-1<br/>ap-southeast-2<br/>ca-central-1<br/>cn-north-1<br/>cn-northwest-1<br/>eu-central-1<br/>eu-north-1<br/>eu-south-1<br/>eu-west-1<br/>eu-west-2<br/>eu-west-3<br/>me-south-1<br/>sa-east-1<br/>us-east-1<br/>us-east-2<br/>us-gov-east-1<br/>us-gov-west-1<br/>us-west-1<br/>us-west-2|AWS Region|
-|**Communications Timeout**|30 sec||Sets the timeout of the communication between the AWS server and the client|
-|Endpoint Override URL|||Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints.<br/>**Supports Expression Language: true**|
-|Proxy Host|||Proxy host name or IP<br/>**Supports Expression Language: true**|
-|Proxy Port|||The port number of the proxy host<br/>**Supports Expression Language: true**|
-|Proxy Username|||Username to set when authenticating against proxy<br/>**Supports Expression Language: true**|
-|Proxy Password|||Password to set when authenticating against proxy<br/>**Supports Expression Language: true**|
-|Delimiter|||The string used to delimit directories within the bucket. Please consult the AWS documentation for the correct use of this field.|
-|Prefix|||The prefix used to filter the object list. In most cases, it should end with a forward slash ('/').|
-|**Use Versions**|false||Specifies whether to use S3 versions, if applicable. If false, only the latest version of each object will be returned.|
-|**Minimum Object Age**|0 sec||The minimum age that an S3 object must be in order to be considered; any object younger than this amount of time (according to last modification date) will be ignored.|
-|**Write Object Tags**|false||If set to 'true', the tags associated with the S3 object will be written as FlowFile attributes.|
-|**Write User Metadata**|false||If set to 'true', the user defined metadata associated with the S3 object will be added to FlowFile attributes/records.|
-|**Requester Pays**|false||If true, indicates that the requester consents to pay any charges associated with listing the S3 bucket. This sets the 'x-amz-request-payer' header to 'requester'. Note that this setting is only used if Write User Metadata is true.|
+| Name | Default Value | Allowable Values | Description [...]
+|----------------------------------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------ [...]
+| **Bucket** | | | The S3 bucket<br/>**Supports Expression L [...]
+| Access Key | | | AWS account access key<br/>**Supports Exp [...]
+| Secret Key | | | AWS account secret key<br/>**Supports Exp [...]
+| Credentials File | | | Path to a file containing AWS access key [...]
+| AWS Credentials Provider service | | | The name of the AWS Credentials Provider [...]
+| **Region** | us-west-2 | af-south-1<br/>ap-east-1<br/>ap-northeast-1<br/>ap-northeast-2<br/>ap-northeast-3<br/>ap-south-1<br/>ap-southeast-1<br/>ap-southeast-2<br/>ca-central-1<br/>cn-north-1<br/>cn-northwest-1<br/>eu-central-1<br/>eu-north-1<br/>eu-south-1<br/>eu-west-1<br/>eu-west-2<br/>eu-west-3<br/>me-south-1<br/>sa-east-1<br/>us-east-1<br/>us-east-2<br/>us-gov-east-1<br/>us-gov-west-1<br/>us-west-1<br/>us-west-2 | AWS Region [...]
+| **Communications Timeout** | 30 sec | | Sets the timeout of the communication bet [...]
+| Endpoint Override URL | | | Endpoint URL to use instead of the AWS de [...]
+| Proxy Host | | | Proxy host name or IP<br/>**Supports Expr [...]
+| Proxy Port | | | The port number of the proxy host<br/>**S [...]
+| Proxy Username | | | Username to set when authenticating again [...]
+| Proxy Password | | | Password to set when authenticating again [...]
+| Delimiter | | | The string used to delimit directories wi [...]
+| Prefix | | | The prefix used to filter the object list [...]
+| **Use Versions** | false | | Specifies whether to use S3 versions, if [...]
+| **Minimum Object Age** | 0 sec | | The minimum age that an S3 object must be [...]
+| **Write Object Tags** | false | | If set to 'true', the tags associated wit [...]
+| **Write User Metadata** | false | | If set to 'true', the user defined metada [...]
+| **Requester Pays** | false | | If true, indicates that the requester con [...]
## ListSFTP
@@ -1293,51 +1360,51 @@ Performs a listing of the files residing on an SFTP server. For each file that i
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Connection Timeout**|30 sec||Amount of time to wait before timing out while creating a connection|
-|**Data Timeout**|30 sec||When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems|
-|Entity Tracking Initial Listing Target|All Available|All Available<br>Tracking Time Window<br>|Specify how initial listing should be handled. Used by 'Tracking Entities' strategy.|
-|Entity Tracking Time Window|||Specify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed ent [...]
-|File Filter Regex|||Provides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetched|
-|**Follow symlink**|false||If true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectories|
-|Host Key File|||If supplied, the given file will be used as the Host Key; otherwise, no use host key file will be used|
-|**Hostname**|||The fully qualified hostname or IP address of the remote system<br/>**Supports Expression Language: true**|
-|Http Proxy Password|||Http Proxy Password<br/>**Supports Expression Language: true**|
-|Http Proxy Username|||Http Proxy Username<br/>**Supports Expression Language: true**|
-|**Ignore Dotted Files**|true||If true, files whose names begin with a dot (".") will be ignored|
-|**Listing Strategy**|Tracking Timestamps|Tracking Entities<br>Tracking Timestamps<br>|Specify how to determine new/updated entities. See each strategy descriptions for detail.|
-|Maximum File Age|||The maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignored|
-|Maximum File Size|||The maximum size that a file must be in order to be pulled|
-|**Minimum File Age**|0 sec||The minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored|
-|**Minimum File Size**|0 B||The minimum size that a file must be in order to be pulled|
-|Password|||Password for the user account<br/>**Supports Expression Language: true**|
-|Path Filter Regex|||When Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scanned|
-|**Port**|||The port that the remote system is listening on for file transfers<br/>**Supports Expression Language: true**|
-|Private Key Passphrase|||Password for the private key<br/>**Supports Expression Language: true**|
-|Private Key Path|||The fully qualified path to the Private Key file<br/>**Supports Expression Language: true**|
-|Proxy Host|||The fully qualified hostname or IP address of the proxy server<br/>**Supports Expression Language: true**|
-|Proxy Port|||The port of the proxy server<br/>**Supports Expression Language: true**|
-|Proxy Type|DIRECT|DIRECT<br>HTTP<br>SOCKS<br>|Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthN|
-|Remote Path|||The fully qualified filename on the remote system<br/>**Supports Expression Language: true**|
-|**Search Recursively**|false||If true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectories|
-|**Send Keep Alive On Timeout**|true||Indicates whether or not to send a single Keep Alive message when SSH socket times out|
-|**State File**|ListSFTP||Specifies the file that should be used for storing state about what data has been ingested so that upon restart MiNiFi can resume from where it left off|
-|**Strict Host Key Checking**|false||Indicates whether or not strict enforcement of hosts keys should be applied|
-|**Target System Timestamp Precision**|Auto Detect|Auto Detect<br>Milliseconds<br>Minutes<br>Seconds<br>|Specify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision.|
-|**Username**|||Username<br/>**Supports Expression Language: true**|
-### Relationships
-
-| Name | Description |
-| - | - |
-|success|All FlowFiles that are received are routed to success|
-
-
-### Relationships
-
-| Name | Description |
-| - | - |
-|success|FlowFiles are routed to success relationship|
+| Name | Default Value | Allowable Values | Description [...]
+|----------------------------------------|---------------------|-------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ [...]
+| **Connection Timeout** | 30 sec | | Amount of time to wait before timing out while creating a connection [...]
+| **Data Timeout** | 30 sec | | When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems [...]
+| Entity Tracking Initial Listing Target | All Available | All Available<br>Tracking Time Window<br> | Specify how initial listing should be handled. Used by 'Tracking Entities' strategy. [...]
+| Entity Tracking Time Window | | | Specify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile i [...]
+| File Filter Regex | | | Provides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetched [...]
+| **Follow symlink** | false | | If true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectories [...]
+| Host Key File | | | If supplied, the given file will be used as the Host Key; otherwise, no use host key file will be used [...]
+| **Hostname** | | | The fully qualified hostname or IP address of the remote system<br/>**Supports Expression Language: true** [...]
+| Http Proxy Password | | | Http Proxy Password<br/>**Supports Expression Language: true** [...]
+| Http Proxy Username | | | Http Proxy Username<br/>**Supports Expression Language: true** [...]
+| **Ignore Dotted Files** | true | | If true, files whose names begin with a dot (".") will be ignored [...]
+| **Listing Strategy** | Tracking Timestamps | Tracking Entities<br>Tracking Timestamps<br> | Specify how to determine new/updated entities. See each strategy descriptions for detail. [...]
+| Maximum File Age | | | The maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignored [...]
+| Maximum File Size | | | The maximum size that a file must be in order to be pulled [...]
+| **Minimum File Age** | 0 sec | | The minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored [...]
+| **Minimum File Size** | 0 B | | The minimum size that a file must be in order to be pulled [...]
+| Password | | | Password for the user account<br/>**Supports Expression Language: true** [...]
+| Path Filter Regex | | | When Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scanned [...]
+| **Port** | | | The port that the remote system is listening on for file transfers<br/>**Supports Expression Language: true** [...]
+| Private Key Passphrase | | | Password for the private key<br/>**Supports Expression Language: true** [...]
+| Private Key Path | | | The fully qualified path to the Private Key file<br/>**Supports Expression Language: true** [...]
+| Proxy Host | | | The fully qualified hostname or IP address of the proxy server<br/>**Supports Expression Language: true** [...]
+| Proxy Port | | | The port of the proxy server<br/>**Supports Expression Language: true** [...]
+| Proxy Type | DIRECT | DIRECT<br>HTTP<br>SOCKS<br> | Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthN [...]
+| Remote Path | | | The fully qualified filename on the remote system<br/>**Supports Expression Language: true** [...]
+| **Search Recursively** | false | | If true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectories [...]
+| **Send Keep Alive On Timeout** | true | | Indicates whether or not to send a single Keep Alive message when SSH socket times out [...]
+| **State File** | ListSFTP | | Specifies the file that should be used for storing state about what data has been ingested so that upon restart MiNiFi can resume from where it left off [...]
+| **Strict Host Key Checking** | false | | Indicates whether or not strict enforcement of hosts keys should be applied [...]
+| **Target System Timestamp Precision** | Auto Detect | Auto Detect<br>Milliseconds<br>Minutes<br>Seconds<br> | Specify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision. [...]
+| **Username** | | | Username<br/>**Supports Expression Language: true** [...]
+### Relationships
+
+| Name | Description |
+|---------|-------------------------------------------------------|
+| success | All FlowFiles that are received are routed to success |
+
+
+### Relationships
+
+| Name | Description |
+|---------|----------------------------------------------|
+| success | FlowFiles are routed to success relationship |
## LogAttribute
@@ -1348,21 +1415,21 @@ Logs attributes of flow files in the MiNiFi application log.
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Attributes to Ignore|||A comma-separated list of Attributes to ignore. If not specified, no attributes will be ignored.|
-|Attributes to Log|||A comma-separated list of Attributes to Log. If not specified, all attributes will be logged.|
-|FlowFiles To Log|1||Number of flow files to log. If set to zero all flow files will be logged. Please note that this may block other threads from running if not used judiciously.|
-|Hexencode Payload|false||If true, the FlowFile's payload will be logged in a hexencoded format|
-|Log Level||debug<br>error<br>info<br>trace<br>warn<br>|The Log Level to use when logging the Attributes|
-|Log Payload|false||If true, the FlowFile's payload will be logged, in addition to its attributes.otherwise, just the Attributes will be logged|
-|Log Prefix|||Log prefix appended to the log lines. It helps to distinguish the output of multiple LogAttribute processors.|
-|Maximum Payload Line Length|0||The logged payload will be broken into lines this long. 0 means no newlines will be added.|
+| Name | Default Value | Allowable Values | Description |
+|-----------------------------|---------------|---------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Attributes to Ignore | | | A comma-separated list of Attributes to ignore. If not specified, no attributes will be ignored. |
+| Attributes to Log | | | A comma-separated list of Attributes to Log. If not specified, all attributes will be logged. |
+| FlowFiles To Log | 1 | | Number of flow files to log. If set to zero all flow files will be logged. Please note that this may block other threads from running if not used judiciously. |
+| Hexencode Payload | false | | If true, the FlowFile's payload will be logged in a hexencoded format |
+| Log Level | | debug<br>error<br>info<br>trace<br>warn<br> | The Log Level to use when logging the Attributes |
+| Log Payload | false | | If true, the FlowFile's payload will be logged, in addition to its attributes.otherwise, just the Attributes will be logged |
+| Log Prefix | | | Log prefix appended to the log lines. It helps to distinguish the output of multiple LogAttribute processors. |
+| Maximum Payload Line Length | 0 | | The logged payload will be broken into lines this long. 0 means no newlines will be added. |
### Relationships
-| Name | Description |
-| - | - |
-|success|success operational on the flow record|
+| Name | Description |
+|---------|----------------------------------------|
+| success | success operational on the flow record |
## ManipulateArchive
@@ -1374,19 +1441,19 @@ Performs an operation which manipulates an archive without needing to split the
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|After|||For operations which result in new entries, places the new entry after the entry specified by this property.|
-|Before|||For operations which result in new entries, places the new entry before the entry specified by this property.|
-|Destination|||Destination for operations (touch, move or copy) which result in new entries.|
-|Operation|||Operation to perform on the archive (touch, remove, copy, move).|
-|Target|||An existing entry within the archive to perform the operation on.|
+| Name | Default Value | Allowable Values | Description |
+|-------------|---------------|------------------|---------------------------------------------------------------------------------------------------------------|
+| After | | | For operations which result in new entries, places the new entry after the entry specified by this property. |
+| Before | | | For operations which result in new entries, places the new entry before the entry specified by this property. |
+| Destination | | | Destination for operations (touch, move or copy) which result in new entries. |
+| Operation | | | Operation to perform on the archive (touch, remove, copy, move). |
+| Target | | | An existing entry within the archive to perform the operation on. |
### Relationships
-| Name | Description |
-| - | - |
-|failure|FlowFiles will be transferred to the failure relationship if the operation fails.|
-|success|FlowFiles will be transferred to the success relationship if the operation succeeds.|
+| Name | Description |
+|---------|--------------------------------------------------------------------------------------|
+| failure | FlowFiles will be transferred to the failure relationship if the operation fails. |
+| success | FlowFiles will be transferred to the success relationship if the operation succeeds. |
## MergeContent
@@ -1398,30 +1465,30 @@ Merges a Group of FlowFiles together based on a user-defined strategy and packag
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Attribute Strategy|Keep Only Common Attributes|Keep Only Common Attributes<Br>Keep All Unique Attributes<Br>|Determines which FlowFile attributes should be added to the bundle. If 'Keep All Unique Attributes' is selected, any attribute on any FlowFile that gets bundled will be kept unless its value conflicts with the value from another FlowFile (in which case neither, or none, of the conflicting attributes will be kept). If 'Keep Only Common Attributes' is selected, only the attributes [...]
-|Correlation Attribute Name|||Correlation Attribute Name|
-|Delimiter Strategy|Filename||Determines if Header, Footer, and Demarcator should point to files|
-|Demarcator File|||Filename specifying the demarcator to use|
-|Footer File|||Filename specifying the footer to use|
-|Header File|||Filename specifying the header to use|
-|Keep Path|false||If using the Zip or Tar Merge Format, specifies whether or not the FlowFiles' paths should be included in their entry|
-|Max Bin Age|||The maximum age of a Bin that will trigger a Bin to be complete. Expected format is <duration> <time unit>|
-|Maximum Group Size|||The maximum size for the bundle. If not specified, there is no maximum.|
-|Maximum Number of Entries|||The maximum number of files to include in a bundle. If not specified, there is no maximum.|
-|Maximum number of Bins|100||Specifies the maximum number of bins that can be held in memory at any one time|
-|Merge Format|Binary Concatenation||Merge Format|
-|Merge Strategy|Defragment||Defragment or Bin-Packing Algorithm|
-|Minimum Group Size|0||The minimum size of for the bundle|
-|Minimum Number of Entries|1||The minimum number of files to include in a bundle|
-### Relationships
-
-| Name | Description |
-| - | - |
-|failure|If the bundle cannot be created, all FlowFiles that would have been used to created the bundle will be transferred to failure|
-|merged|The FlowFile containing the merged content|
-|original|The FlowFiles that were used to create the bundle|
+| Name | Default Value | Allowable Values | Description [...]
+|----------------------------|-----------------------------|---------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| Attribute Strategy | Keep Only Common Attributes | Keep Only Common Attributes<Br>Keep All Unique Attributes<Br> | Determines which FlowFile attributes should be added to the bundle. If 'Keep All Unique Attributes' is selected, any attribute on any FlowFile that gets bundled will be kept unless its value conflicts with the value from another FlowFile (in which case neither, or none, of the conflicting attributes will be kept). If 'Keep Only Common Attributes' is selected, only [...]
+| Correlation Attribute Name | | | Correlation Attribute Name [...]
+| Delimiter Strategy | Filename | | Determines if Header, Footer, and Demarcator should point to files [...]
+| Demarcator File | | | Filename specifying the demarcator to use [...]
+| Footer File | | | Filename specifying the footer to use [...]
+| Header File | | | Filename specifying the header to use [...]
+| Keep Path | false | | If using the Zip or Tar Merge Format, specifies whether or not the FlowFiles' paths should be included in their entry [...]
+| Max Bin Age | | | The maximum age of a Bin that will trigger a Bin to be complete. Expected format is <duration> <time unit> [...]
+| Maximum Group Size | | | The maximum size for the bundle. If not specified, there is no maximum. [...]
+| Maximum Number of Entries | | | The maximum number of files to include in a bundle. If not specified, there is no maximum. [...]
+| Maximum number of Bins | 100 | | Specifies the maximum number of bins that can be held in memory at any one time [...]
+| Merge Format | Binary Concatenation | | Merge Format [...]
+| Merge Strategy | Defragment | | Defragment or Bin-Packing Algorithm [...]
+| Minimum Group Size | 0 | | The minimum size of for the bundle [...]
+| Minimum Number of Entries | 1 | | The minimum number of files to include in a bundle [...]
+### Relationships
+
+| Name | Description |
+|----------|-------------------------------------------------------------------------------------------------------------------------------|
+| failure | If the bundle cannot be created, all FlowFiles that would have been used to created the bundle will be transferred to failure |
+| merged | The FlowFile containing the merged content |
+| original | The FlowFiles that were used to create the bundle |
## MotionDetector
@@ -1433,19 +1500,19 @@ Detect motion from captured images.
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Dilate iteration**|10||For image processing, if an object is detected as 2 separate objects, increase this value|
-|**Image Encoding**|.jpg|.jpg<br>.png<br>|The encoding that should be applied to the output|
-|**Minimum Area**|100||We only consider the movement regions with area greater than this.|
-|**Path to background frame**|||If not provided then the processor will take the first input frame as background|
-|**Threshold for segmentation**|42||Pixel greater than this will be white, otherwise black.|
+| Name | Default Value | Allowable Values | Description |
+|--------------------------------|---------------|------------------|-------------------------------------------------------------------------------------------|
+| **Dilate iteration** | 10 | | For image processing, if an object is detected as 2 separate objects, increase this value |
+| **Image Encoding** | .jpg | .jpg<br>.png<br> | The encoding that should be applied to the output |
+| **Minimum Area** | 100 | | We only consider the movement regions with area greater than this. |
+| **Path to background frame** | | | If not provided then the processor will take the first input frame as background |
+| **Threshold for segmentation** | 42 | | Pixel greater than this will be white, otherwise black. |
### Relationships
-| Name | Description |
-| - | - |
-|failure|Failure to detect motion|
-|success|Successful to detect motion|
+| Name | Description |
+|---------|-----------------------------|
+| failure | Failure to detect motion |
+| success | Successful to detect motion |
## PerformanceDataMonitor
@@ -1457,13 +1524,13 @@ Windows Performance Counters provide a high-level abstraction layer with a consi
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Predefined Groups||CPU<br>IO<br>Disk<br>Network<br>Memory<br>System<br>Process|Comma separated list from the allowable values, to monitor multiple common Windows Performance counters related to these groups. (e.g. "CPU,Network")|
-|Custom PDH Counters|||Comma separated list of Windows Performance Counters to monitor. (e.g. "\\System\\Threads,\\Process(*)\\ID Process")|
-|**Output Format**|JSON|JSON<br>OpenTelemetry|The output format of the new flowfile|
-|**Output Compactness**|Pretty|Pretty<br>Compact|The output format of the new flowfile|
-|Round to decimal places||integers|The number of decimal places to round the values to (blank for no rounding)|
+| Name | Default Value | Allowable Values | Description |
+|-------------------------|---------------|-------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Predefined Groups | | CPU<br>IO<br>Disk<br>Network<br>Memory<br>System<br>Process | Comma separated list from the allowable values, to monitor multiple common Windows Performance counters related to these groups. (e.g. "CPU,Network") |
+| Custom PDH Counters | | | Comma separated list of Windows Performance Counters to monitor. (e.g. "\\System\\Threads,\\Process(*)\\ID Process") |
+| **Output Format** | JSON | JSON<br>OpenTelemetry | The output format of the new flowfile |
+| **Output Compactness** | Pretty | Pretty<br>Compact | The output format of the new flowfile |
+| Round to decimal places | | integers | The number of decimal places to round the values to (blank for no rounding) |
#### Predefined Groups
<ul>
@@ -1568,9 +1635,9 @@ In the list below, the names of required properties appear in bold. Any other pr
### Relationships
-| Name | Description |
-| - | - |
-|success|All files are routed to success|
+| Name | Description |
+|---------|---------------------------------|
+| success | All files are routed to success |
## ProcFsMonitor
@@ -1606,44 +1673,44 @@ This Processor puts the contents of a FlowFile to a Topic in Apache Kafka. The c
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Attributes to Send as Headers|||Any attribute whose name matches the regex will be added to the Kafka messages as a Header|
-|Batch Size|10||Maximum number of messages batched in one MessageSet|
-|**Client Name**|||Client Name to use when communicating with Kafka<br/>**Supports Expression Language: true**|
-|Compress Codec|none||compression codec to use for compressing message sets|
-|Debug contexts|||A comma-separated list of debug contexts to enable.Including: generic, broker, topic, metadata, feature, queue, msg, protocol, cgrp, security, fetch, interceptor, plugin, consumer, admin, eos, all|
-|Delivery Guarantee|1||Specifies the requirement for guaranteeing that a message is sent to Kafka. Valid values are 0 (do not wait for acks), -1 or all (block until message is committed by all in sync replicas) or any concrete number of nodes.<br/>**Supports Expression Language: true**|
-|Kerberos Keytab Path|||The path to the location on the local filesystem where the kerberos keytab is located. Read permission on the file is required.|
-|Kerberos Principal|||Keberos Principal|
-|Kerberos Service Name|||Kerberos Service Name|
-|**Known Brokers**|||A comma-separated list of known Kafka Brokers in the format <host>:<port><br/>**Supports Expression Language: true**|
-|Max Flow Segment Size|0 B||Maximum flow content payload segment size for the kafka record. 0 B means unlimited.|
-|Max Request Size|||Maximum Kafka protocol request message size|
-|Kafka Key|||The key to use for the message. If not specified, the UUID of the flow file is used as the message key.<br/>**Supports Expression Language: true**|
-|Message Key Field|||DEPRECATED, does not work -- use Kafka Key instead|
-|Message Timeout|30 sec||The total time sending a message could take|
-|Password|||The password for the given username when the SASL Mechanism is sasl_plaintext|
-|Queue Buffering Max Time|||Delay to wait for messages in the producer queue to accumulate before constructing message batches|
-|Queue Max Buffer Size|||Maximum total message size sum allowed on the producer queue|
-|Queue Max Message|||Maximum number of messages allowed on the producer queue|
-|Request Timeout|10 sec||The ack timeout of the producer request|
-|SASL Mechanism|GSSAPI|GSSAPI<br/>PLAIN|The SASL mechanism to use for authentication. Corresponds to Kafka's 'sasl.mechanism' property.|
-|SSL Context Service|||SSL Context Service Name|
-|Security CA|||DEPRECATED in favor of SSL Context Service. File or directory path to CA certificate(s) for verifying the broker's key|
-|Security Cert|||DEPRECATED in favor of SSL Context Service. Path to client's public key (PEM) used for authentication|
-|Security Pass Phrase|||DEPRECATED in favor of SSL Context Service. Private key passphrase|
-|Security Private Key|||DEPRECATED in favor of SSL Context Service. Path to client's private key (PEM) used for authentication|
-|**Security Protocol**|plaintext|plaintext<br/>ssl<br/>sasl_plaintext<br/>sasl_ssl|Protocol used to communicate with brokers. Corresponds to Kafka's 'security.protocol' property.|
-|Username|||The username when the SASL Mechanism is sasl_plaintext|
-|Target Batch Payload Size|512 KB||The target total payload size for a batch. 0 B means unlimited (Batch Size is still applied).|
-|**Topic Name**|||The Kafka Topic of interest<br/>**Supports Expression Language: true**|
-### Relationships
-
-| Name | Description |
-| - | - |
-|failure|Any FlowFile that cannot be sent to Kafka will be routed to this Relationship|
-|success|Any FlowFile that is successfully sent to Kafka will be routed to this Relationship|
+| Name | Default Value | Allowable Values | Description |
+|-------------------------------|---------------|---------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Attributes to Send as Headers | | | Any attribute whose name matches the regex will be added to the Kafka messages as a Header |
+| Batch Size | 10 | | Maximum number of messages batched in one MessageSet |
+| **Client Name** | | | Client Name to use when communicating with Kafka<br/>**Supports Expression Language: true** |
+| Compress Codec | none | | compression codec to use for compressing message sets |
+| Debug contexts | | | A comma-separated list of debug contexts to enable.Including: generic, broker, topic, metadata, feature, queue, msg, protocol, cgrp, security, fetch, interceptor, plugin, consumer, admin, eos, all |
+| Delivery Guarantee | 1 | | Specifies the requirement for guaranteeing that a message is sent to Kafka. Valid values are 0 (do not wait for acks), -1 or all (block until message is committed by all in sync replicas) or any concrete number of nodes.<br/>**Supports Expression Language: true** |
+| Kerberos Keytab Path | | | The path to the location on the local filesystem where the kerberos keytab is located. Read permission on the file is required. |
+| Kerberos Principal | | | Keberos Principal |
+| Kerberos Service Name | | | Kerberos Service Name |
+| **Known Brokers** | | | A comma-separated list of known Kafka Brokers in the format <host>:<port><br/>**Supports Expression Language: true** |
+| Max Flow Segment Size | 0 B | | Maximum flow content payload segment size for the kafka record. 0 B means unlimited. |
+| Max Request Size | | | Maximum Kafka protocol request message size |
+| Kafka Key | | | The key to use for the message. If not specified, the UUID of the flow file is used as the message key.<br/>**Supports Expression Language: true** |
+| Message Key Field | | | DEPRECATED, does not work -- use Kafka Key instead |
+| Message Timeout | 30 sec | | The total time sending a message could take |
+| Password | | | The password for the given username when the SASL Mechanism is sasl_plaintext |
+| Queue Buffering Max Time | | | Delay to wait for messages in the producer queue to accumulate before constructing message batches |
+| Queue Max Buffer Size | | | Maximum total message size sum allowed on the producer queue |
+| Queue Max Message | | | Maximum number of messages allowed on the producer queue |
+| Request Timeout | 10 sec | | The ack timeout of the producer request |
+| SASL Mechanism | GSSAPI | GSSAPI<br/>PLAIN | The SASL mechanism to use for authentication. Corresponds to Kafka's 'sasl.mechanism' property. |
+| SSL Context Service | | | SSL Context Service Name |
+| Security CA | | | DEPRECATED in favor of SSL Context Service. File or directory path to CA certificate(s) for verifying the broker's key |
+| Security Cert | | | DEPRECATED in favor of SSL Context Service. Path to client's public key (PEM) used for authentication |
+| Security Pass Phrase | | | DEPRECATED in favor of SSL Context Service. Private key passphrase |
+| Security Private Key | | | DEPRECATED in favor of SSL Context Service. Path to client's private key (PEM) used for authentication |
+| **Security Protocol** | plaintext | plaintext<br/>ssl<br/>sasl_plaintext<br/>sasl_ssl | Protocol used to communicate with brokers. Corresponds to Kafka's 'security.protocol' property. |
+| Username | | | The username when the SASL Mechanism is sasl_plaintext |
+| Target Batch Payload Size | 512 KB | | The target total payload size for a batch. 0 B means unlimited (Batch Size is still applied). |
+| **Topic Name** | | | The Kafka Topic of interest<br/>**Supports Expression Language: true** |
+### Relationships
+
+| Name | Description |
+|---------|-------------------------------------------------------------------------------------|
+| failure | Any FlowFile that cannot be sent to Kafka will be routed to this Relationship |
+| success | Any FlowFile that is successfully sent to Kafka will be routed to this Relationship |
## PublishMQTT
@@ -1654,25 +1721,25 @@ PublishMQTT serializes FlowFile content as an MQTT payload, sending the message
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Broker URI|||The URI to use to connect to the MQTT broker|
-|Client ID|||MQTT client ID to use|
-|Connection Timeout|30 sec||Maximum time interval the client will wait for the network connection to the MQTT server|
-|Keep Alive Interval|60 sec||Defines the maximum time interval between messages sent or received|
-|Max Flow Segment Size|||Maximum flow content payload segment size for the MQTT record|
-|Password|||Password to use when connecting to the broker|
-|Quality of Service|MQTT_QOS_0||The Quality of Service(QoS) to send the message with. Accepts three values '0', '1' and '2'|
-|Retain|false||Retain MQTT published record in broker|
-|Session state|true||Whether to start afresh or resume previous flows. See the allowable value descriptions for more details|
-|Topic|||The topic to publish the message to|
-|Username|||Username to use when connecting to the broker|
-### Relationships
-
-| Name | Description |
-| - | - |
-|failure|FlowFiles that failed to send to the destination are transferred to this relationship|
-|success|FlowFiles that are sent successfully to the destination are transferred to this relationship|
+| Name | Default Value | Allowable Values | Description |
+|-----------------------|---------------|------------------|---------------------------------------------------------------------------------------------------------|
+| Broker URI | | | The URI to use to connect to the MQTT broker |
+| Client ID | | | MQTT client ID to use |
+| Connection Timeout | 30 sec | | Maximum time interval the client will wait for the network connection to the MQTT server |
+| Keep Alive Interval | 60 sec | | Defines the maximum time interval between messages sent or received |
+| Max Flow Segment Size | | | Maximum flow content payload segment size for the MQTT record |
+| Password | | | Password to use when connecting to the broker |
+| Quality of Service | MQTT_QOS_0 | | The Quality of Service(QoS) to send the message with. Accepts three values '0', '1' and '2' |
+| Retain | false | | Retain MQTT published record in broker |
+| Session state | true | | Whether to start afresh or resume previous flows. See the allowable value descriptions for more details |
+| Topic | | | The topic to publish the message to |
+| Username | | | Username to use when connecting to the broker |
+### Relationships
+
+| Name | Description |
+|---------|----------------------------------------------------------------------------------------------|
+| failure | FlowFiles that failed to send to the destination are transferred to this relationship |
+| success | FlowFiles that are sent successfully to the destination are transferred to this relationship |
## PutAzureBlobStorage
@@ -1684,25 +1751,25 @@ Puts content into an Azure Storage Blob
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Azure Storage Credentials Service|||Name of the Azure Storage Credentials Service used to retrieve the connection string from.|
-|**Blob**|||The filename of the blob. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true**|
-|Common Storage Account Endpoint Suffix|||Storage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).<br/>**Supports Expression Language: true**|
-|Connection String|||Connection string used to connect to Azure Storage service. This overrides all other set credential properties if Managed Identity is not used.<br/>**Supports Expression Language: true**|
-|**Container Name**|||Name of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.<br/>**Supports Expression Language: true**|
-|**Create Container**|false||Specifies whether to check if the container exists and to automatically create it if it does not. Permission to list containers is required. If false, this check is not made, but the Put operation will fail if the container does not exist.|
-|SAS Token|||Shared Access Signature token. Specify either SAS Token (recommended) or Storage Account Key together with Storage Account Name if Managed Identity is not used.<br/>**Supports Expression Language: true**|
-|Storage Account Key|||The storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token instead for fine-grained control with policies.<br/>**Supports Expression Language: true**|
-|Storage Account Name|||The storage account name.<br/>**Supports Expression Language: true**|
-|**Use Managed Identity Credentials**|false||If true Managed Identity credentials will be used together with the Storage Account Name for authentication.|
+| Name | Default Value | Allowable Values | Description |
+|----------------------------------------|---------------|------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Azure Storage Credentials Service | | | Name of the Azure Storage Credentials Service used to retrieve the connection string from. |
+| **Blob** | | | The filename of the blob. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true** |
+| Common Storage Account Endpoint Suffix | | | Storage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions).<br/>**Supports Expression Language: true** |
+| Connection String | | | Connection string used to connect to Azure Storage service. This overrides all other set credential properties if Managed Identity is not used.<br/>**Supports Expression Language: true** |
+| **Container Name** | | | Name of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.<br/>**Supports Expression Language: true** |
+| **Create Container** | false | | Specifies whether to check if the container exists and to automatically create it if it does not. Permission to list containers is required. If false, this check is not made, but the Put operation will fail if the container does not exist. |
+| SAS Token | | | Shared Access Signature token. Specify either SAS Token (recommended) or Storage Account Key together with Storage Account Name if Managed Identity is not used.<br/>**Supports Expression Language: true** |
+| Storage Account Key | | | The storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token instead for fine-grained control with policies.<br/>**Supports Expression Language: true** |
+| Storage Account Name | | | The storage account name.<br/>**Supports Expression Language: true** |
+| **Use Managed Identity Credentials** | false | | If true Managed Identity credentials will be used together with the Storage Account Name for authentication. |
### Relationships
-| Name | Description |
-| - | - |
-|failure|Unsuccessful operations will be transferred to the failure relationship|
-|success|All successfully processed FlowFiles are routed to this relationship|
+| Name | Description |
+|---------|-------------------------------------------------------------------------|
+| failure | Unsuccessful operations will be transferred to the failure relationship |
+| success | All successfully processed FlowFiles are routed to this relationship |
## PutAzureDataLakeStorage
@@ -1714,20 +1781,20 @@ Puts content into an Azure Data Lake Storage Gen 2
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Azure Storage Credentials Service**|||Name of the Azure Storage Credentials Service used to retrieve the connection string from.|
-|**Conflict Resolution Strategy**|fail|fail<br/>replace<br/>ignore|Indicates what should happen when a file with the same name already exists in the output directory.|
-|File Name|||The filename in Azure Storage. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true**|
-|**Filesystem Name**|||Name of the Azure Storage File System. It is assumed to be already existing.<br/>**Supports Expression Language: true**|
-|Directory Name|||Name of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. If left empty it designates the root directory. The directory will be created if not already existing.<br/>**Supports Expression Language: true**|
+| Name | Default Value | Allowable Values | Description |
+|---------------------------------------|---------------|-----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Azure Storage Credentials Service** | | | Name of the Azure Storage Credentials Service used to retrieve the connection string from. |
+| **Conflict Resolution Strategy** | fail | fail<br/>replace<br/>ignore | Indicates what should happen when a file with the same name already exists in the output directory. |
+| File Name | | | The filename in Azure Storage. If left empty the filename attribute will be used by default.<br/>**Supports Expression Language: true** |
+| **Filesystem Name** | | | Name of the Azure Storage File System. It is assumed to be already existing.<br/>**Supports Expression Language: true** |
+| Directory Name | | | Name of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. If left empty it designates the root directory. The directory will be created if not already existing.<br/>**Supports Expression Language: true** |
### Relationships
-| Name | Description |
-| - | - |
-|failure|Files that could not be written to Azure storage for some reason are transferred to this relationship|
-|success|Files that have been successfully written to Azure storage are transferred to this relationship|
+| Name | Description |
+|---------|-------------------------------------------------------------------------------------------------------|
+| failure | Files that could not be written to Azure storage for some reason are transferred to this relationship |
+| success | Files that have been successfully written to Azure storage are transferred to this relationship |
## PutGCSObject
@@ -1798,20 +1865,20 @@ Writes the contents of a FlowFile to the local file system
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Conflict Resolution Strategy|fail|fail<br>ignore<br>replace<br>|Indicates what should happen when a file with the same name already exists in the output directory|
-|**Create Missing Directories**|true||If true, then missing destination directories will be created. If false, flowfiles are penalized and sent to failure.|
-|Directory|.||The output directory to which to put files<br/>**Supports Expression Language: true**|
-|Maximum File Count|-1||Specifies the maximum number of files that can exist in the output directory|
-|Permissions|||Sets the permissions on the output file to the value of this attribute. Must be an octal number (e.g. 644 or 0755). Not supported on Windows systems.|
-|Directory Permissions|||Sets the permissions on the directories being created if 'Create Missing Directories' property is set. Must be an octal number (e.g. 644 or 0755). Not supported on Windows systems.|
+| Name | Default Value | Allowable Values | Description |
+|--------------------------------|---------------|-------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Conflict Resolution Strategy | fail | fail<br>ignore<br>replace<br> | Indicates what should happen when a file with the same name already exists in the output directory |
+| **Create Missing Directories** | true | | If true, then missing destination directories will be created. If false, flowfiles are penalized and sent to failure. |
+| Directory | . | | The output directory to which to put files<br/>**Supports Expression Language: true** |
+| Maximum File Count | -1 | | Specifies the maximum number of files that can exist in the output directory |
+| Permissions | | | Sets the permissions on the output file to the value of this attribute. Must be an octal number (e.g. 644 or 0755). Not supported on Windows systems. |
+| Directory Permissions | | | Sets the permissions on the directories being created if 'Create Missing Directories' property is set. Must be an octal number (e.g. 644 or 0755). Not supported on Windows systems. |
### Relationships
-| Name | Description |
-| - | - |
-|failure|Failed files (conflict, write failure, etc.) are transferred to failure|
-|success|All files are routed to success|
+| Name | Description |
+|---------|-------------------------------------------------------------------------|
+| failure | Failed files (conflict, write failure, etc.) are transferred to failure |
+| success | All files are routed to success |
## PutOPCProcessor
@@ -1823,29 +1890,29 @@ Creates/updates OPC nodes
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Application URI|||Application URI of the client in the format 'urn:unconfigured:application'. Mandatory, if using Secure Channel and must match the URI included in the certificate's Subject Alternative Names.|
-|Certificate path|||Path to the DER-encoded cert file|
-|Key path|||Path to the DER-encoded key file|
-|**OPC server endpoint**|||Specifies the address, port and relative path of an OPC endpoint|
-|**Parent node ID**|||Specifies the ID of the root node to traverse|
-|**Parent node ID type**||Int<br>Path<br>String<br>|Specifies the type of the provided node ID|
-|Parent node namespace index|0||The index of the namespace. Used only if node ID type is not path.|
-|Password|||Password to log in with.|
-|Target node ID|||ID of target node.<br/>**Supports Expression Language: true**|
-|Target node ID type|||ID type of target node. Allowed values are: Int, String.<br/>**Supports Expression Language: true**|
-|Target node browse name|||Browse name of target node. Only used when new node is created.<br/>**Supports Expression Language: true**|
-|Target node namespace index|||The index of the namespace. Used only if node ID type is not path.<br/>**Supports Expression Language: true**|
-|Trusted server certificate path|||Path to the DER-encoded trusted server certificate|
-|Username|||Username to log in with.|
-|**Value type**||Boolean<br>Double<br>Float<br>Int32<br>Int64<br>String<br>UInt32<br>UInt64<br>|Set the OPC value type of the created nodes|
-### Relationships
-
-| Name | Description |
-| - | - |
-|failure|Failed to put OPC-UA node|
-|success|Successfully put OPC-UA node|
+| Name | Default Value | Allowable Values | Description |
+|---------------------------------|---------------|--------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Application URI | | | Application URI of the client in the format 'urn:unconfigured:application'. Mandatory, if using Secure Channel and must match the URI included in the certificate's Subject Alternative Names. |
+| Certificate path | | | Path to the DER-encoded cert file |
+| Key path | | | Path to the DER-encoded key file |
+| **OPC server endpoint** | | | Specifies the address, port and relative path of an OPC endpoint |
+| **Parent node ID** | | | Specifies the ID of the root node to traverse |
+| **Parent node ID type** | | Int<br>Path<br>String<br> | Specifies the type of the provided node ID |
+| Parent node namespace index | 0 | | The index of the namespace. Used only if node ID type is not path. |
+| Password | | | Password to log in with. |
+| Target node ID | | | ID of target node.<br/>**Supports Expression Language: true** |
+| Target node ID type | | | ID type of target node. Allowed values are: Int, String.<br/>**Supports Expression Language: true** |
+| Target node browse name | | | Browse name of target node. Only used when new node is created.<br/>**Supports Expression Language: true** |
+| Target node namespace index | | | The index of the namespace. Used only if node ID type is not path.<br/>**Supports Expression Language: true** |
+| Trusted server certificate path | | | Path to the DER-encoded trusted server certificate |
+| Username | | | Username to log in with. |
+| **Value type** | | Boolean<br>Double<br>Float<br>Int32<br>Int64<br>String<br>UInt32<br>UInt64<br> | Set the OPC value type of the created nodes |
+### Relationships
+
+| Name | Description |
+|---------|------------------------------|
+| failure | Failed to put OPC-UA node |
+| success | Successfully put OPC-UA node |
## PutS3Object
@@ -1857,35 +1924,35 @@ Puts FlowFiles to an Amazon S3 Bucket. The upload uses either the PutS3Object me
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | <div style="width:7em">Default Value</div> | <div style="width:8em">Allowable Values</div> | Description |
-| - | - | - | - |
-|Object Key|||The key of the S3 object. If none is given the filename attribute will be used by default.<br/>**Supports Expression Language: true**|
-|**Bucket**|||The S3 bucket<br/>**Supports Expression Language: true**|
-|Content Type|||Sets the Content-Type HTTP header indicating the type of content stored in the associated object. The value of this header is a standard MIME type. If no content type is provided the default content type "application/octet-stream" will be used.<br/>**Supports Expression Language: true**|
-|**Use Default Credentials**|false||If true, uses the Default Credential chain to obtain AWS credentials, including EC2 instance profiles or roles, environment variables, default user credentials, etc.|
-|Access Key|||AWS account access key<br/>**Supports Expression Language: true**|
-|Secret Key|||AWS account secret key<br/>**Supports Expression Language: true**|
-|Credentials File|||Path to a file containing AWS access key and secret key in properties file format. Properties used: accessKey and secretKey|
-|AWS Credentials Provider service|||The name of the AWS Credentials Provider controller service that is used to obtain AWS credentials.|
-|**Storage Class**|Standard|Standard<br/>ReducedRedundancy<br/>StandardIA<br/>OnezoneIA<br/>IntelligentTiering<br/>Glacier<br/>DeepArchive|AWS S3 Storage Class|
-|**Region**|us-west-2|af-south-1<br/>ap-east-1<br/>ap-northeast-1<br/>ap-northeast-2<br/>ap-northeast-3<br/>ap-south-1<br/>ap-southeast-1<br/>ap-southeast-2<br/>ca-central-1<br/>cn-north-1<br/>cn-northwest-1<br/>eu-central-1<br/>eu-north-1<br/>eu-south-1<br/>eu-west-1<br/>eu-west-2<br/>eu-west-3<br/>me-south-1<br/>sa-east-1<br/>us-east-1<br/>us-east-2<br/>us-gov-east-1<br/>us-gov-west-1<br/>us-west-1<br/>us-west-2|AWS Region|
-|**Communications Timeout**|30 sec||Sets the timeout of the communication between the AWS server and the client|
-|FullControl User List|||A comma-separated list of Amazon User ID's or E-mail addresses that specifies who should have Full Control for an object.<br/>**Supports Expression Language: true**|
-|Read Permission User List|||A comma-separated list of Amazon User ID's or E-mail addresses that specifies who should have Read Access for an object.<br/>**Supports Expression Language: true**|
-|Read ACL User List|||A comma-separated list of Amazon User ID's or E-mail addresses that specifies who should have permissions to read the Access Control List for an object.<br/>**Supports Expression Language: true**|
-|Write ACL User List|||A comma-separated list of Amazon User ID's or E-mail addresses that specifies who should have permissions to change the Access Control List for an object.<br/>**Supports Expression Language: true**||Amazon Canned ACL for an object. Allowed values: BucketOwnerFullControl, BucketOwnerRead, AuthenticatedRead, PublicReadWrite, PublicRead, Private, AwsExecRead; will be ignored if any other ACL/permission property is specified.<br/>**Supports Expression Language: true**|
-|Endpoint Override URL|||Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints.<br/>**Supports Expression Language: true**|
-|**Server Side Encryption**|None|None<br/>AES256<br/>aws_kms|Specifies the algorithm used for server side encryption.|
-|Proxy Host|||Proxy host name or IP<br/>**Supports Expression Language: true**|
-|Proxy Port|||The port number of the proxy host<br/>**Supports Expression Language: true**|
-|Proxy Username|||Username to set when authenticating against proxy<br/>**Supports Expression Language: true**|
-|Proxy Password|||Password to set when authenticating against proxy<br/>**Supports Expression Language: true**|
+| Name | <div style="width:7em">Default Value</div> | <div style="width:8em">Allowable Values</div> | Description [...]
+|----------------------------------|--------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------- [...]
+| Object Key | | | The key of t [...]
+| **Bucket** | | | The S3 bucke [...]
+| Content Type | | | Sets the Con [...]
+| **Use Default Credentials** | false | | If true, use [...]
+| Access Key | | | AWS account [...]
+| Secret Key | | | AWS account [...]
+| Credentials File | | | Path to a fi [...]
+| AWS Credentials Provider service | | | The name of [...]
+| **Storage Class** | Standard | Standard<br/>ReducedRedundancy<br/>StandardIA<br/>OnezoneIA<br/>IntelligentTiering<br/>Glacier<br/>DeepArchive | AWS S3 Stora [...]
+| **Region** | us-west-2 | af-south-1<br/>ap-east-1<br/>ap-northeast-1<br/>ap-northeast-2<br/>ap-northeast-3<br/>ap-south-1<br/>ap-southeast-1<br/>ap-southeast-2<br/>ca-central-1<br/>cn-north-1<br/>cn-northwest-1<br/>eu-central-1<br/>eu-north-1<br/>eu-south-1<br/>eu-west-1<br/>eu-west-2<br/>eu-west-3<br/>me-south-1<br/>sa-east-1<br/>us-east-1<br/>us-east-2<br/>us-gov-east-1<br/>us-gov-west-1<br/>us-west-1<br/>us-west-2 | AWS Region [...]
+| **Communications Timeout** | 30 sec | | Sets the tim [...]
+| FullControl User List | | | A comma-sepa [...]
+| Read Permission User List | | | A comma-sepa [...]
+| Read ACL User List | | | A comma-sepa [...]
+| Write ACL User List | | | A comma-sepa [...]
+| Endpoint Override URL | | | Endpoint URL [...]
+| **Server Side Encryption** | None | None<br/>AES256<br/>aws_kms | Specifies th [...]
+| Proxy Host | | | Proxy host n [...]
+| Proxy Port | | | The port num [...]
+| Proxy Username | | | Username to [...]
+| Proxy Password | | | Password to [...]
### Relationships
-| Name | Description |
-| - | - |
-|failure|FlowFiles are routed to failure relationship|
-|success|FlowFiles are routed to success relationship|
+| Name | Description |
+|---------|----------------------------------------------|
+| failure | FlowFiles are routed to failure relationship |
+| success | FlowFiles are routed to success relationship |
## PutSFTP
@@ -1897,44 +1964,44 @@ Sends FlowFiles to an SFTP Server
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Batch Size**|500||The maximum number of FlowFiles to send in a single connection|
-|**Conflict Resolution**|NONE|FAIL<br>IGNORE<br>NONE<br>REJECT<br>RENAME<br>REPLACE<br>|Determines how to handle the problem of filename collisions|
-|**Connection Timeout**|30 sec||Amount of time to wait before timing out while creating a connection|
-|**Create Directory**|false||Specifies whether or not the remote directory should be created if it does not exist.|
-|**Data Timeout**|30 sec||When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems|
-|Disable Directory Listing|false||If set to 'true', directory listing is not performed prior to create missing directories. By default, this processor executes a directory listing command to see target directory existence before creating missing directories. However, there are situations that you might need to disable the directory listing such as the following. Directory listing might fail with some permission setups (e.g. chmod 100) on a directory. Also, if any other SFTP client create [...]
-|Dot Rename|true||If true, then the filename of the sent file is prepended with a "." and then renamed back to the original once the file is completely sent. Otherwise, there is no rename. This property is ignored if the Temporary Filename property is set.|
-|Host Key File|||If supplied, the given file will be used as the Host Key; otherwise, no use host key file will be used|
-|**Hostname**|||The fully qualified hostname or IP address of the remote system<br/>**Supports Expression Language: true**|
-|Http Proxy Password|||Http Proxy Password<br/>**Supports Expression Language: true**|
-|Http Proxy Username|||Http Proxy Username<br/>**Supports Expression Language: true**|
-|Last Modified Time|||The lastModifiedTime to assign to the file after transferring it. If not set, the lastModifiedTime will not be changed. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}. If the value is invalid, the processor will not be invalid but will fail to change lastModifiedTime of the file.<br/>**Supports Expression Language: true**|
-|Password|||Password for the user account<br/>**Supports Expression Language: true**|
-|Permissions|||The permissions to assign to the file after transferring it. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). If not set, the permissions will not be changed. You may also use expression language such as ${file.permissions}. If the value is invalid, the processor will not be invalid but will fail to change permissions of the file.<br/>**Supports Expression Language: true**|
-|**Port**|||The port that the remote system is listening on for file transfers<br/>**Supports Expression Language: true**|
-|Private Key Passphrase|||Password for the private key<br/>**Supports Expression Language: true**|
-|Private Key Path|||The fully qualified path to the Private Key file<br/>**Supports Expression Language: true**|
-|Proxy Host|||The fully qualified hostname or IP address of the proxy server<br/>**Supports Expression Language: true**|
-|Proxy Port|||The port of the proxy server<br/>**Supports Expression Language: true**|
-|Proxy Type|DIRECT|DIRECT<br>HTTP<br>SOCKS<br>|Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthN|
-|Reject Zero-Byte Files|true||Determines whether or not Zero-byte files should be rejected without attempting to transfer|
-|Remote Group|||Integer value representing the Group ID to set on the file after transferring it. If not set, the group will not be set. You may also use expression language such as ${file.group}. If the value is invalid, the processor will not be invalid but will fail to change the group of the file.<br/>**Supports Expression Language: true**|
-|Remote Owner|||Integer value representing the User ID to set on the file after transferring it. If not set, the owner will not be set. You may also use expression language such as ${file.owner}. If the value is invalid, the processor will not be invalid but will fail to change the owner of the file.<br/>**Supports Expression Language: true**|
-|Remote Path|||The path on the remote system from which to pull or push files<br/>**Supports Expression Language: true**|
-|**Send Keep Alive On Timeout**|true||Indicates whether or not to send a single Keep Alive message when SSH socket times out|
-|**Strict Host Key Checking**|false||Indicates whether or not strict enforcement of hosts keys should be applied|
-|Temporary Filename|||If set, the filename of the sent file will be equal to the value specified during the transfer and after successful completion will be renamed to the original filename. If this value is set, the Dot Rename property is ignored.<br/>**Supports Expression Language: true**|
-|**Use Compression**|false||Indicates whether or not ZLIB compression should be used when transferring files|
-|**Username**|||Username<br/>**Supports Expression Language: true**|
-### Relationships
-
-| Name | Description |
-| - | - |
-|failure|FlowFiles that failed to send to the remote system; failure is usually looped back to this processor|
-|reject|FlowFiles that were rejected by the destination system|
-|success|FlowFiles that are successfully sent will be routed to success|
+| Name | Default Value | Allowable Values | Description [...]
+|--------------------------------|---------------|-----------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| **Batch Size** | 500 | | The maximum number of FlowFiles to send in a single connection [...]
+| **Conflict Resolution** | NONE | FAIL<br>IGNORE<br>NONE<br>REJECT<br>RENAME<br>REPLACE<br> | Determines how to handle the problem of filename collisions [...]
+| **Connection Timeout** | 30 sec | | Amount of time to wait before timing out while creating a connection [...]
+| **Create Directory** | false | | Specifies whether or not the remote directory should be created if it does not exist. [...]
+| **Data Timeout** | 30 sec | | When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems [...]
+| Disable Directory Listing | false | | If set to 'true', directory listing is not performed prior to create missing directories. By default, this processor executes a directory listing command to see target directory existence before creating missing directories. However, there are situations that you might need to disable the directory listing such as the following. Directory listing might fail with some permission [...]
+| Dot Rename | true | | If true, then the filename of the sent file is prepended with a "." and then renamed back to the original once the file is completely sent. Otherwise, there is no rename. This property is ignored if the Temporary Filename property is set. [...]
+| Host Key File | | | If supplied, the given file will be used as the Host Key; otherwise, no use host key file will be used [...]
+| **Hostname** | | | The fully qualified hostname or IP address of the remote system<br/>**Supports Expression Language: true** [...]
+| Http Proxy Password | | | Http Proxy Password<br/>**Supports Expression Language: true** [...]
+| Http Proxy Username | | | Http Proxy Username<br/>**Supports Expression Language: true** [...]
+| Last Modified Time | | | The lastModifiedTime to assign to the file after transferring it. If not set, the lastModifiedTime will not be changed. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}. If the value is invalid, the processor will not be invalid but will fail to change lastModifiedTime of the file.<br/>**Supports Expression Language: tr [...]
+| Password | | | Password for the user account<br/>**Supports Expression Language: true** [...]
+| Permissions | | | The permissions to assign to the file after transferring it. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). If not set, the permissions will not be changed. You may also use expression language such as ${file.permissions}. If the value is invalid, the processor will not be invalid but will fail to chan [...]
+| **Port** | | | The port that the remote system is listening on for file transfers<br/>**Supports Expression Language: true** [...]
+| Private Key Passphrase | | | Password for the private key<br/>**Supports Expression Language: true** [...]
+| Private Key Path | | | The fully qualified path to the Private Key file<br/>**Supports Expression Language: true** [...]
+| Proxy Host | | | The fully qualified hostname or IP address of the proxy server<br/>**Supports Expression Language: true** [...]
+| Proxy Port | | | The port of the proxy server<br/>**Supports Expression Language: true** [...]
+| Proxy Type | DIRECT | DIRECT<br>HTTP<br>SOCKS<br> | Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthN [...]
+| Reject Zero-Byte Files | true | | Determines whether or not Zero-byte files should be rejected without attempting to transfer [...]
+| Remote Group | | | Integer value representing the Group ID to set on the file after transferring it. If not set, the group will not be set. You may also use expression language such as ${file.group}. If the value is invalid, the processor will not be invalid but will fail to change the group of the file.<br/>**Supports Expression Language: true** [...]
+| Remote Owner | | | Integer value representing the User ID to set on the file after transferring it. If not set, the owner will not be set. You may also use expression language such as ${file.owner}. If the value is invalid, the processor will not be invalid but will fail to change the owner of the file.<br/>**Supports Expression Language: true** [...]
+| Remote Path | | | The path on the remote system from which to pull or push files<br/>**Supports Expression Language: true** [...]
+| **Send Keep Alive On Timeout** | true | | Indicates whether or not to send a single Keep Alive message when SSH socket times out [...]
+| **Strict Host Key Checking** | false | | Indicates whether or not strict enforcement of hosts keys should be applied [...]
+| Temporary Filename | | | If set, the filename of the sent file will be equal to the value specified during the transfer and after successful completion will be renamed to the original filename. If this value is set, the Dot Rename property is ignored.<br/>**Supports Expression Language: true** [...]
+| **Use Compression** | false | | Indicates whether or not ZLIB compression should be used when transferring files [...]
+| **Username** | | | Username<br/>**Supports Expression Language: true** [...]
+### Relationships
+
+| Name | Description |
+|---------|------------------------------------------------------------------------------------------------------|
+| failure | FlowFiles that failed to send to the remote system; failure is usually looped back to this processor |
+| reject | FlowFiles that were rejected by the destination system |
+| success | FlowFiles that are successfully sent will be routed to success |
## PutSplunkHTTP
@@ -1964,25 +2031,25 @@ also filled, based on the Splunk response code.
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| -------------------------- | ------------- | --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **Hostname** | | | The ip address or hostname of the Splunk server. |
-| **Port** | 8088 | | The HTTP Event Collector HTTP Port Number. |
-| **Token** | | Splunk <guid> | HTTP Event Collector token starting with the string Splunk. For example `Splunk 1234578-abcd-1234-abcd-1234abcd` |
-| **Splunk Request Channel** | | <guid> | Identifier of the used request channel. |
-| SSL Context Service | | | The SSL Context Service used to provide client certificate information for TLS/SSL (https) connections. |
-| Source | | | Basic field describing the source of the event. If unspecified, the event will use the default defined in splunk. |
-| SourceType | | | Basic field describing the source type of the event. If unspecified, the event will use the default defined in splunk. |
-| Host | | | Basic field describing the host of the event. If unspecified, the event will use the default defined in splunk. |
-| Index | | | Identifies the index where to send the event. If unspecified, the event will use the default defined in splunk. |
-| Content Type | | | The media type of the event sent to Splunk. If not set, "mime.type" flow file attribute will be used. In case of neither of them is specified, this information will not be sent to the server. |
+| Name | Default Value | Allowable Values | Description |
+|----------------------------|---------------|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Hostname** | | | The ip address or hostname of the Splunk server. |
+| **Port** | 8088 | | The HTTP Event Collector HTTP Port Number. |
+| **Token** | | Splunk <guid> | HTTP Event Collector token starting with the string Splunk. For example `Splunk 1234578-abcd-1234-abcd-1234abcd` |
+| **Splunk Request Channel** | | <guid> | Identifier of the used request channel. |
+| SSL Context Service | | | The SSL Context Service used to provide client certificate information for TLS/SSL (https) connections. |
+| Source | | | Basic field describing the source of the event. If unspecified, the event will use the default defined in splunk. |
+| SourceType | | | Basic field describing the source type of the event. If unspecified, the event will use the default defined in splunk. |
+| Host | | | Basic field describing the host of the event. If unspecified, the event will use the default defined in splunk. |
+| Index | | | Identifies the index where to send the event. If unspecified, the event will use the default defined in splunk. |
+| Content Type | | | The media type of the event sent to Splunk. If not set, "mime.type" flow file attribute will be used. In case of neither of them is specified, this information will not be sent to the server. |
### Relationships
-| Name | Description |
-| -------- | -------------------------------------------------------------------------------------- |
-| success | FlowFiles that are sent successfully to the destination are sent to this relationship. |
-| failure | FlowFiles that failed to be sent to the destination are sent to this relationship. |
+| Name | Description |
+|---------|----------------------------------------------------------------------------------------|
+| success | FlowFiles that are sent successfully to the destination are sent to this relationship. |
+| failure | FlowFiles that failed to be sent to the destination are sent to this relationship. |
## PutSQL
@@ -1994,15 +2061,15 @@ Executes a SQL UPDATE or INSERT command. The content of an incoming FlowFile is
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**DB Controller Service**|||Database Controller Service.|
-|SQL Statement|||The SQL statement to execute. The statement can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming flowfiles. If this property is empty, the content of the incoming flow file is expected to contain a valid SQL statement, to be issued by the processor to the database.|
+| Name | Default Value | Allowable Values | Description |
+|---------------------------|---------------|------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **DB Controller Service** | | | Database Controller Service. |
+| SQL Statement | | | The SQL statement to execute. The statement can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming flowfiles. If this property is empty, the content of the incoming flow file is expected to contain a valid SQL statement, to be issued by the processor to the database. |
### Relationships
-| Name | Description |
-| - | - |
-|success|After a successful SQL update operation, the incoming FlowFile sent here|
+| Name | Description |
+|---------|--------------------------------------------------------------------------|
+| success | After a successful SQL update operation, the incoming FlowFile sent here |
## PutUDP
@@ -2036,25 +2103,25 @@ Fetches all rows of a table, whose values in the specified Maximum-value Columns
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Columns to Return|||A comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.|
-|**DB Controller Service**|||Database Controller Service.|
-|**Max Rows Per Flow File**|0||The maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile.|
-|Maximum-value Columns|||A comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables.|
-|**Output Format**|JSON-Pretty|JSON<br/>JSON-Pretty|Set the output format type.|
-|**Table Name**|||The name of the database table to be queried.|
-|Where Clause|||A custom clause to be added in the WHERE condition when building SQL queries.|
-### Relationships
-
-| Name | Description |
-| - | - |
-|success|Successfully created FlowFile from SQL query result set.|
+| Name | Default Value | Allowable Values | Description [...]
+|----------------------------|---------------|----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| Columns to Return | | | A comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly. [...]
+| **DB Controller Service** | | | Database Controller Service. [...]
+| **Max Rows Per Flow File** | 0 | | The maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile. [...]
+| Maximum-value Columns | | | A comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used fo [...]
+| **Output Format** | JSON-Pretty | JSON<br/>JSON-Pretty | Set the output format type. [...]
+| **Table Name** | | | The name of the database table to be queried. [...]
+| Where Clause | | | A custom clause to be added in the WHERE condition when building SQL queries. [...]
+### Relationships
+
+| Name | Description |
+|---------|----------------------------------------------------------|
+| success | Successfully created FlowFile from SQL query result set. |
### Dynamic Properties:
-| Name | Value | Description |
-| - | - | - |
-|initial.maxvalue.<max_value_column>|Initial maximum value for the specified column|Specifies an initial max value for max value column(s). Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified).<br/>**Supports Expression Language: true**|
+| Name | Value | Description |
+|-------------------------------------|------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| initial.maxvalue.<max_value_column> | Initial maximum value for the specified column | Specifies an initial max value for max value column(s). Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified).<br/>**Supports Expression Language: true** |
## QuerySplunkIndexingStatus
@@ -2091,23 +2158,23 @@ batch but the processor might execute the query with smaller number of undetermi
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| -------------------------- | ------------- | --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **Maximum Waiting Time** | 1 hour | <duration> <time unit> | The maximum time the processor tries to acquire acknowledgement confirmation for an index, from the point of registration.<br> After the given amount of time, the processor considers the index as not acknowledged and transfers the FlowFile to the _unacknowledged_ relationship. |
-| **Maximum Query Size** | 1000 | integers | The maximum number of acknowledgement identifiers the outgoing query contains in one batch. It is recommended not to set it too low in order to reduce network communication. |
-| **Hostname** | | | The ip address or hostname of the Splunk server. |
-| **Port** | 8088 | | The HTTP Event Collector HTTP Port Number. |
-| **Token** | | Splunk <guid> | HTTP Event Collector token starting with the string Splunk. For example `Splunk 1234578-abcd-1234-abcd-1234abcd` |
-| **Splunk Request Channel** | | <guid> | Identifier of the used request channel. |
-| SSL Context Service | | | The SSL Context Service used to provide client certificate information for TLS/SSL (https) connections. |
+| Name | Default Value | Allowable Values | Description |
+|----------------------------|---------------|------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Maximum Waiting Time** | 1 hour | <duration> <time unit> | The maximum time the processor tries to acquire acknowledgement confirmation for an index, from the point of registration.<br> After the given amount of time, the processor considers the index as not acknowledged and transfers the FlowFile to the _unacknowledged_ relationship. |
+| **Maximum Query Size** | 1000 | integers | The maximum number of acknowledgement identifiers the outgoing query contains in one batch. It is recommended not to set it too low in order to reduce network communication. |
+| **Hostname** | | | The ip address or hostname of the Splunk server. |
+| **Port** | 8088 | | The HTTP Event Collector HTTP Port Number. |
+| **Token** | | Splunk <guid> | HTTP Event Collector token starting with the string Splunk. For example `Splunk 1234578-abcd-1234-abcd-1234abcd` |
+| **Splunk Request Channel** | | <guid> | Identifier of the used request channel. |
+| SSL Context Service | | | The SSL Context Service used to provide client certificate information for TLS/SSL (https) connections. |
### Relationships
| Name | Description |
-| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| acknowledged | A FlowFile is transferred to this relationship when the acknowledgement was successful. |
| unacknowledged | A FlowFile is transferred to this relationship when the acknowledgement was not successful.<br>This can happen when the acknowledgement did not happened within the time period set for Maximum Waiting Time.<br>FlowFiles with acknowledgement id unknown for the Splunk server will be transferred to this relationship after the Maximum Waiting Time is reached. |
-| undetermined | A FlowFile is transferred to this relationship when the acknowledgement state is not determined.<br> This can happens when Splunk returns with HTTP 200 but with false response for the acknowledgement id in the flow file attribute.<br>FlowFiles transferred to this relationship might be penalized.<br> |
+| undetermined | A FlowFile is transferred to this relationship when the acknowledgement state is not determined.<br> This can happens when Splunk returns with HTTP 200 but with false response for the acknowledgement id in the flow file attribute.<br>FlowFiles transferred to this relationship might be penalized.<br> |
| failure | A FlowFile is transferred to this relationship when the acknowledgement was not successful due to errors during the communication, or if the flowfile was missing the acknowledgement id |
@@ -2119,19 +2186,19 @@ Updates the content of a FlowFile by replacing parts of it using various replace
### Properties
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|**Evaluation Mode**|Line-by-Line|Entire text<br>Line-by-Line<br>|Run the 'Replacement Strategy' against each line separately (Line-by-Line) or against the whole input treated as a single string (Entire Text).|
-|Line-by-Line Evaluation Mode|All|All<br>Except-First-Line<br>Except-Last-Line<br>First-Line<br>Last-Line<br>|Run the 'Replacement Strategy' against each line separately (Line-by-Line) for All lines in the FlowFile, First Line (Header) only, Last Line (Footer) only, all Except the First Line (Header) or all Except the Last Line (Footer).|
-|**Replacement Strategy**|Regex Replace|Always Replace<br>Append<br>Literal Replace<br>Prepend<br>Regex Replace<br>Substitute Variables<br>|The strategy for how and what to replace within the FlowFile's text content. Substitute Variables replaces ${attribute_name} placeholders with the corresponding attribute's value (if an attribute is not found, the placeholder is kept as it was).|
-|**Replacement Value**|||The value to insert using the 'Replacement Strategy'. Using 'Regex Replace' back-references to Regular Expression capturing groups are supported: $& is the entire matched substring, $1, $2, ... are the matched capturing groups. Use $$1 for a literal $1. Back-references to non-existent capturing groups will be replaced by empty strings. Supports expression language except in Regex Replace mode.<br/>**Supports Expression Language: true**|
-|Search Value|||The Search Value to search for in the FlowFile content. Only used for 'Literal Replace' and 'Regex Replace' matching strategies. Supports expression language except in Regex Replace mode.<br/>**Supports Expression Language: true**|
+| Name | Default Value | Allowable Values | Description [...]
+|------------------------------|---------------|-----------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ [...]
+| **Evaluation Mode** | Line-by-Line | Entire text<br>Line-by-Line<br> | Run the 'Replacement Strategy' against each line separately (Line-by-Line) or against the whole input treated as a single string (Entire Text). [...]
+| Line-by-Line Evaluation Mode | All | All<br>Except-First-Line<br>Except-Last-Line<br>First-Line<br>Last-Line<br> | Run the 'Replacement Strategy' against each line separately (Line-by-Line) for All lines in the FlowFile, First Line (Header) only, Last Line (Footer) only, all Except the First Line (Header) or all Except the Last Line (Footer). [...]
+| **Replacement Strategy** | Regex Replace | Always Replace<br>Append<br>Literal Replace<br>Prepend<br>Regex Replace<br>Substitute Variables<br> | The strategy for how and what to replace within the FlowFile's text content. Substitute Variables replaces ${attribute_name} placeholders with the corresponding attribute's value (if an attribute is not found, the placeholder is kept as it was). [...]
+| **Replacement Value** | | | The value to insert using the 'Replacement Strategy'. Using 'Regex Replace' back-references to Regular Expression capturing groups are supported: $& is the entire matched substring, $1, $2, ... are the matched capturing groups. Use $$1 for a literal $1. Back-references to non-existent capturing groups will be replaced by empty strings. Sup [...]
+| Search Value | | | The Search Value to search for in the FlowFile content. Only used for 'Literal Replace' and 'Regex Replace' matching strategies. Supports expression language except in Regex Replace mode.<br/>**Supports Expression Language: true** [...]
### Relationships
-| Name | Description |
-| - | - |
-|failure|FlowFiles that could not be updated are routed to this relationship.|
-|success|FlowFiles that have been successfully processed are routed to this relationship. This includes both FlowFiles that had text replaced and those that did not.|
+| Name | Description |
+|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| failure | FlowFiles that could not be updated are routed to this relationship. |
+| success | FlowFiles that have been successfully processed are routed to this relationship. This includes both FlowFiles that had text replaced and those that did not. |
## RetryFlowFile
@@ -2144,31 +2211,31 @@ FlowFiles passed to this Processor have a 'Retry Attribute' value checked agains
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Retry Attribute|"flowfile.retries"||The name of the attribute that contains the current retry count for the FlowFile. WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, the processor will either overwrite that attribute with '1' or fail based on configuration.<br/>**Supports Expression Language: true**|
-|Maximum Retries|3||The maximum number of times a FlowFile can be retried before being passed to the 'retries_exceeded' relationship.<br/>**Supports Expression Language: true**|
-|Penalize Retries|true||"If set to 'true', this Processor will penalize input FlowFiles before passing them to the 'retry' relationship. This does not apply to the 'retries_exceeded' relationship.|
-|Fail on Non-numerical Overwrite|false||If the FlowFile already has the attribute defined in 'Retry Attribute' that is *not* a number, fail the FlowFile instead of resetting that value to '1'.|
-|Reuse Mode|"Fail on Reuse"|"Fail on Reuse"<br/>"Warn on Reuse"<br/>"Reset Reuse"<br/>|Defines how the Processor behaves if the retry FlowFile has a different retry UUID than the instance that received the FlowFile. This generally means that the attribute was not reset after being successfully retried by a previous instance of this processor. Warn on reuse and Fail on Reuse both resets the retry property value to 1 and marks the flowfile to be last retried by this processor.|
-### Relationships
-
-| Name | Description |
-| - | - |
-|retry|Input FlowFile has not exceeded the configured maximum retry count, pass this relationship back to the input Processor to create a limited feedback loop.|
-|retries_exceeded|Input FlowFile has exceeded the configured maximum retry count, do not pass this relationship back to the input Processor to terminate the limited feedback loop.|
-|failure|The processor is configured such that a non-numerical value on 'Retry Attribute' results in a failure instead of resetting that value to '1'. This will immediately terminate the limited feedback loop. Might also include when 'Maximum Retries' contains attribute expression language that does not resolve to an Integer.|
+| Name | Default Value | Allowable Values | Description [...]
+|---------------------------------|--------------------|------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| Retry Attribute | "flowfile.retries" | | The name of the attribute that contains the current retry count for the FlowFile. WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, the processor will either overwrite that attribute with '1' or fail based on configuration.<br/>**Supports Expression Language: true** [...]
+| Maximum Retries | 3 | | The maximum number of times a FlowFile can be retried before being passed to the 'retries_exceeded' relationship.<br/>**Supports Expression Language: true** [...]
+| Penalize Retries | true | | "If set to 'true', this Processor will penalize input FlowFiles before passing them to the 'retry' relationship. This does not apply to the 'retries_exceeded' relationship. [...]
+| Fail on Non-numerical Overwrite | false | | If the FlowFile already has the attribute defined in 'Retry Attribute' that is *not* a number, fail the FlowFile instead of resetting that value to '1'. [...]
+| Reuse Mode | "Fail on Reuse" | "Fail on Reuse"<br/>"Warn on Reuse"<br/>"Reset Reuse"<br/> | Defines how the Processor behaves if the retry FlowFile has a different retry UUID than the instance that received the FlowFile. This generally means that the attribute was not reset after being successfully retried by a previous instance of this processor. Warn on reuse and Fail on Reuse both resets the retry property value to 1 and marks the flowfile to be last retried [...]
+### Relationships
+
+| Name | Description |
+|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| retry | Input FlowFile has not exceeded the configured maximum retry count, pass this relationship back to the input Processor to create a limited feedback loop. |
+| retries_exceeded | Input FlowFile has exceeded the configured maximum retry count, do not pass this relationship back to the input Processor to terminate the limited feedback loop. |
+| failure | The processor is configured such that a non-numerical value on 'Retry Attribute' results in a failure instead of resetting that value to '1'. This will immediately terminate the limited feedback loop. Might also include when 'Maximum Retries' contains attribute expression language that does not resolve to an Integer. |
### Dynamic Properties:
| Name | Value | Description |
-| - | - | - |
+| --- | --- | - |
|Exceeded FlowFile Attribute Key|The value of the attribute added to the FlowFile|One or more dynamic properties can be used to add attributes to FlowFiles passed to the 'retries_exceeded' relationship.<br/>**Supports Expression Language: true**|
### Writes Attributes:
-| Name | Description |
-| - | - |
-|Retry Attribute|User defined retry attribute is updated with the current retry count|
-|Retry Attribute .uuid| User defined retry attribute with .uuid that determines what processor retried the FlowFile last|
+| Name | Description |
+|-----------------------|---------------------------------------------------------------------------------------------------|
+| Retry Attribute | User defined retry attribute is updated with the current retry count |
+| Retry Attribute .uuid | User defined retry attribute with .uuid that determines what processor retried the FlowFile last |
## RouteOnAttribute
@@ -2180,13 +2247,13 @@ Routes FlowFiles based on their Attributes using the Attribute Expression Langua
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
+|------|---------------|------------------|-------------|
### Relationships
-| Name | Description |
-| - | - |
-|failure|Failed files are transferred to failure|
-|unmatched|Files which do not match any expression are routed here|
+| Name | Description |
+|-----------|---------------------------------------------------------|
+| failure | Failed files are transferred to failure |
+| unmatched | Files which do not match any expression are routed here |
## RouteText
@@ -2194,35 +2261,55 @@ In the list below, the names of required properties appear in bold. Any other pr
Routes textual data based on a set of user-defined rules. Each segment in an incoming FlowFile is compared against the values specified by user-defined Properties. The mechanism by which the text is compared to these user-defined properties is defined by the 'Matching Strategy'. The data is then routed according to these rules, routing each segment of the text individually.
### Properties
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-| Grouping Fallback Value |empty string ("")|| If the 'Grouping Regular Expression' is specified and the matching fails, this value will be considered the group of the segment. |
-| Grouping Regular Expression ||| Specifies a Regular Expression to evaluate against each segment to determine which Group it should be placed in. The Regular Expression must have at least one Capturing Group that defines the segment's Group. If multiple Capturing Groups exist in the Regular Expression, the values from all Capturing Groups will be concatenated together. Two segments will not be placed into the same FlowFile unless they both have the same value for the Group (or neither m [...]
-| Ignore Case | false || If true, capitalization will not be taken into account when comparing values. E.g., matching against 'HELLO' or 'hello' will have the same result. This property is ignored if the 'Matching Strategy' is set to 'Satisfies Expression'. |
-| Ignore Leading/Trailing Whitespace | true || Indicates whether or not the whitespace at the beginning and end should be ignored when evaluating a segment. |
-| Matching Strategy | Starts With<br>Ends With<br>Contains<br>Equals<br>Matches Regex<br>Contains Regex<br>Satisfies Expression || Specifies how to evaluate each segment of incoming text against the user-defined properties. |
-| Routing Strategy | Dynamic Routing | Dynamic Routing<br>Route On All<br>Route On Any | Specifies how to determine which Relationship(s) to use when evaluating the segments of incoming text against the 'Matching Strategy' and user-defined properties. 'Dynamic Routing' routes to all the matching dynamic relationships (or 'unmatched' if none matches). 'Route On All' routes to 'matched' iff all dynamic relationships match. 'Route On Any' routes to 'matched' iff any of the dynamic relations [...]
-| Segmentation Strategy | Per Line | Per Line<br>Full Text | Specifies what portions of the FlowFile content constitutes a single segment to be processed. |
+| Name | Default Value | Allowable Values | Description [...]
+|------------------------------------|-----------------------------------------------------------------------------------------------------------|-------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| Grouping Fallback Value | empty string ("") | | If the 'Grouping Regular Expression' is specified and the matching fails, this value will be considered the group of the segment. [...]
+| Grouping Regular Expression | | | Specifies a Regular Expression to evaluate against each segment to determine which Group it should be placed in. The Regular Expression must have at least one Capturing Group that defines the segment's Group. If multiple Capturing Groups exist in the Regular Expression, the values from all Capt [...]
+| Ignore Case | false | | If true, capitalization will not be taken into account when comparing values. E.g., matching against 'HELLO' or 'hello' will have the same result. This property is ignored if the 'Matching Strategy' is set to 'Satisfies Expression'. [...]
+| Ignore Leading/Trailing Whitespace | true | | Indicates whether or not the whitespace at the beginning and end should be ignored when evaluating a segment. [...]
+| Matching Strategy | Starts With<br>Ends With<br>Contains<br>Equals<br>Matches Regex<br>Contains Regex<br>Satisfies Expression | | Specifies how to evaluate each segment of incoming text against the user-defined properties. [...]
+| Routing Strategy | Dynamic Routing | Dynamic Routing<br>Route On All<br>Route On Any | Specifies how to determine which Relationship(s) to use when evaluating the segments of incoming text against the 'Matching Strategy' and user-defined properties. 'Dynamic Routing' routes to all the matching dynamic relationships (or 'unmatched' if none matches). 'Route On All' routes to 'match [...]
+| Segmentation Strategy | Per Line | Per Line<br>Full Text | Specifies what portions of the FlowFile content constitutes a single segment to be processed. [...]
### Dynamic Properties
| Name | Value | Description |
-| - | - | - |
+| --- | --- | - |
|Relationship Name|value to match against|Routes data that matches the value specified in the Dynamic Property Value to the Relationship specified in the Dynamic Property Key.<br>**Supports Expression Language: true**|
### Relationships
-| Name | Description |
-| - | - |
-|matched|Segments that satisfy the required user-defined rules will be routed to this Relationship|
-|unmatched|Segments that do not satisfy the required user-defined rules will be routed to this Relationship|
-|original|The original input file will be routed to this destination|
+| Name | Description |
+|-----------|--------------------------------------------------------------------------------------------------|
+| matched | Segments that satisfy the required user-defined rules will be routed to this Relationship |
+| unmatched | Segments that do not satisfy the required user-defined rules will be routed to this Relationship |
+| original | The original input file will be routed to this destination |
### Writes Attributes
-| Name | Description |
-| - | - |
-|RouteText.Group|The value captured by all capturing groups in the 'Grouping Regular Expression' property. If this property is not set, this attribute will not be added.|
+| Name | Description |
+|-----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|
+| RouteText.Group | The value captured by all capturing groups in the 'Grouping Regular Expression' property. If this property is not set, this attribute will not be added. |
+
+## TailEventLog
+
+### Description
+
+Windows event log reader that functions as a stateful tail of the provided Windows event log name
+### Properties
+
+In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
+
+| Name | Default Value | Allowable Values | Description |
+|-------------------------|---------------|------------------|--------------------------------------|
+| Log Source | | | Log Source from which to read events |
+| Max Events Per FlowFile | 1 | | Events per flow file |
+### Relationships
+
+| Name | Description |
+|---------|---------------------------------------------------------|
+| success | All files, containing log events, are routed to success |
+
## TailFile
@@ -2233,20 +2320,20 @@ Routes textual data based on a set of user-defined rules. Each segment in an inc
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
-|Attribute Provider Service|||Provides a list of key-value pair records which can be used in the Base Directory property using Expression Language. Requires Multiple file mode.|
-|File to Tail|||Fully-qualified filename of the file that should be tailed when using single file mode, or a file regex when using multifile mode|
-|**Initial Start Position**|Beginning of File|Beginning of Time<br>Beginning of File<br>Current Time|When the Processor first begins to tail data, this property specifies where the Processor should begin reading data. Once data has been ingested from a file, the Processor will continue from the last point from which it has received data.<br>Beginning of Time: Start with the oldest data that matches the Rolling Filename Pattern and then begin reading from the File to Tail.<br>Beginning of [...]
-|Input Delimiter|||Specifies the character that should be used for delimiting the data being tailedfrom the incoming file.If none is specified, data will be ingested as it becomes available.|
-|State File|TailFileState||Specifies the file that should be used for storing state about what data has been ingested so that upon restart NiFi can resume from where it left off|
-|tail-base-directory|||Base directory used to look for files to tail. This property is required when using Multiple file mode. Can contain expression language placeholders if Attribute Provider Service is set.<br/>**Supports Expression Language: true**|
-|**tail-mode**|Single file|Single file<br>Multiple file<br>|Specifies the tail file mode. In 'Single file' mode only a single file will be watched. In 'Multiple file' mode a regex may be used. Note that in multiple file mode we will still continue to watch for rollover on the initial set of watched files. The Regex used to locate multiple files will be run during the schedule phrase. Note that if rotated files are matched by the regex, those files will be tailed.|
+| Name | Default Value | Allowable Values | Description [...]
+|----------------------------|-------------------|--------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| Attribute Provider Service | | | Provides a list of key-value pair records which can be used in the Base Directory property using Expression Language. Requires Multiple file mode. [...]
+| File to Tail | | | Fully-qualified filename of the file that should be tailed when using single file mode, or a file regex when using multifile mode [...]
+| **Initial Start Position** | Beginning of File | Beginning of Time<br>Beginning of File<br>Current Time | When the Processor first begins to tail data, this property specifies where the Processor should begin reading data. Once data has been ingested from a file, the Processor will continue from the last point from which it has received data.<br>Beginning of Time: Start with the oldest data that matches the Rolling Filename Pattern and then begin reading from the File to Tail.<br>Begin [...]
+| Input Delimiter | | | Specifies the character that should be used for delimiting the data being tailedfrom the incoming file.If none is specified, data will be ingested as it becomes available. [...]
+| State File | TailFileState | | Specifies the file that should be used for storing state about what data has been ingested so that upon restart NiFi can resume from where it left off [...]
+| tail-base-directory | | | Base directory used to look for files to tail. This property is required when using Multiple file mode. Can contain expression language placeholders if Attribute Provider Service is set.<br/>**Supports Expression Language: true** [...]
+| **tail-mode** | Single file | Single file<br>Multiple file<br> | Specifies the tail file mode. In 'Single file' mode only a single file will be watched. In 'Multiple file' mode a regex may be used. Note that in multiple file mode we will still continue to watch for rollover on the initial set of watched files. The Regex used to locate multiple files will be run during the schedule phrase. Note that if rotated files are matched by the regex, thos [...]
### Relationships
-| Name | Description |
-| - | - |
-|success|All files are routed to success|
+| Name | Description |
+|---------|---------------------------------|
+| success | All files are routed to success |
## UnfocusArchiveEntry
@@ -2259,12 +2346,12 @@ Restores a FlowFile which has had an archive entry focused via FocusArchiveEntry
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
+|------|---------------|------------------|-------------|
### Relationships
-| Name | Description |
-| - | - |
-|success|success operational on the flow record|
+| Name | Description |
+|---------|----------------------------------------|
+| success | success operational on the flow record |
## UpdateAttribute
@@ -2277,10 +2364,10 @@ This processor updates the attributes of a FlowFile using properties that are ad
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
| Name | Default Value | Allowable Values | Description |
-| - | - | - | - |
+|------|---------------|------------------|-------------|
### Relationships
-| Name | Description |
-| - | - |
-|failure|Failed files are transferred to failure|
-|success|All files are routed to success|
+| Name | Description |
+|---------|-----------------------------------------|
+| failure | Failed files are transferred to failure |
+| success | All files are routed to success |
diff --git a/README.md b/README.md
index 2ffb244e1..36b171586 100644
--- a/README.md
+++ b/README.md
@@ -63,8 +63,8 @@ MiNiFi - C++ supports the following C++ processors:
The following table lists the base set of processors.
-| Extension Set | Processors [...]
-|---------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ [...]
+| Extension Set | Processors [...]
+|---------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ [...]
| **Base** | [AppendHostInfo](PROCESSORS.md#appendhostinfo)<br/>[DefragmentText](PROCESSORS.md#defragmenttext)<br/>[ExecuteProcess](PROCESSORS.md#executeprocess)<br/>[ExtractText](PROCESSORS.md#extracttext)<br/>[FetchFile](PROCESSORS.md#fetchfile)<br/>[GenerateFlowFile](PROCESSORS.md#generateflowfile)<br/>[GetFile](PROCESSORS.md#getfile)<br/>[GetTCP](PROCESSORS.md#gettcp)<br/>[HashContent](PROCESSORS.md#hashcontent)<br/>[ListenSyslog](PROCESSORS.md#listensyslog)<br/>[ListFile](PROCE [...]
The next table outlines CMAKE flags that correspond with MiNiFi extensions. Extensions that are enabled by default ( such as CURL ), can be disabled with the respective CMAKE flag on the command line.
@@ -72,34 +72,34 @@ The next table outlines CMAKE flags that correspond with MiNiFi extensions. Exte
Through JNI extensions you can run NiFi processors using NARs. The JNI extension set allows you to run these Java processors. MiNiFi C++ will favor C++ implementations over Java implements. In the case where a processor is implemented in either language, the one in C++ will be selected; however, will remain transparent to the consumer.
-| Extension Set | Processors and Controller Services [...]
-|----------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
-| Archive Extensions | [ApplyTemplate](PROCESSORS.md#applytemplate)<br/>[CompressContent](PROCESSORS.md#compresscontent)<br/>[ManipulateArchive](PROCESSORS.md#manipulatearchive)<br/>[MergeContent](PROCESSORS.md#mergecontent)<br/>[FocusArchiveEntry](PROCESSORS.md#focusarchiveentry)<br/>[UnfocusArchiveEntry](PROCESSORS.md#unfocusarchiveentry) [...]
-| AWS | [AWSCredentialsService](CONTROLLERS.md#awscredentialsservice)<br/>[PutS3Object](PROCESSORS.md#puts3object)<br/>[DeleteS3Object](PROCESSORS.md#deletes3object)<br/>[FetchS3Object](PROCESSORS.md#fetchs3object)<br/>[ListS3](PROCESSORS.md#lists3) [...]
+| Extension Set | Processors and Controller Services [...]
+|----------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| Archive Extensions | [ApplyTemplate](PROCESSORS.md#applytemplate)<br/>[CompressContent](PROCESSORS.md#compresscontent)<br/>[ManipulateArchive](PROCESSORS.md#manipulatearchive)<br/>[MergeContent](PROCESSORS.md#mergecontent)<br/>[FocusArchiveEntry](PROCESSORS.md#focusarchiveentry)<br/>[UnfocusArchiveEntry](PROCESSORS.md#unfocusarchiveentry) [...]
+| AWS | [AWSCredentialsService](CONTROLLERS.md#awscredentialsservice)<br/>[PutS3Object](PROCESSORS.md#puts3object)<br/>[DeleteS3Object](PROCESSORS.md#deletes3object)<br/>[FetchS3Object](PROCESSORS.md#fetchs3object)<br/>[ListS3](PROCESSORS.md#lists3) [...]
| Azure | [AzureStorageCredentialsService](CONTROLLERS.md#azurestoragecredentialsservice)<br/>[PutAzureBlobStorage](PROCESSORS.md#putazureblobatorage)<br/>[DeleteAzureBlobStorage](#deleteazureblobstorage)<br/>[FetchAzureBlobStorage](#fetchazureblobstorage)<br/>[ListAzureBlobStorage](#listazureblobstorage)<br/>[PutAzureDataLakeStorage](#putazuredatalakestorage)<br/>[DeleteAzureDataLakeStorage](#deleteazuredatalakestorage)<br/>[FetchAzureDataLakeStorage](#fetchaz [...]
-| CivetWeb | [ListenHTTP](PROCESSORS.md#listenhttp) [...]
-| CURL | [InvokeHTTP](PROCESSORS.md#invokehttp) [...]
-| GPS | GetGPS [...]
-| Google Cloud Platform | [DeleteGCSObject](PROCESSORS.md#deletegcsobject)<br>[FetchGCSObject](PROCESSORS.md#fetchgcsobject)<br>[GCPCredentialsControllerService](CONTROLLERS.md#GCPCredentialsControllerService)<br>[ListGCSBucket](PROCESSORS.md#listgcsbucket)<br>[PutGCSObject](PROCESSORS.md#putgcsobject) [...]
-| Kafka | [PublishKafka](PROCESSORS.md#publishkafka) [...]
-| Kubernetes | [KubernetesControllerService](CONTROLLERS.md#kubernetesControllerService) [...]
-| JNI | **NiFi Processors** [...]
-| MQTT | [ConsumeMQTT](PROCESSORS.md#consumeMQTT)<br/>[PublishMQTT](PROCESSORS.md#publishMQTT) [...]
-| OPC | [FetchOPCProcessor](PROCESSORS.md#fetchopcprocessor) [...]
-| OpenCV | [CaptureRTSPFrame](PROCESSORS.md#captureRTSPFrame) [...]
-| OpenWSMAN | SourceInitiatedSubscriptionListener [...]
-| PCAP | [CapturePacket](PROCESSORS.md#capturepacket) [...]
-| PDH (Windows only) | [PerformanceDataMonitor](PROCESSORS.md#performancedatamonitor) [...]
-| ProcFs (Linux only) | [ProcFsMonitor](PROCESSORS.md#procfsmonitor) [...]
-| Scripting | [ExecuteScript](PROCESSORS.md#executescript)<br/>**Custom Python Processors** [...]
-| Sensors | GetEnvironmentalSensors<br/>GetMovementSensors [...]
-| SFTP | [FetchSFTP](PROCESSORS.md#fetchsftp)<br/>[ListSFTP](PROCESSORS.md#listsftp)<br/>[PutSFTP](PROCESSORS.md#putsftp) [...]
-| SQL | [ExecuteSQL](PROCESSORS.md#executesql)<br/>[PutSQL](PROCESSORS.md#putsql)<br/>[QueryDatabaseTable](PROCESSORS.md#querydatabasetable)<br/> [...]
-| Splunk | [PutSplunkHTTP](PROCESSORS.md#putsplunkhttp)<br/>[QuerySplunkIndexingStatus](PROCESSORS.md#querysplunkindexingstatus) [...]
-| Systemd | [ConsumeJournald](PROCESSORS.md#consumejournald) [...]
-| Tensorflow | TFApplyGraph<br/>TFConvertImageToTensor<br/>TFExtractTopLabels<br/> [...]
-| USB Camera | [GetUSBCamera](PROCESSORS.md#getusbcamera) [...]
-| Windows Event Log (Windows only) | CollectorInitiatedSubscription<br/>ConsumeWindowsEventLog<br/>TailEventLog [...]
+| CivetWeb | [ListenHTTP](PROCESSORS.md#listenhttp) [...]
+| CURL | [InvokeHTTP](PROCESSORS.md#invokehttp) [...]
+| GPS | [GetGPS](PROCESSORS.md#getgps) [...]
+| Google Cloud Platform | [DeleteGCSObject](PROCESSORS.md#deletegcsobject)<br>[FetchGCSObject](PROCESSORS.md#fetchgcsobject)<br>[GCPCredentialsControllerService](CONTROLLERS.md#GCPCredentialsControllerService)<br>[ListGCSBucket](PROCESSORS.md#listgcsbucket)<br>[PutGCSObject](PROCESSORS.md#putgcsobject) [...]
+| Kafka | [PublishKafka](PROCESSORS.md#publishkafka) [...]
+| Kubernetes | [KubernetesControllerService](CONTROLLERS.md#kubernetesControllerService) [...]
+| JNI | **NiFi Processors** [...]
+| MQTT | [ConsumeMQTT](PROCESSORS.md#consumeMQTT)<br/>[PublishMQTT](PROCESSORS.md#publishMQTT) [...]
+| OPC | [FetchOPCProcessor](PROCESSORS.md#fetchopcprocessor) [...]
+| OpenCV | [CaptureRTSPFrame](PROCESSORS.md#captureRTSPFrame) [...]
+| OpenWSMAN | SourceInitiatedSubscriptionListener [...]
+| PCAP | [CapturePacket](PROCESSORS.md#capturepacket) [...]
+| PDH (Windows only) | [PerformanceDataMonitor](PROCESSORS.md#performancedatamonitor) [...]
+| ProcFs (Linux only) | [ProcFsMonitor](PROCESSORS.md#procfsmonitor) [...]
+| Scripting | [ExecuteScript](PROCESSORS.md#executescript)<br/>**Custom Python Processors** [...]
+| Sensors | GetEnvironmentalSensors<br/>GetMovementSensors [...]
+| SFTP | [FetchSFTP](PROCESSORS.md#fetchsftp)<br/>[ListSFTP](PROCESSORS.md#listsftp)<br/>[PutSFTP](PROCESSORS.md#putsftp) [...]
+| SQL | [ExecuteSQL](PROCESSORS.md#executesql)<br/>[PutSQL](PROCESSORS.md#putsql)<br/>[QueryDatabaseTable](PROCESSORS.md#querydatabasetable)<br/> [...]
+| Splunk | [PutSplunkHTTP](PROCESSORS.md#putsplunkhttp)<br/>[QuerySplunkIndexingStatus](PROCESSORS.md#querysplunkindexingstatus) [...]
+| Systemd | [ConsumeJournald](PROCESSORS.md#consumejournald) [...]
+| Tensorflow | TFApplyGraph<br/>TFConvertImageToTensor<br/>TFExtractTopLabels<br/> [...]
+| USB Camera | [GetUSBCamera](PROCESSORS.md#getusbcamera) [...]
+| Windows Event Log (Windows only) | [CollectorInitiatedSubscription](PROCESSORS.md#collectorinitiatedsubscription)<br/>[ConsumeWindowsEventLog](PROCESSORS.md#consumewindowseventlog)<br/>[TailEventLog](PROCESSORS.md#taileventlog) [...]
Please see our [Python guide](extensions/script/README.md) on how to write Python processors and use them within MiNiFi C++.
@@ -470,12 +470,12 @@ If you have docker installed on your machine you can build for CentOS 7, Fedora
provides the command to build your distro and the output file in your build directory. Since the versions are limited ( except for Ubuntu ) we output the archive based on the distro's name.
-| Distro | command | Output File |
-| ---------------------- | :---------------- | :----------- |
-| CentOS 7 | make centos | nifi-minifi-cpp-centos-$VERSION.tar.gz
-| Fedora 34 | make fedora | nifi-minifi-cpp-fedora-$VERSION.tar.gz
-| Ubuntu 18.04 (bionic) | make u18 | nifi-minifi-cpp-bionic-$VERSION.tar.gz
-| Ubuntu 20.04 (focal) | make u20 | nifi-minifi-cpp-focal-$VERSION.tar.gz
+| Distro | command | Output File |
+|-----------------------|:------------|:---------------------------------------|
+| CentOS 7 | make centos | nifi-minifi-cpp-centos-$VERSION.tar.gz |
+| Fedora 34 | make fedora | nifi-minifi-cpp-fedora-$VERSION.tar.gz |
+| Ubuntu 18.04 (bionic) | make u18 | nifi-minifi-cpp-bionic-$VERSION.tar.gz |
+| Ubuntu 20.04 (focal) | make u20 | nifi-minifi-cpp-focal-$VERSION.tar.gz |
You can avoid the requirement of an up-to-date compiler when generating the build system by adding `-DDOCKER_BUILD_ONLY=ON` to the cmake command line. This disables all cmake targets except the docker build and test ones.