You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by al...@apache.org on 2016/11/30 19:25:18 UTC
[1/2] nifi-minifi git commit: MINIFI-127: Added quick start document.
Repository: nifi-minifi
Updated Branches:
refs/heads/master 5ac121fae -> bcf6c6cb2
MINIFI-127: Added quick start document.
Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi/commit/9ea13c65
Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi/tree/9ea13c65
Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi/diff/9ea13c65
Branch: refs/heads/master
Commit: 9ea13c65fa2689841dd2de0ef93ea1695e03dc4e
Parents: 5ac121f
Author: Sarah Olson <so...@yahoo.com>
Authored: Tue Oct 25 13:42:15 2016 -0700
Committer: Aldrin Piri <al...@apache.org>
Committed: Wed Nov 30 14:24:38 2016 -0500
----------------------------------------------------------------------
.../markdown/minifi-java-agent-quick-start.md | 288 +++++++++++++++++++
1 file changed, 288 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/9ea13c65/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
----------------------------------------------------------------------
diff --git a/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md b/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
new file mode 100644
index 0000000..80ca1cb
--- /dev/null
+++ b/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
@@ -0,0 +1,288 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# MiNiFi Java Agent Quick Start Guide
+
+[Apache NiFi Team](dev@nifi.apache.org>)
+
+[NiFi Homepage](http://nifi.apache.org)
+
+# Overview
+
+Apache NiFi MiNiFi is an Apache NiFi project, designed to collect data at its source. MiNiFi was developed with the following objectives in mind:
+* Small and lightweight footprint
+* Central agent management
+* Data provenance generation
+* NiFi integration for follow-on dataflow management and chain of custody information
+
+# Before You Begin
+MiNiFi Java Agent is supported on the following operating systems:
+* Red Hat Enterprise Linux / CentOS 6 (64-bit)
+* Red Hat Enterprise Linux / CentOS 7 (64-bit)
+* Ubuntu Precise (12.04) (64-bit)
+* Ubuntu Trusty (14.04) (64-bit)
+* Debian 7
+* SUSE Linux Enterprise Server (SLES) 11 SP3 (64-bit)
+
+You can download the MiNiFi Java Agent and the MiNiFi Toolkit from the [MiNiFi download page](http://nifi.apache.org/minifi/download.html).
+
+# Installing and Starting MiNiFi
+You have several options for installing and starting MiNiFi.
+
+## For Linux and Mac OS X Users
+To install MiNiFi:
+1. [Download](http://nifi.apache.org/minifi/download.html) MiNiFi.
+2. Extract the file to the location from which you want to the application.
+
+You can also install MiNiFi as a service:
+1. Navigate to the MiNiFi installation directory.
+2. Enter:
+```
+bin/minifi.sh install
+```
+**Note:** You can also specify a custom name for your MiNiFi installation, by specifying that name during your install command. For example, to install MiNiFi as a service and named dataflow, enter:
+
+```
+bin/minifi.sh install dataflow
+```
+Once you have downloaded and installed MiNiFi, you need to start MiNiF. You can start NiFi in the foreground, background, or as a service.
+
+To launch MiNiFi in the foreground:
+
+1. From a terminal window, navigate to the MiNiFi installation directory.Enter:
+
+```
+bin/minifi.sh run
+```
+To launch MiNiFi in the background:
+1. From a terminal window, navigate to the MiNiFi installation directory.
+2. Enter:
+```
+bin/minifi.sh start
+```
+
+To launch MiNiFi as a service:
+1. From a terminal window, enter:
+```
+sudo service minifi start
+```
+## For Windows Users
+For Windows users, navigate to the folder where MiNiFi was installed. Navigate to the /bin subdirectory and double-click the run-minifi.bat file.
+
+This launches MiNiFi and leaves it running in the foreground. To shut down NiFi, select the window that was launched and hold the Ctrl key while pressing C.
+
+#Working with dataflows
+When you are working with a MiNiFi dataflow, you should design it, add any additional configuration your environment or use case requires, and then deploy your dataflow. MiNiFi is not designed to accommodate substantial mid-dataflow configuration.
+
+##Setting up Your Dataflow
+Before you begin, you should be aware that the following NiFi components are not supported in MiNiFi dataflows:
+* Controller services
+* Funnels
+* Multiple source relationships for a single connection
+* Process groups
+
+Additionally, each processor requires a distinct name.
+
+You can use the MiNiFi Toolkit, located in your MiNiFi installation directory, and any NiFi instance to set up the dataflow you want MiNiFi to run:
+
+1. Launch NiFi
+2. Create a dataflow.
+3. Convert your dataflow into a template.
+4. Download your template as an .xml file. For more information on working with templates, see the [Templates](http://docs.hortonworks.com/HDPDocuments/HDF2/HDF-2.0.0/bk_user-guide/content/templates.html) section in the *User Guide*.
+5. From the MiNiFi Toolkit, run the following command to turn your .xml file into a .yml file:
+```
+config.sh transform input_file output_file
+```
+6. Move your new .yml file to minifi/conf.
+7. Rename your .yml file config.yml.
+**Note:** You can use one template at a time, per MiNiFi instance.
+
+
+**Result:** Once you have your config.yml file in the minifi/conf directory, launch that instance of MiNiFi and your dataflow begins automatically.
+
+##Using Processors Not Packaged with MiNiFi
+MiNiFi is shipped with the following processors:
+* UpdateAttribute
+* AttributesToJSON
+* Base64EncodeContent
+* CompressContent
+* ControlRate
+* ConvertCharacterSet
+* DetectDuplicate
+* DistributeLoad
+* DuplicateFlowFile
+* EncryptContent
+* EvaluateJsonPath
+* EvaluateRegularExpression
+* EvaluateXPath
+* EvaluateXQuery
+* ExecuteProcess
+* ExecuteSQL
+* ExecuteStreamCommand
+* ExtractText
+* FetchDistributedMapCache
+* FetchFile
+* FetchSFTP
+* GenerateFlowFile
+* GetFTP
+* GetFile
+* GetHTTP
+* GetJMSQueue
+* GetJMSTopic
+* GetSFTP
+* HandleHttpRequest
+* HandleHttpResponse
+* HashAttribute
+* HashContent
+* IdentifyMimeType
+* InvokeHTTP
+* ListFile
+* ListSFTP
+* ListenHTTP
+* ListenRELP
+* ListenSyslog
+* ListenTCP
+* ListenUDP
+* LogAttribute
+* MergeContent
+* ModifyBytes
+* MonitorActivity
+* ParseSyslog
+* PostHTTP
+* PutDistributedMapCache
+* PutEmail
+* PutFTP
+* PutFile
+* PutJMS
+* PutSFTP
+* PutSQL
+* PutSyslog
+* QueryDatabaseTable
+* ReplaceText
+* ReplaceTextWithMapping
+* RouteOnAttribute
+* RouteOnContent
+* RouteText
+* ScanAttribute
+* ScanContent
+* SegmentContent
+* SplitContent
+* SplitJson
+* SplitText
+* SplitXml
+* TailFile
+* TransformXml
+* UnpackContent
+* ValidateXml
+
+If you want to create a dataflow with a processor not shipped with MiNiFi, you can do so.
+1. Set up your dataflow as described above.
+2. Copy the desired NAR file into the MiNiFi lib directory.
+3. Restart your MiNiFi instance.
+
+# Securing your Dataflow
+You can secure your MiNiFi dataflow using keystore or trust store SSL protocols, however, this information is not automatically generated. You will need to generate your security configuration information yourself.
+
+To run a MiNiFi dataflow securely, modify the Security Properties section of your config.yml file.
+1. Create your dataflow template as discussed above.
+2. Move it to minifi.conf and rename config.yml.
+3. Manually modify the Security Properties section of config.yml. For example:
+
+```
+Security Properties:
+keystore:
+keystore type:
+keystore password:
+key password:
+truststore:
+truststore type:
+truststore password:
+ssl protocol: TLS
+Sensitive Props:
+key:
+algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
+provider: BC
+```
+
+# Managing MiNiFi
+You can also perform some management tasks using MiNiFi.
+
+## Monitoring Status
+You can use the minifi.sh flowStatus option to monitor a range of aspects of your MiNiFi operational and dataflow status. You can use the flowStatus option to get information dataflow component health and functionality, a MiNiFi instance, or system diagnostics.
+
+FlowStatus accepts the following flags and options:
+* processors
+ * health
+ * bulletins
+ * status
+* connections
+ * health
+ * stats
+* remoteProcessGroups
+ * health
+ * bulletins
+ * status
+ * authorizationIssues
+ * inputPorts
+* controllerServices
+ * health
+ * bulletins
+* provenancereporting
+ * health
+ * bulletins
+* instance
+ * health
+ * bulletins
+ * status
+* systemdiagnostics
+ * heap
+ * processorstats
+ * contentrepositoryusage
+ * flowfilerepositoryusage
+ * garbagecollection
+
+For example, this query gets the health, stats, and bulletins for the TailFile processors:
+
+**Note:** Currently, the script only accepts one high level option at a time.
+
+**Note:** Any connections, remote process groups or processors names that contain ":", ";" or "," will cause parsing errors when querying.
+
+For details on the flowStatus option, see the _FlowStatus Query Option_ section of the [Administration Guide](http://nifi.apache.org/minifi/system-admin-guide.html).
+
+## Loading a New Dataflow
+You can load a new dataflow for a MiNiFi instance to run:
+
+1. Create a new config.yml file with the new dataflow.
+2. Replace the existing config.yml in minifi/conf with the new file.
+3. Restart MiNiFi.
+
+## Stopping MiNiFi
+
+You can MiNiFi at any time.
+
+Stopping MiNiFi:
+1. From a terminal window, navigate to the MiNiFi installation directory.
+2. Enter:
+```
+bin/minifi.sh stop
+```
+
+Stopping MiNiFi as a service:
+1. From a terminal window, enter:
+```
+sudo service minifi stop
+```
[2/2] nifi-minifi git commit: MINIFI-127 In quick start guide,
removing processors that are currently unusable due to reliance on
Controller Services other than SSL Context
Posted by al...@apache.org.
MINIFI-127 In quick start guide, removing processors that are currently unusable due to reliance on Controller Services other than SSL Context
This closes #49.
Signed-off-by: Aldrin Piri <al...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi/commit/bcf6c6cb
Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi/tree/bcf6c6cb
Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi/diff/bcf6c6cb
Branch: refs/heads/master
Commit: bcf6c6cb2fe1c00c83559a2bee490d5b468ec98f
Parents: 9ea13c6
Author: Joseph Percivall <JP...@apache.org>
Authored: Wed Nov 30 12:07:28 2016 -0500
Committer: Aldrin Piri <al...@apache.org>
Committed: Wed Nov 30 14:24:42 2016 -0500
----------------------------------------------------------------------
.../markdown/minifi-java-agent-quick-start.md | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/nifi-minifi/blob/bcf6c6cb/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
----------------------------------------------------------------------
diff --git a/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md b/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
index 80ca1cb..63ee343 100644
--- a/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
+++ b/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
@@ -115,15 +115,13 @@ config.sh transform input_file output_file
**Result:** Once you have your config.yml file in the minifi/conf directory, launch that instance of MiNiFi and your dataflow begins automatically.
##Using Processors Not Packaged with MiNiFi
-MiNiFi is shipped with the following processors:
+MiNiFi is able to use following processors out of the box:
* UpdateAttribute
* AttributesToJSON
* Base64EncodeContent
* CompressContent
* ControlRate
* ConvertCharacterSet
-* DetectDuplicate
-* DistributeLoad
* DuplicateFlowFile
* EncryptContent
* EvaluateJsonPath
@@ -131,10 +129,8 @@ MiNiFi is shipped with the following processors:
* EvaluateXPath
* EvaluateXQuery
* ExecuteProcess
-* ExecuteSQL
* ExecuteStreamCommand
* ExtractText
-* FetchDistributedMapCache
* FetchFile
* FetchSFTP
* GenerateFlowFile
@@ -144,8 +140,6 @@ MiNiFi is shipped with the following processors:
* GetJMSQueue
* GetJMSTopic
* GetSFTP
-* HandleHttpRequest
-* HandleHttpResponse
* HashAttribute
* HashContent
* IdentifyMimeType
@@ -163,15 +157,12 @@ MiNiFi is shipped with the following processors:
* MonitorActivity
* ParseSyslog
* PostHTTP
-* PutDistributedMapCache
* PutEmail
* PutFTP
* PutFile
* PutJMS
* PutSFTP
-* PutSQL
* PutSyslog
-* QueryDatabaseTable
* ReplaceText
* ReplaceTextWithMapping
* RouteOnAttribute
@@ -194,6 +185,10 @@ If you want to create a dataflow with a processor not shipped with MiNiFi, you c
2. Copy the desired NAR file into the MiNiFi lib directory.
3. Restart your MiNiFi instance.
+**Note:** Currently only the StandardSSLContextService is supported as a controller service. It is created automatically if the the "Security Properties" section is set and can be referenced in the processor configuration using the ID "SSL-Context-Service".
+
+
+
# Securing your Dataflow
You can secure your MiNiFi dataflow using keystore or trust store SSL protocols, however, this information is not automatically generated. You will need to generate your security configuration information yourself.
@@ -272,9 +267,10 @@ You can load a new dataflow for a MiNiFi instance to run:
## Stopping MiNiFi
-You can MiNiFi at any time.
+You can stop MiNiFi at any time.
Stopping MiNiFi:
+
1. From a terminal window, navigate to the MiNiFi installation directory.
2. Enter:
```
@@ -282,6 +278,7 @@ bin/minifi.sh stop
```
Stopping MiNiFi as a service:
+
1. From a terminal window, enter:
```
sudo service minifi stop