You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by al...@apache.org on 2017/04/13 14:39:35 UTC
nifi-site git commit: MINIFI-260: Added MiNiFi getting-started page,
formatting tweaks to the minifi index.html page Added overview of
agent differences. Combined setup steps into one set of instructions
Repository: nifi-site
Updated Branches:
refs/heads/master 355e42dc8 -> 4d7a200e0
MINIFI-260: Added MiNiFi getting-started page, formatting tweaks to the minifi index.html page
Added overview of agent differences. Combined setup steps into one set of instructions
This closes #17.
Signed-off-by: Aldrin Piri <al...@gmail.com>
Project: http://git-wip-us.apache.org/repos/asf/nifi-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi-site/commit/4d7a200e
Tree: http://git-wip-us.apache.org/repos/asf/nifi-site/tree/4d7a200e
Diff: http://git-wip-us.apache.org/repos/asf/nifi-site/diff/4d7a200e
Branch: refs/heads/master
Commit: 4d7a200e032ce353ca48d23338ba69335f3e20d6
Parents: 355e42d
Author: Randy Gelhausen <rg...@gmail.com>
Authored: Sat Apr 1 15:29:42 2017 -0400
Committer: Aldrin Piri <al...@gmail.com>
Committed: Thu Apr 13 10:39:06 2017 -0400
----------------------------------------------------------------------
.gitignore | 2 +
src/images/minifi/minifi-flow.png | Bin 0 -> 131996 bytes
src/images/minifi/nifi-receive.png | Bin 0 -> 92125 bytes
src/images/minifi/nifi-received.png | Bin 0 -> 95205 bytes
src/includes/minifi/minifi-topbar.hbs | 7 +-
src/includes/topbar.hbs | 2 +-
src/pages/html/minifi/index.hbs | 8 +-
src/pages/markdown/minifi/getting-started.md | 122 ++++++++++++++++++++++
8 files changed, 133 insertions(+), 8 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/nifi-site/blob/4d7a200e/.gitignore
----------------------------------------------------------------------
diff --git a/.gitignore b/.gitignore
index d0335c5..4a8f655 100644
--- a/.gitignore
+++ b/.gitignore
@@ -8,6 +8,8 @@ nb-configuration.xml
.metadata
.recommenders
+*.swp
+
# Intellij
.idea/
*.iml
http://git-wip-us.apache.org/repos/asf/nifi-site/blob/4d7a200e/src/images/minifi/minifi-flow.png
----------------------------------------------------------------------
diff --git a/src/images/minifi/minifi-flow.png b/src/images/minifi/minifi-flow.png
new file mode 100644
index 0000000..a868013
Binary files /dev/null and b/src/images/minifi/minifi-flow.png differ
http://git-wip-us.apache.org/repos/asf/nifi-site/blob/4d7a200e/src/images/minifi/nifi-receive.png
----------------------------------------------------------------------
diff --git a/src/images/minifi/nifi-receive.png b/src/images/minifi/nifi-receive.png
new file mode 100644
index 0000000..6ecced2
Binary files /dev/null and b/src/images/minifi/nifi-receive.png differ
http://git-wip-us.apache.org/repos/asf/nifi-site/blob/4d7a200e/src/images/minifi/nifi-received.png
----------------------------------------------------------------------
diff --git a/src/images/minifi/nifi-received.png b/src/images/minifi/nifi-received.png
new file mode 100644
index 0000000..48d6da2
Binary files /dev/null and b/src/images/minifi/nifi-received.png differ
http://git-wip-us.apache.org/repos/asf/nifi-site/blob/4d7a200e/src/includes/minifi/minifi-topbar.hbs
----------------------------------------------------------------------
diff --git a/src/includes/minifi/minifi-topbar.hbs b/src/includes/minifi/minifi-topbar.hbs
index d12b42a..7dcc98b 100644
--- a/src/includes/minifi/minifi-topbar.hbs
+++ b/src/includes/minifi/minifi-topbar.hbs
@@ -26,16 +26,17 @@
<li class="has-dropdown">
<a href="#">Documentation</a>
<ul class="dropdown">
- <li><a href="faq.html"><i class="fa"></i>FAQ</a></li>
+ <li><a href="getting-started.html"><i class="fa"></i>Getting Started</a></li>
<li><a href="system-admin-guide.html"><i class="fa"></i>Administrator's Guide</a></li>
<li><a href="minifi-toolkit.html"><i class="fa"></i>Converter Toolkit</a></li>
+ <li><a href="faq.html"><i class="fa"></i>FAQ</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/MiNiFi"><i class="fa fa-external-link external-link"></i>Wiki</a></li>
</ul>
</li>
<li class="has-dropdown">
- <a href="#">Download</a>
+ <a href="#">Downloads</a>
<ul class="dropdown">
- <li><a href="download.html">Download MiNiFi</a></li>
+ <li><a href="download.html">Download MiNiFi Components</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Version0.0.1"><i class="fa fa-external-link external-link"></i>Release Notes</a></li>
</ul>
</li>
http://git-wip-us.apache.org/repos/asf/nifi-site/blob/4d7a200e/src/includes/topbar.hbs
----------------------------------------------------------------------
diff --git a/src/includes/topbar.hbs b/src/includes/topbar.hbs
index 0d354e0..016d781 100644
--- a/src/includes/topbar.hbs
+++ b/src/includes/topbar.hbs
@@ -34,7 +34,7 @@
</ul>
</li>
<li class="has-dropdown">
- <a href="#">Download</a>
+ <a href="#">Downloads</a>
<ul class="dropdown">
<li><a href="download.html">Download NiFi</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/NIFI/Release+Notes"><i class="fa fa-external-link external-link"></i>Release Notes</a></li>
http://git-wip-us.apache.org/repos/asf/nifi-site/blob/4d7a200e/src/pages/html/minifi/index.hbs
----------------------------------------------------------------------
diff --git a/src/pages/html/minifi/index.hbs b/src/pages/html/minifi/index.hbs
index 354399b..3a93b74 100644
--- a/src/pages/html/minifi/index.hbs
+++ b/src/pages/html/minifi/index.hbs
@@ -34,10 +34,10 @@ title: Apache NiFi - MiNiFi
<p class="description">
Specific goals for the initial thrust of the MiNiFi effort comprise:
<ul>
- <li>small and lightweight footprint,</li>
- <li>central management of agents,</li>
- <li>generation of data provenance, and</li>
- <li>integration with NiFi for follow-on dataflow management and full chain of custody of information.</li>
+ <li>Small size and low resource consumption</li>
+ <li>Central management of agents</li>
+ <li>Generation of data provenance (full chain of custody of information)</li>
+ <li>Integration with NiFi for follow-on dataflow management</li>
</ul>
Perspectives of the role of MiNiFi should be from the perspective of the agent acting immediately at, or directly adjacent to, source sensors, systems, or servers.
http://git-wip-us.apache.org/repos/asf/nifi-site/blob/4d7a200e/src/pages/markdown/minifi/getting-started.md
----------------------------------------------------------------------
diff --git a/src/pages/markdown/minifi/getting-started.md b/src/pages/markdown/minifi/getting-started.md
new file mode 100644
index 0000000..c411750
--- /dev/null
+++ b/src/pages/markdown/minifi/getting-started.md
@@ -0,0 +1,122 @@
+---
+title: Apache NiFi MiNiFi: Getting Started
+---
+
+# Getting started with MiNiFi
+
+This page explains how to configure and deploy MiNiFi agents.
+
+The Java agent is able to run most of [NiFi's available processors](http://nifi.apache.org/docs.html), but is a larger binary distribution (49MB) and consumes greater system resources (24MB max JVM heapsize by default). If you need maximum flexibility to make routing and processing decisions at your data's point of origin, the Java agent is a good fit.
+
+The C++ agent is a smaller binary (3.2MB), consumes low system memory (about 5MB at idle) but has [a limited subset of processors](https://github.com/apache/nifi-minifi-cpp#caveats). If your primary concern is gathering and pushing data to downstream consumers and minimizing system impact, the C++ agent is a good fit.
+
+
+1. Install the appropriate OS level dependencies:
+
+ #### MiNiFi Java:
+
+ - Java 1.8+
+
+ #### MiNiFi C++:
+ ###### RHEL/CentOS:
+ - yum install -y epel-release
+ - yum install -y leveldb
+
+ ###### Debian/Ubuntu:
+ - apt install -y libleveldb-dev
+ - apt install -y libxml2
+2. Download the relevant compressed binary from the [Downloads](download.html) page
+3. Copy and decompress the binary to your target deployment environment
+4. Set the MINIFI_HOME environment variable to your decompressed binary directory
+5. From $MINIFI_HOME, ./bin/minifi.sh {start|stop|run|restart|status|flowStatus|dump|install}
+
+*flowStatus* and *dump* are available only for MiNiFi Java.
+
+For MiNiFi Java, the agent logs to $MINIFI_HOME/logs/minifi-app.log.
+
+For MiNiFi C++, the agent logs to $MINIFI_HOME/minifi-app.log.txt
+
+# Configuring MiNiFi Agent Flows
+
+Agents execute the flow defined in $MINIFI_HOME/conf/config.yml. The shipped config.yml file has an "empty" flow.
+
+While it's possible to create and modify config.yml programmatically or by hand, the more common scenario is to:
+1. Use NiFi's web interface to define a flow
+2. Save the flow as a template and download its XML file
+3. Use the [Converter Toolkit](minifi-toolkit.html) to convert the template XML to an equivalent config.yml
+4. Put the config.yml file in $MINIFI_HOME/conf and start the agent.
+
+# Pushing Data From MiNiFi to NiFi
+
+What follows is an illustrated walkthrough of configuring MiNiFi and NiFi to generate and collect log data.
+
+If you're new to NiFi, watch the [NiFi introductory video series](http://nifi.apache.org/videos.html) before proceeding. If you don't already have NiFi, you can download it from [here](http://nifi.apache.org/download.html).
+
+## Configuring NiFi to Receive Data
+
+First, check that $NIFI_HOME/conf/nifi.properties has a Site to Site input socket port specified:
+```
+# Site to Site properties
+nifi.remote.input.host=
+nifi.remote.input.secure=false
+nifi.remote.input.socket.port=1026
+nifi.remote.input.http.enabled=true
+nifi.remote.input.http.transaction.ttl=30 sec
+```
+
+Restart NiFi so changes take effect.
+
+On NiFi's web UI, place an Input Port "From MiNiFi" and connect it to a downstream processor:
+
+
+## Creating the MiNiFi Flow
+
+Create a new process group, and double click on it to enter it. Working inside this new processor group will let us create a clean template that includes flow components meant to run in MiNiFi.
+
+Inside the process group, create a Remote Process Group (RPG) and give it the URL of your NiFi instance. The RPG will connect to NiFi and discover the input port created earlier.
+
+Create a TailFile processor and connect it to the RPG, selecting the "From MiNiFi" input port.
+
+
+Now create a template, download it as an XML file, and download the [Converter ToolKit](download.html).
+
+From the toolkit directory:
+```
+[user@host minifi-toolkit-0.0.1]# ./bin/config.sh transform /path/to/template.xml /output/path/config.yml
+
+Java home: /usr/lib/jvm/jre-1.8.0-openjdk
+MiNiFi Toolkit home: /minifi-toolkit-0.0.1
+
+
+No validation errors found in template.
+```
+
+Open the config.yml file and add Port and Host to a Properties section at the end of the Input Ports section (at the end of the file):
+```
+Remote Processing Groups:
+- name: http://localhost:8080/nifi
+ url: http://localhost:8080/nifi
+ comment: ''
+ timeout: 30 sec
+ yield period: 10 sec
+ Input Ports:
+ - id: AUTOGENERATED_NIFI_PORT_ID_HERE
+ name: MiNiFi-input
+ comment: ''
+ max concurrent tasks: 1
+ use compression: false
+ Properties: # Deviates from spec and will later be removed when this is autonegotiated
+ Port: 1026
+ Host Name: localhost
+```
+
+Copy the generated config.yml file into $MINIFI/conf/config.yml, and start MiNiFi:
+```
+[user@host nifi-minifi-cpp-0.1.0]# ./bin/minifi.sh start
+Starting MiNiFi with PID 463 and pid file /nifi-minifi-cpp-0.1.0/bin/.minifi.pid
+```
+
+After a few seconds, you should see NiFi receiving data on the input port:
+
+
+If you do not see data in the queue, it may have already been consumed by the downstream processor (PutFile in this case). If no data is arriving on the input port at all, check the MiNiFi log files, and confirm that the file being tailed exists and is not empty.