You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@streams.apache.org by sb...@apache.org on 2016/04/25 18:07:08 UTC
[11/11] incubator-streams-master git commit: Add tutorial
Add tutorial
Project: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/commit/283d0fc4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/tree/283d0fc4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/diff/283d0fc4
Branch: refs/heads/newwebpage
Commit: 283d0fc46896a64dafd31879173ac1f81f32a8df
Parents: 749300a
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Authored: Mon Apr 25 11:04:59 2016 -0500
Committer: Steve Blackmon @steveblackmon <sb...@apache.org>
Committed: Mon Apr 25 11:04:59 2016 -0500
----------------------------------------------------------------------
src/site/markdown/tutorial/build.md | 81 ++++++++++++++++++++++++++
src/site/markdown/tutorial/database.md | 44 ++++++++++++++
src/site/markdown/tutorial/docker.md | 18 ++++++
src/site/markdown/tutorial/environment.md | 28 +++++++++
src/site/site.xml | 37 +++++++-----
src/site/site_en.xml | 81 ++++++++++++++------------
6 files changed, 235 insertions(+), 54 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/markdown/tutorial/build.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/tutorial/build.md b/src/site/markdown/tutorial/build.md
new file mode 100644
index 0000000..1608b28
--- /dev/null
+++ b/src/site/markdown/tutorial/build.md
@@ -0,0 +1,81 @@
+## Build streams and streams-examples from source
+
+This tutorial assumes you are using linux or Mac OS X.
+
+### Setup Tools
+
+You'll need the following tools installed in your command line:
+
+* Git
+* Java SDK
+* Maven
+* Docker
+
+#### Git
+
+ `git -version`
+
+| Possible result | Explanation |
+|-----------------|-------------|
+| bash: git: No such file or directory | You need to install git |
+| git version < 2.7 | You should upgrade git for security reasons |
+| git version > 2.7 | You are all good |
+
+#### Maven and Java SDK
+
+Run from your command line:
+
+ `mvn -version`
+
+| Possible result | Explanation |
+|-----------------|-------------|
+| -bash: mvn: command not found | You need to install maven |
+| Error: JAVA_HOME is not defined correctly. | You need to install JDK |
+| Apache Maven >= 3.2.5+\nJava Version >= 1.7.0u72) | You're all good |
+| Apache Maven >= 3.2.5+\nJava Version >= 1.8.0u25) | You're all good |
+| Apache Maven < 3.2.5 | You need a newer version of maven |
+| Java Version < 1.7.0u72 | You need a newer version of maven |
+| Java Version < 1.8.0u25 | You need a newer JDK |
+
+#### Docker
+
+Run from your command line:
+
+ `docker version`
+
+| Possible result | Explanation |
+|-----------------|-------------|
+| bash: docker: No such file or directory | You need to install docker |
+| Client: Version: < 1.0.0 | You need a newer version of docker |
+| Server: Version: < 1.0.0 | You need a newer version of docker |
+| Client: Version: > 1.0.0\nServer: Version: > 1.0.0 | You are all good |
+
+See [streams-project-index.html](http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html "streams-project/index.html") for more information.
+
+### Download Sources
+
+Run from your command line:
+
+ `git clone https://github.com/apache/incubator-streams`
+ `git clone https://github.com/apache/incubator-streams-examples`
+
+### Build Projects
+
+Run from your command line:
+
+ `export MAVEN_OPTS="-Xmx2G"`
+ `cd incubator-streams`
+ `mvn clean install -Dmaven.test.skip.exec=true`
+
+| Possible result | Explanation |
+|-----------------|-------------|
+| BUILD SUCCESSFUL | You are all good |
+| BUILD FAILED | Check yourself |
+
+ `cd ../incubator-streams-examples`
+ `mvn clean package`
+
+| Possible result | Explanation |
+|-----------------|-------------|
+| BUILD SUCCESSFUL | You are all good |
+| BUILD FAILED | Check yourself |
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/markdown/tutorial/database.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/tutorial/database.md b/src/site/markdown/tutorial/database.md
new file mode 100644
index 0000000..b546826
--- /dev/null
+++ b/src/site/markdown/tutorial/database.md
@@ -0,0 +1,44 @@
+## Set up databases to store and analyze streams content
+
+This tutorial assumes you are using linux or Mac OS X.
+
+### Confirm Docker is healthy
+
+We'll assume you've got docker up and running.
+
+Run from your command line:
+
+ `docker ps`
+
+If you see a (possibly empty) list of running containers, you are good.
+
+### Run Elasticsearch
+
+Elasticsearch is a great database for storing content from your streams.
+
+ `docker run -d --name elasticsearch elasticsearch`
+
+### Add Elasticsearch container details to your configuration
+
+ `cd $STREAMS`
+ `export DOCKERHOST=$(docker-machine ip)`
+
+Put the following into elasticsearch.conf
+
+ include "reference.conf"
+ elasticsearch {
+ hosts = [
+ ${DOCKERHOST}
+ ]
+ protocol = "tcp"
+ port = 9300
+ index = "streams"
+ indexes = [
+ "streams"
+ ]
+ types = [
+ "page"
+ "post"
+ ]
+ }
+
http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/markdown/tutorial/docker.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/tutorial/docker.md b/src/site/markdown/tutorial/docker.md
new file mode 100644
index 0000000..fa5ffec
--- /dev/null
+++ b/src/site/markdown/tutorial/docker.md
@@ -0,0 +1,18 @@
+## Set up a local environment to run streams
+
+First of all, you do not run 'Streams' as software. Rather, you run 'streams' which use Streams components and libraries under the cover.
+
+Streams components can be embedded in a variety of data processing frameworks based on the problem and performance requirements at hand.
+
+A great way to get started collecting and indexing data is with streams-runtime-local and Docker.
+
+### Confirm Docker is healthy
+
+We'll assume you've got docker up and running.
+
+Run from your command line:
+
+ `docker ps`
+
+If you see a (possibly empty) list of running containers, you are good.
+
http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/markdown/tutorial/environment.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/tutorial/environment.md b/src/site/markdown/tutorial/environment.md
new file mode 100644
index 0000000..b86dd7e
--- /dev/null
+++ b/src/site/markdown/tutorial/environment.md
@@ -0,0 +1,28 @@
+### Create a folder to hold streams configuration files
+
+Pick a place on your file system that's easy to remember and reference with an absolution path.
+
+Create a new empty folder there and export the path to a variables.
+
+ `mkdir ~/streams`
+ `export STREAMS=$(cd ~streams; pwd)`
+
+### Create a configuration file to hold secrets
+
+ `cd $STREAMS`
+ `touch reference.conf`
+
+To get started, put the following into reference.conf
+
+ twitter {
+ oauth {
+ consumerKey = ""
+ consumerSecret = ""
+ accessToken = ""
+ accessTokenSecret = ""
+ }
+ }
+
+Visit developer.twitter.com to obtain the above fields and put them your reference.conf
+
+
http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/site.xml
----------------------------------------------------------------------
diff --git a/src/site/site.xml b/src/site/site.xml
index e06e9f2..4087ae7 100644
--- a/src/site/site.xml
+++ b/src/site/site.xml
@@ -75,26 +75,31 @@
<item name="streams-examples" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/" />
</menu>
<menu name="Getting Started">
- <item name="Learn more about Activity Streams" href="http://activitystrea.ms" />
- <item name="Check out streams-project web site" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/" />
- <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-pojo/index.html" />
- <item name="Set up a local environment to run streams" />
- <item name="Set up a local database to store streams data" />
- <item name="Build and run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html" />
- <item name="Browse and search the twitter data you collected" />
- <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/streams-provider-twitter/index.html" />
- <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/index.html" />
- <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-components/index.html" />
- <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/index.html" />
+ <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-pojo/index.html" />
+ <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/streams-provider-twitter/index.html" />
+ <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
+ <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-components/index.html" />
+ <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
+ <item name="Browse streams providers, processors, and persisters" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
</menu>
- <menu name="Foundation">
- <item name="Foundation Info" href="http://www.apache.org/" />
- <item name="Sponsorship" href="http://www.apache.org/foundation/sponsorship.html" />
- <item name="Thanks" href="http://www.apache.org/foundation/thanks.html" />
+ <menu name="Tutorial">
+ <item name="Build streams and streams-examples from source" href="tutorial/build.html" />
+ <item name="Create a local folder to configure streams" href="tutorial/environment.html"/>
+ <item name="Set up a local docker environment" href="tutorial/docker.html" />
+ <item name="Run a local database to store activity documents" href="tutorial/database.html" />
+ <item name="Run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html" />
+ <item name="Browse and search the twitter data you collected" href="tutorial/browse.html" />
</menu>
<menu name="Other Resources">
- <item name="Streams Proposal Wiki Page" href="http://wiki.apache.org/incubator/StreamsProposal" />
+ <item name="ActivityStreams Homepage" href="http://activitystrea.ms" />
<item name="ActivityStreams Specifications" href="http://activitystrea.ms/specs/" />
+ <item name="Streams Blog" href="http://blogs.apache.org/streams/" />
+ <item name="Streams Proposal Wiki Page" href="http://wiki.apache.org/incubator/StreamsProposal" />
+ </menu>
+ <menu name="Foundation">
+ <item name="Foundation Info" href="http://www.apache.org/" />
+ <item name="Sponsorship" href="http://www.apache.org/foundation/sponsorship.html" />
+ <item name="Thanks" href="http://www.apache.org/foundation/thanks.html" />
</menu>
</body>
</project>
http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/site_en.xml
----------------------------------------------------------------------
diff --git a/src/site/site_en.xml b/src/site/site_en.xml
index e06e9f2..effd07e 100644
--- a/src/site/site_en.xml
+++ b/src/site/site_en.xml
@@ -25,11 +25,11 @@
<version>1.4</version>
</skin>
<custom>
- <fluidoSkin>
- <topBarEnabled>true</topBarEnabled>
- <navBarStyle>navbar-inverse</navBarStyle>
- <sideBarEnabled>false</sideBarEnabled>
- </fluidoSkin>
+ <fluidoSkin>
+ <topBarEnabled>true</topBarEnabled>
+ <navBarStyle>navbar-inverse</navBarStyle>
+ <sideBarEnabled>false</sideBarEnabled>
+ </fluidoSkin>
</custom>
<bannerLeft>
<name>Apache Streams</name>
@@ -45,8 +45,8 @@
</bannerRight>
<body>
<breadcrumbs>
- <item name="Incubator" href="http://incubator.apache.org/"/>
- <item name="Streams" href="http://streams.incubator.apache.org/"/>
+ <item name="Incubator" href="http://incubator.apache.org/"/>
+ <item name="Streams" href="http://streams.incubator.apache.org/"/>
</breadcrumbs>
<menu ref="parent" inherit="top"/>
<menu name="Overview">
@@ -57,44 +57,49 @@
<item name="Frequently Asked Questions" href="faq.html" />
</menu>
<menu name="Details">
- <item name="Project License" href="license.html" />
- <item name="Mailing Lists" href="mail-lists.html" />
- <item name="Project Team" href="team-list.html" />
- <item name="Continuous Integration" href="integration.html"/>
- <item name="Issue Tracking" href="issue-tracking.html" />
- <item name="Source Repositories" href="source-repository.html" />
- <item name="Dependency Info" href="dependency-info.html" />
- <item name="Distribution Management" href="distribution-management.html" />
- <item name="Plugin Management" href="plugin-management.html" />
- <item name="Release Setup" href="release-setup.html" />
- <item name="Release Process" href="release.html" />
+ <item name="Project License" href="license.html" />
+ <item name="Mailing Lists" href="mail-lists.html" />
+ <item name="Project Team" href="team-list.html" />
+ <item name="Continuous Integration" href="integration.html"/>
+ <item name="Issue Tracking" href="issue-tracking.html" />
+ <item name="Source Repositories" href="source-repository.html" />
+ <item name="Dependency Info" href="dependency-info.html" />
+ <item name="Distribution Management" href="distribution-management.html" />
+ <item name="Plugin Management" href="plugin-management.html" />
+ <item name="Release Setup" href="release-setup.html" />
+ <item name="Release Process" href="release.html" />
</menu>
<menu name="Projects">
- <item name="streams-master" />
- <item name="streams-project" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project" />
- <item name="streams-examples" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/" />
+ <item name="streams-master" />
+ <item name="streams-project" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project" />
+ <item name="streams-examples" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/" />
</menu>
<menu name="Getting Started">
- <item name="Learn more about Activity Streams" href="http://activitystrea.ms" />
- <item name="Check out streams-project web site" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/" />
- <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-pojo/index.html" />
- <item name="Set up a local environment to run streams" />
- <item name="Set up a local database to store streams data" />
- <item name="Build and run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html" />
- <item name="Browse and search the twitter data you collected" />
- <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/streams-provider-twitter/index.html" />
- <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/index.html" />
- <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-components/index.html" />
- <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/index.html" />
+ <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-pojo/index.html" />
+ <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/streams-provider-twitter/index.html" />
+ <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
+ <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-components/index.html" />
+ <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
+ <item name="Browse streams providers, processors, and persisters" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
</menu>
- <menu name="Foundation">
- <item name="Foundation Info" href="http://www.apache.org/" />
- <item name="Sponsorship" href="http://www.apache.org/foundation/sponsorship.html" />
- <item name="Thanks" href="http://www.apache.org/foundation/thanks.html" />
+ <menu name="Tutorial">
+ <item name="Build streams and streams-examples from source" href="tutorial/build.html" />
+ <item name="Create a local folder to configure streams" href="tutorial/environment.html"/>
+ <item name="Set up a local docker environment" href="tutorial/docker.html" />
+ <item name="Run a local database to store activity documents" href="tutorial/database.html" />
+ <item name="Run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html" />
+ <item name="Browse and search the twitter data you collected" href="tutorial/browse.html" />
</menu>
<menu name="Other Resources">
- <item name="Streams Proposal Wiki Page" href="http://wiki.apache.org/incubator/StreamsProposal" />
- <item name="ActivityStreams Specifications" href="http://activitystrea.ms/specs/" />
+ <item name="ActivityStreams Homepage" href="http://activitystrea.ms" />
+ <item name="ActivityStreams Specifications" href="http://activitystrea.ms/specs/" />
+ <item name="Streams Blog" href="http://blogs.apache.org/streams/" />
+ <item name="Streams Proposal Wiki Page" href="http://wiki.apache.org/incubator/StreamsProposal" />
+ </menu>
+ <menu name="Foundation">
+ <item name="Foundation Info" href="http://www.apache.org/" />
+ <item name="Sponsorship" href="http://www.apache.org/foundation/sponsorship.html" />
+ <item name="Thanks" href="http://www.apache.org/foundation/thanks.html" />
</menu>
</body>
</project>