You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@twill.apache.org by ch...@apache.org on 2014/12/16 20:12:38 UTC
[1/2] incubator-twill git commit: (TWILL-100) Fix examples POMs and
update documentations - Updates to examples - Updates to BundleJarExample
application and documentation.
Repository: incubator-twill
Updated Branches:
refs/heads/master 73ecf8e3a -> b1adf69d6
(TWILL-100) Fix examples POMs and update documentations
- Updates to examples
- Updates to BundleJarExample application and documentation.
Signed-off-by: Terence Yim <ch...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/incubator-twill/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-twill/commit/b9e668c4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-twill/tree/b9e668c4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-twill/diff/b9e668c4
Branch: refs/heads/master
Commit: b9e668c41c9b89a8984d46a65bf6dfbfcb5e119e
Parents: 73ecf8e
Author: Matt Anderson <ma...@liaison.com>
Authored: Mon Dec 15 16:48:25 2014 -0500
Committer: Terence Yim <ch...@apache.org>
Committed: Tue Dec 16 10:41:33 2014 -0800
----------------------------------------------------------------------
.gitignore | 6 ++
src/site/markdown/GettingStarted.md | 67 +++++++++++++++++++-
twill-examples/echo/pom.xml | 6 ++
twill-examples/yarn/pom.xml | 8 ++-
.../twill/example/yarn/BundledJarExample.java | 4 +-
5 files changed, 85 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-twill/blob/b9e668c4/.gitignore
----------------------------------------------------------------------
diff --git a/.gitignore b/.gitignore
index bcb6216..87061cf 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,6 +16,11 @@ out/
lib/
.idea
+# Eclipse Files and Dir #
+.project
+.classpath
+.settings
+
# Gradle Files & Dir #
build/
.stickyStorage
@@ -26,3 +31,4 @@ target/
logs/
zookeeper.out
+/bin
http://git-wip-us.apache.org/repos/asf/incubator-twill/blob/b9e668c4/src/site/markdown/GettingStarted.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/GettingStarted.md b/src/site/markdown/GettingStarted.md
index c8384a3..85345ab 100644
--- a/src/site/markdown/GettingStarted.md
+++ b/src/site/markdown/GettingStarted.md
@@ -123,4 +123,69 @@ You can also stop the application through the `TwillController`:
controller.stop().get();
```
-This will shut down the application master and all the containers.
\ No newline at end of file
+This will shut down the application master and all the containers.
+
+### Hello World
+
+To see Twill in action, you can run the "hello world" example applications located in the twill-examples module.
+
+#### Prerequisites
+
+* Single Node or Cluster installation of Hadoop with YARN (Hadoop >= 2.2.0) set-up and running.
+* Single Node or Cluster installation of ZooKeeper set-up and running.
+* Build of Twill Library Code (minimum, build of twill-examples module)
+
+#### Running the Examples
+
+There are two example applications you can run: HelloWorld and BundledJarExample.
+
+##### HelloWorld Application
+
+The HelloWorld application creates a simple YARN application that prints a line to the log.
+
+You can run the HelloWorld application from any node of the Hadoop cluster using the below command
+(be sure to add your ZooKeeper Host and Port):
+
+```sh
+$ CP=twill-examples-yarn-0.5.0-incubating-SNAPSHOT.jar:`hadoop classpath`; java -cp $CP org.apache.twill.example.yarn.HelloWorld {zookeeper_host:port}
+```
+
+If successful, you should see logs output to the terminal with details of the running application. Once the application
+is finished running, check the YARN logs and you should see output like the following:
+
+```
+14:49:45.944 [TwillContainerService] INFO o.a.twill.example.yarn.HelloWorld - Hello World. My first distributed application.
+```
+
+##### BundledJarExample Application
+
+The BundledJarExample application demonstrates the Twill functionality that allows you to run any Java application
+in Twill without worrying about library version conflicts between your application and Hadoop. The example
+calls the main class in a sample application `Echo`, which simply logs the command line argument(s) passed to it.
+The `Echo` application uses a different version of Guava from Twill and Hadoop distributions. BundledJarExample
+looks for the dependency in a `lib` folder packaged at the root of the `Echo` jar.
+
+You can run the BundleJarExample application from any node of the Hadoop cluster using the below command
+(be sure to add your ZooKeeper Host and Port):
+
+```sh
+$ CP=twill-examples-yarn-0.5.0-incubating-SNAPSHOT.jar:`hadoop classpath`;
+ java -cp $CP org.apache.twill.example.yarn.BundledJarExample {zookeeper_host:port} twill-examples-echo-0.5.0-incubating-SNAPSHOT.jar echo.EchoMain arg1
+```
+
+Like with the HelloWorld example, you should see logs output to the terminal. Once the application is complete, check
+the YARN logs as before and you should see output like the following:
+
+```
+[TwillContainerService] INFO echo.EchoMain - Hello from EchoMain: 6
+err HELLO from scatch
+[TwillContainerService] INFO echo.EchoMain - Got args: [arg1]
+
+...
+
+out HELLO from scatch
+Got args: [arg1]
+
+```
+
+Congratulations! You have run your first Twill applications.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-twill/blob/b9e668c4/twill-examples/echo/pom.xml
----------------------------------------------------------------------
diff --git a/twill-examples/echo/pom.xml b/twill-examples/echo/pom.xml
index f05cef7..cd6376a 100644
--- a/twill-examples/echo/pom.xml
+++ b/twill-examples/echo/pom.xml
@@ -46,6 +46,12 @@ limitations under the License.
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
+ <version>1.7.7</version>
+ </dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-simple</artifactId>
+ <version>1.7.7</version>
</dependency>
</dependencies>
http://git-wip-us.apache.org/repos/asf/incubator-twill/blob/b9e668c4/twill-examples/yarn/pom.xml
----------------------------------------------------------------------
diff --git a/twill-examples/yarn/pom.xml b/twill-examples/yarn/pom.xml
index 19c04a7..7c3e344 100644
--- a/twill-examples/yarn/pom.xml
+++ b/twill-examples/yarn/pom.xml
@@ -30,24 +30,26 @@ limitations under the License.
<name>Apache Twill examples: YARN</name>
<description>Examples demonstrating usage of twill-yarn</description>
<artifactId>twill-examples-yarn</artifactId>
+
+ <properties>
+ <guava.version>13.0.1</guava.version>
+ </properties>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>twill-yarn</artifactId>
<version>${project.version}</version>
- <scope>provided</scope>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>twill-ext</artifactId>
<version>${project.version}</version>
- <scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
- <scope>provided</scope>
+ <version>${guava.version}</version>
</dependency>
</dependencies>
http://git-wip-us.apache.org/repos/asf/incubator-twill/blob/b9e668c4/twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java
----------------------------------------------------------------------
diff --git a/twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java b/twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java
index 4a5cbf4..cec5bdf 100644
--- a/twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java
+++ b/twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java
@@ -79,8 +79,8 @@ public class BundledJarExample {
}
String zkStr = args[0];
- BundledJarRunner.Arguments arguments = BundledJarRunner.Arguments.fromArray(
- Arrays.copyOfRange(args, 1, args.length));
+ BundledJarRunner.Arguments arguments = new BundledJarRunner.Arguments(
+ args[1], "/lib", args[2], Arrays.copyOfRange(args, 3, args.length));
File jarFile = new File(arguments.getJarFileName());
Preconditions.checkState(jarFile.exists());
[2/2] incubator-twill git commit: Temporarily disable Travis-CI
hadoop-2.3 profile as it always fail due to hostname issue. Looks like it is
a known issue (https://github.com/travis-ci/travis-ci/issues/2970)
Posted by ch...@apache.org.
Temporarily disable Travis-CI hadoop-2.3 profile as it always fail due to hostname issue. Looks like it is a known issue (https://github.com/travis-ci/travis-ci/issues/2970)
Project: http://git-wip-us.apache.org/repos/asf/incubator-twill/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-twill/commit/b1adf69d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-twill/tree/b1adf69d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-twill/diff/b1adf69d
Branch: refs/heads/master
Commit: b1adf69d69b1922bc9af900ecdb8d12319d233af
Parents: b9e668c
Author: Terence Yim <ch...@apache.org>
Authored: Tue Dec 16 11:13:20 2014 -0800
Committer: Terence Yim <ch...@apache.org>
Committed: Tue Dec 16 11:13:20 2014 -0800
----------------------------------------------------------------------
.travis.yml | 1 -
1 file changed, 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-twill/blob/b1adf69d/.travis.yml
----------------------------------------------------------------------
diff --git a/.travis.yml b/.travis.yml
index e051942..b8a16bf 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -35,7 +35,6 @@ env:
- PROFILE='hadoop-2.0'
- PROFILE='hadoop-2.1'
- PROFILE='hadoop-2.2'
- - PROFILE='hadoop-2.3'
- PROFILE='hadoop-2.4'
- PROFILE='hadoop-2.5'
- PROFILE='cdh-4.4.0'