You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@twill.apache.org by Krishnachaitanya C Potluri <Kr...@Progressive.com> on 2016/12/23 18:24:24 UTC
Twill example issue
Hello All,
We could really use Twill if there is some good working examples out there. I am trying to run hello world and I am seeing this issue. Can someone tell me where is twill looking for the jar and how can I tell it look in a certain folder?
java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.FileNotFoundException: /twillsample-0.0.1-SNAPSHOT.one-jar.jar (No such file or directory)
at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:281)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:132)
at org.apache.twill.internal.AbstractExecutionServiceController.awaitTerminated(AbstractExecutionServiceController.java:117)
at com.progressive.twillsample.HelloWorld.main(HelloWorld.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /twillsample-0.0.1-SNAPSHOT.one-jar.jar (No such file or directory)
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at org.apache.twill.internal.ApplicationBundler.saveEntry(ApplicationBundler.java:309)
at org.apache.twill.internal.ApplicationBundler.putEntry(ApplicationBundler.java:232)
at org.apache.twill.internal.ApplicationBundler.access$200(ApplicationBundler.java:57)
at org.apache.twill.internal.ApplicationBundler$3.accept(ApplicationBundler.java:208)
at org.apache.twill.internal.utils.Dependencies.findClassDependencies(Dependencies.java:80)
Command to run this : /opt/share/oracle/jvm/jdk1.8.0_92/bin/java -jar twillsample-0.0.1-SNAPSHOT.one-jar.jar zk:2181
Code is used from twill examples:
public class HelloWorld {
public static final Logger LOG = LoggerFactory.getLogger(HelloWorld.class);
/**
* Hello World runnable that is provided to TwillRunnerService to be run.
*/
private static class HelloWorldRunnable extends AbstractTwillRunnable {
@Override
public void run() {
LOG.info("Hello World. My first distributed application.");
}
@Override
public void stop() {
}
}
public static void main(String[] args) {
if (args.length < 1) {
System.err.println("Arguments format: <host:port of zookeeper server>");
System.exit(1);
}
String zkStr = args[0];
YarnConfiguration yarnConfiguration = new YarnConfiguration(getConf());
final TwillRunnerService twillRunner = new YarnTwillRunnerService(yarnConfiguration, zkStr);
twillRunner.start();
LOG.info("Twill Runner Started");
String yarnClasspath =
yarnConfiguration.get(YarnConfiguration.YARN_APPLICATION_CLASSPATH,
Joiner.on(",").join(YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH));
List<String> applicationClassPaths = Lists.newArrayList();
Iterables.addAll(applicationClassPaths, Splitter.on(",").split(yarnClasspath));
final TwillController controller = twillRunner.prepare(new HelloWorldRunnable())
.addLogHandler(new PrinterLogHandler(new PrintWriter(System.out, true)))
.withApplicationClassPaths(applicationClassPaths)
.withBundlerClassAcceptor(new HadoopClassExcluder())
.start();
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
try {
Futures.getUnchecked(controller.terminate());
} finally {
twillRunner.stop();
}
}
});
try {
controller.awaitTerminated();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
private static Configuration getConf() {
Configuration conf = new Configuration();
conf.set(MRConfig.FRAMEWORK_NAME, MRConfig.YARN_FRAMEWORK_NAME);
conf.addResource(new Path("file:///etc/hadoop/conf/yarn-site.xml"));
conf.addResource(new Path("file:///etc/hadoop/conf/core-site.xml"));
conf.addResource(new Path("file:///etc/hadoop/conf/hdfs-site.xml"));
conf.addResource(new Path("file:///etc/hadoop/conf/mapred-site.xml"));
return conf;
}
static class HadoopClassExcluder extends ClassAcceptor {
@Override
public boolean accept(String className, URL classUrl, URL classPathUrl) {
// exclude hadoop but not hbase package
return !(className.startsWith("org.apache.hadoop") && !className.startsWith("org.apache.hadoop.hbase"));
}
}
}
Thanks,
Krishna
Re: Twill example issue
Posted by Terence Yim <ch...@gmail.com>.
Hi,
You'll need twill classes (and it's dependencies) in the classpath. Would
you mind letting me know what's the content of your "
twillsample-0.0.1-SNAPSHOT.one-jar.jar" file? Does it contain twill and
hadoop classes as well?
Terence
On Fri, Dec 23, 2016 at 10:24 AM, Krishnachaitanya C Potluri <
Krishnachaitanya_C_Potluri@progressive.com> wrote:
> Hello All,
>
> We could really use Twill if there is some good working examples out
> there. I am trying to run hello world and I am seeing this issue. Can
> someone tell me where is twill looking for the jar and how can I tell it
> look in a certain folder?
>
> java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> java.io.FileNotFoundException: /twillsample-0.0.1-SNAPSHOT.one-jar.jar
> (No such file or directory)
> at com.google.common.util.concurrent.AbstractFuture$
> Sync.getValue(AbstractFuture.java:294)
> at com.google.common.util.concurrent.AbstractFuture$
> Sync.get(AbstractFuture.java:281)
> at com.google.common.util.concurrent.AbstractFuture.get(
> AbstractFuture.java:116)
> at com.google.common.util.concurrent.Uninterruptibles.
> getUninterruptibly(Uninterruptibles.java:132)
> at org.apache.twill.internal.AbstractExecutionServiceContro
> ller.awaitTerminated(AbstractExecutionServiceController.java:117)
> at com.progressive.twillsample.HelloWorld.main(HelloWorld.java:83)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.simontuffs.onejar.Boot.run(Boot.java:340)
> at com.simontuffs.onejar.Boot.main(Boot.java:166)
> Caused by: java.lang.RuntimeException: java.io.FileNotFoundException:
> /twillsample-0.0.1-SNAPSHOT.one-jar.jar (No such file or directory)
> at com.google.common.base.Throwables.propagate(
> Throwables.java:160)
> at org.apache.twill.internal.ApplicationBundler.saveEntry(
> ApplicationBundler.java:309)
> at org.apache.twill.internal.ApplicationBundler.putEntry(
> ApplicationBundler.java:232)
> at org.apache.twill.internal.ApplicationBundler.access$200(
> ApplicationBundler.java:57)
> at org.apache.twill.internal.ApplicationBundler$3.accept(
> ApplicationBundler.java:208)
> at org.apache.twill.internal.utils.Dependencies.
> findClassDependencies(Dependencies.java:80)
>
>
> Command to run this : /opt/share/oracle/jvm/jdk1.8.0_92/bin/java -jar
> twillsample-0.0.1-SNAPSHOT.one-jar.jar zk:2181
>
> Code is used from twill examples:
>
> public class HelloWorld {
> public static final Logger LOG = LoggerFactory.getLogger(
> HelloWorld.class);
>
> /**
> * Hello World runnable that is provided to TwillRunnerService to
> be run.
> */
> private static class HelloWorldRunnable extends
> AbstractTwillRunnable {
> @Override
> public void run() {
> LOG.info("Hello World. My first distributed application.");
> }
>
> @Override
> public void stop() {
> }
> }
>
> public static void main(String[] args) {
> if (args.length < 1) {
> System.err.println("Arguments format: <host:port of zookeeper
> server>");
> System.exit(1);
> }
>
> String zkStr = args[0];
> YarnConfiguration yarnConfiguration = new
> YarnConfiguration(getConf());
> final TwillRunnerService twillRunner = new
> YarnTwillRunnerService(yarnConfiguration, zkStr);
> twillRunner.start();
> LOG.info("Twill Runner Started");
> String yarnClasspath =
> yarnConfiguration.get(YarnConfiguration.YARN_
> APPLICATION_CLASSPATH,
> Joiner.on(",").join(
> YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH));
> List<String> applicationClassPaths = Lists.newArrayList();
> Iterables.addAll(applicationClassPaths, Splitter.on(",").split(
> yarnClasspath));
>
> final TwillController controller = twillRunner.prepare(new
> HelloWorldRunnable())
> .addLogHandler(new PrinterLogHandler(new
> PrintWriter(System.out, true)))
> .withApplicationClassPaths(applicationClassPaths)
> .withBundlerClassAcceptor(new HadoopClassExcluder())
> .start();
>
> Runtime.getRuntime().addShutdownHook(new Thread() {
> @Override
> public void run() {
> try {
> Futures.getUnchecked(controller.terminate());
> } finally {
> twillRunner.stop();
> }
> }
> });
>
> try {
> controller.awaitTerminated();
> } catch (ExecutionException e) {
> e.printStackTrace();
> }
> }
>
> private static Configuration getConf() {
> Configuration conf = new Configuration();
> conf.set(MRConfig.FRAMEWORK_NAME,
> MRConfig.YARN_FRAMEWORK_NAME);
> conf.addResource(new Path("file:///etc/hadoop/conf/
> yarn-site.xml"));
> conf.addResource(new Path("file:///etc/hadoop/conf/
> core-site.xml"));
> conf.addResource(new Path("file:///etc/hadoop/conf/
> hdfs-site.xml"));
> conf.addResource(new Path("file:///etc/hadoop/conf/
> mapred-site.xml"));
> return conf;
> }
>
> static class HadoopClassExcluder extends ClassAcceptor {
> @Override
> public boolean accept(String className, URL classUrl, URL
> classPathUrl) {
> // exclude hadoop but not hbase package
> return !(className.startsWith("org.apache.hadoop") &&
> !className.startsWith("org.apache.hadoop.hbase"));
> }
> }
>
> }
>
> Thanks,
> Krishna
>