You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@twill.apache.org by "Meier, Caleb" <Ca...@parsons.com> on 2017/02/06 22:51:41 UTC
Zookeeper Issue when Creating a Simple Twill Application
Hello,
I’m trying to create my own simple HelloWorld in Twill. The run method for the runnable is the following:
@Override
public void run() {
System.err.println("Starting No-Op notification task: " + new Date());
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.err.println("Ending No-Op notification task: " + new Date());
}
The client has the following main:
public static void main(String[] args) {
String zkStr = args[0];
YarnConfiguration yarnConfiguration = new YarnConfiguration();
TwillRunnerService runner = new YarnTwillRunnerService(yarnConfiguration, zkStr);
runner.start();
TwillController controller = runner.prepare(new NotificationRunnable())
.addLogHandler(new PrinterLogHandler(new PrintWriter(System.out)))
.start();
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
try {
Futures.getUnchecked(controller.terminate());
} finally {
runner.stop();
}
}
});
try {
controller.awaitTerminated();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
I’m running the application from a shaded jar on my cluster (I’ve tried running it using both the BundledJarRunner and directly). The application is getting accepted by Yarn and appears to run, but I’m not seeing my output and I’m seeing the following stacktrace in the logs for the container that my runnable is running in. Any suggestions?
Exception in thread "CompositeService STOPPING" java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:267)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96)
at org.apache.twill.internal.appmaster.ApplicationMasterMain$AppMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:57)
... 1 more
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.twill.internal.zookeeper.DefaultZKClientService$Callbacks$4.processResult(DefaultZKClientService.java:619)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:89)
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1015)
at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001)
at com.google.common.util.concurrent.AbstractService.stopAndWait(AbstractService.java:225)
at com.google.common.util.concurrent.AbstractIdleService.stopAndWait(AbstractIdleService.java:122)
at org.apache.twill.internal.ServiceMain.doMain(ServiceMain.java:108)
at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:85)
... 5 more
Caused by: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:267)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96)
at org.apache.twill.internal.appmaster.ApplicationMasterMain$AppMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:57)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.twill.internal.zookeeper.DefaultZKClientService$Callbacks$4.processResult(DefaultZKClientService.java:619)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
Caleb A. Meier, Ph.D.
Software Engineer II ♦ Analyst
Parsons Corporation
1911 N. Fort Myer Drive, Suite 800 ♦ Arlington, VA 22209
Office: (703)797-3066
Caleb.Meier@Parsons.com<ma...@Parsons.com> ♦ www.parsons.com<https://webportal.parsons.com/,DanaInfo=www.parsons.com+>
Re: Zookeeper Issue when Creating a Simple Twill Application
Posted by Terence Yim <ch...@gmail.com>.
Hi,
Can you try to replace the "System.err.println" with "Logger" from slf4j?
Twill only collect logs emitted via the slf4j API.
Terence
On Mon, Feb 6, 2017 at 3:18 PM, Martin Serrano <ma...@attivio.com> wrote:
> Caleb,
>
> If you want to see your output, you should be using a logger within your
> runnable instead of the System.err.println. The zookeeper ERROR you see on
> exit is happening after your application has shutdown I believe. I have
> seen this error myself after the application stops.
>
> -Martin
>
>
> On 02/06/2017 05:51 PM, Meier, Caleb wrote:
>
>> Hello,
>>
>> I’m trying to create my own simple HelloWorld in Twill. The run method
>> for the runnable is the following:
>>
>> @Override
>> public void run() {
>> System.err.println("Starting No-Op notification
>> task: " + new Date());
>> try {
>> Thread.sleep(5000);
>> } catch (InterruptedException e) {
>> e.printStackTrace();
>> }
>> System.err.println("Ending No-Op notification
>> task: " + new Date());
>> }
>>
>> The client has the following main:
>>
>> public static void main(String[] args) {
>>
>> String zkStr = args[0];
>> YarnConfiguration yarnConfiguration = new
>> YarnConfiguration();
>> TwillRunnerService runner = new
>> YarnTwillRunnerService(yarnConfiguration, zkStr);
>> runner.start();
>>
>> TwillController controller = runner.prepare(new
>> NotificationRunnable())
>> .addLogHandler(new
>> PrinterLogHandler(new PrintWriter(System.out)))
>> .start();
>>
>> Runtime.getRuntime().addShutdownHook(new
>> Thread() {
>> @Override
>> public void run() {
>> try {
>>
>> Futures.getUnchecked(controller.terminate());
>> } finally {
>>
>> runner.stop();
>> }
>> }
>> });
>>
>> try {
>> controller.awaitTerminated();
>> } catch (ExecutionException e) {
>> e.printStackTrace();
>> }
>> }
>>
>> I’m running the application from a shaded jar on my cluster (I’ve tried
>> running it using both the BundledJarRunner and directly). The application
>> is getting accepted by Yarn and appears to run, but I’m not seeing my
>> output and I’m seeing the following stacktrace in the logs for the
>> container that my runnable is running in. Any suggestions?
>>
>>
>> Exception in thread "CompositeService STOPPING"
>> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>> NoNode for /TimerRunnable/discoverable
>> at com.google.common.base.Throwables.propagate(Throwables.java:
>> 160)
>> at com.google.common.util.concurrent.AbstractIdleService$1$2.
>> run(AbstractIdleService.java:61)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.util.concurrent.ExecutionException:
>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>> NoNode for /TimerRunnable/discoverable
>> at com.google.common.util.concurrent.AbstractFuture$Sync.
>> getValue(AbstractFuture.java:294)
>> at com.google.common.util.concurrent.AbstractFuture$Sync.get(
>> AbstractFuture.java:267)
>> at com.google.common.util.concurrent.AbstractFuture.get(Abstrac
>> tFuture.java:96)
>> at org.apache.twill.internal.appmaster.ApplicationMasterMain$Ap
>> pMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
>> at com.google.common.util.concurrent.AbstractIdleService$1$2.
>> run(AbstractIdleService.java:57)
>> ... 1 more
>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>> at org.apache.zookeeper.KeeperException.create(KeeperException.
>> java:111)
>> at org.apache.zookeeper.KeeperException.create(KeeperException.
>> java:51)
>> at org.apache.twill.internal.zookeeper.DefaultZKClientService$C
>> allbacks$4.processResult(DefaultZKClientService.java:619)
>> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(
>> ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.
>> java:495)
>> Exception in thread "main" java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:497)
>> at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.
>> java:89)
>> Caused by: com.google.common.util.concurrent.UncheckedExecutionException:
>> java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>> at com.google.common.util.concurrent.Futures.wrapAndThrowUnchec
>> ked(Futures.java:1015)
>> at com.google.common.util.concurrent.Futures.getUnchecked(
>> Futures.java:1001)
>> at com.google.common.util.concurrent.AbstractService.stopAndWai
>> t(AbstractService.java:225)
>> at com.google.common.util.concurrent.AbstractIdleService.stopAn
>> dWait(AbstractIdleService.java:122)
>> at org.apache.twill.internal.ServiceMain.doMain(ServiceMain.
>> java:108)
>> at org.apache.twill.internal.appmaster.ApplicationMasterMain.
>> main(ApplicationMasterMain.java:85)
>> ... 5 more
>> Caused by: java.util.concurrent.ExecutionException:
>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>> NoNode for /TimerRunnable/discoverable
>> at com.google.common.util.concurrent.AbstractFuture$Sync.
>> getValue(AbstractFuture.java:294)
>> at com.google.common.util.concurrent.AbstractFuture$Sync.get(
>> AbstractFuture.java:267)
>> at com.google.common.util.concurrent.AbstractFuture.get(Abstrac
>> tFuture.java:96)
>> at org.apache.twill.internal.appmaster.ApplicationMasterMain$Ap
>> pMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
>> at com.google.common.util.concurrent.AbstractIdleService$1$2.
>> run(AbstractIdleService.java:57)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>> at org.apache.zookeeper.KeeperException.create(KeeperException.
>> java:111)
>> at org.apache.zookeeper.KeeperException.create(KeeperException.
>> java:51)
>> at org.apache.twill.internal.zookeeper.DefaultZKClientService$C
>> allbacks$4.processResult(DefaultZKClientService.java:619)
>> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(
>> ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.
>> java:495)
>>
>>
>> Caleb A. Meier, Ph.D.
>> Software Engineer II ♦ Analyst
>> Parsons Corporation
>> 1911 N. Fort Myer Drive, Suite 800 ♦ Arlington, VA 22209
>> Office: (703)797-3066
>> Caleb.Meier@Parsons.com<ma...@Parsons.com> ♦ www.parsons.com
>> <https://webportal.parsons.com/,DanaInfo=www.parsons.com+>
>>
>>
>
Re: Zookeeper Issue when Creating a Simple Twill Application
Posted by Martin Serrano <ma...@attivio.com>.
Caleb,
If you want to see your output, you should be using a logger within your
runnable instead of the System.err.println. The zookeeper ERROR you see
on exit is happening after your application has shutdown I believe. I
have seen this error myself after the application stops.
-Martin
On 02/06/2017 05:51 PM, Meier, Caleb wrote:
> Hello,
>
> I\u2019m trying to create my own simple HelloWorld in Twill. The run method for the runnable is the following:
>
> @Override
> public void run() {
> System.err.println("Starting No-Op notification task: " + new Date());
> try {
> Thread.sleep(5000);
> } catch (InterruptedException e) {
> e.printStackTrace();
> }
> System.err.println("Ending No-Op notification task: " + new Date());
> }
>
> The client has the following main:
>
> public static void main(String[] args) {
>
> String zkStr = args[0];
> YarnConfiguration yarnConfiguration = new YarnConfiguration();
> TwillRunnerService runner = new YarnTwillRunnerService(yarnConfiguration, zkStr);
> runner.start();
>
> TwillController controller = runner.prepare(new NotificationRunnable())
> .addLogHandler(new PrinterLogHandler(new PrintWriter(System.out)))
> .start();
>
> Runtime.getRuntime().addShutdownHook(new Thread() {
> @Override
> public void run() {
> try {
> Futures.getUnchecked(controller.terminate());
> } finally {
> runner.stop();
> }
> }
> });
>
> try {
> controller.awaitTerminated();
> } catch (ExecutionException e) {
> e.printStackTrace();
> }
> }
>
> I\u2019m running the application from a shaded jar on my cluster (I\u2019ve tried running it using both the BundledJarRunner and directly). The application is getting accepted by Yarn and appears to run, but I\u2019m not seeing my output and I\u2019m seeing the following stacktrace in the logs for the container that my runnable is running in. Any suggestions?
>
>
> Exception in thread "CompositeService STOPPING" java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:61)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
> at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
> at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:267)
> at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96)
> at org.apache.twill.internal.appmaster.ApplicationMasterMain$AppMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
> at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:57)
> ... 1 more
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.twill.internal.zookeeper.DefaultZKClientService$Callbacks$4.processResult(DefaultZKClientService.java:619)
> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> Exception in thread "main" java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:89)
> Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
> at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1015)
> at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001)
> at com.google.common.util.concurrent.AbstractService.stopAndWait(AbstractService.java:225)
> at com.google.common.util.concurrent.AbstractIdleService.stopAndWait(AbstractIdleService.java:122)
> at org.apache.twill.internal.ServiceMain.doMain(ServiceMain.java:108)
> at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:85)
> ... 5 more
> Caused by: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
> at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
> at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:267)
> at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96)
> at org.apache.twill.internal.appmaster.ApplicationMasterMain$AppMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
> at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:57)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.twill.internal.zookeeper.DefaultZKClientService$Callbacks$4.processResult(DefaultZKClientService.java:619)
> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>
>
> Caleb A. Meier, Ph.D.
> Software Engineer II \u2666 Analyst
> Parsons Corporation
> 1911 N. Fort Myer Drive, Suite 800 \u2666 Arlington, VA 22209
> Office: (703)797-3066
> Caleb.Meier@Parsons.com<ma...@Parsons.com> \u2666 www.parsons.com<https://webportal.parsons.com/,DanaInfo=www.parsons.com+>
>