You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@twill.apache.org by "Meier, Caleb" <Ca...@parsons.com> on 2017/02/06 22:51:41 UTC

Zookeeper Issue when Creating a Simple Twill Application

Hello,

I’m trying to create my own simple HelloWorld in Twill. The run method for the runnable is the following:

            @Override
            public void run() {
                        System.err.println("Starting No-Op notification task: " + new Date());
                        try {
                                    Thread.sleep(5000);
                        } catch (InterruptedException e) {
                                    e.printStackTrace();
                        }
                        System.err.println("Ending No-Op notification task: " + new Date());
               }

The client has the following main:

public static void main(String[] args) {

                        String zkStr = args[0];
                        YarnConfiguration yarnConfiguration = new YarnConfiguration();
                        TwillRunnerService runner = new YarnTwillRunnerService(yarnConfiguration, zkStr);
                        runner.start();

                        TwillController controller = runner.prepare(new NotificationRunnable())
                                                .addLogHandler(new PrinterLogHandler(new PrintWriter(System.out)))
                                                .start();

                        Runtime.getRuntime().addShutdownHook(new Thread() {
                                    @Override
                                    public void run() {
                                                try {
                                                            Futures.getUnchecked(controller.terminate());
                                                } finally {
                                                            runner.stop();
                                                }
                                    }
                        });

                        try {
                                    controller.awaitTerminated();
                        } catch (ExecutionException e) {
                                    e.printStackTrace();
                        }
               }

I’m running the application from a shaded jar on my cluster (I’ve tried running it using both the BundledJarRunner and directly). The application is getting accepted by Yarn and appears to run, but I’m not seeing my output and I’m seeing the following stacktrace in the logs for the container that my runnable is running in.  Any suggestions?


Exception in thread "CompositeService STOPPING" java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:61)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
        at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
        at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:267)
        at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96)
        at org.apache.twill.internal.appmaster.ApplicationMasterMain$AppMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
        at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:57)
        ... 1 more
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.twill.internal.zookeeper.DefaultZKClientService$Callbacks$4.processResult(DefaultZKClientService.java:619)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:89)
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
        at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1015)
        at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001)
        at com.google.common.util.concurrent.AbstractService.stopAndWait(AbstractService.java:225)
        at com.google.common.util.concurrent.AbstractIdleService.stopAndWait(AbstractIdleService.java:122)
        at org.apache.twill.internal.ServiceMain.doMain(ServiceMain.java:108)
        at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:85)
        ... 5 more
Caused by: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
        at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
        at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:267)
        at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96)
        at org.apache.twill.internal.appmaster.ApplicationMasterMain$AppMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
        at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:57)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.twill.internal.zookeeper.DefaultZKClientService$Callbacks$4.processResult(DefaultZKClientService.java:619)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)


Caleb A. Meier, Ph.D.
Software Engineer II ♦ Analyst
Parsons Corporation
1911 N. Fort Myer Drive, Suite 800 ♦ Arlington, VA 22209
Office:  (703)797-3066
Caleb.Meier@Parsons.com<ma...@Parsons.com> ♦ www.parsons.com<https://webportal.parsons.com/,DanaInfo=www.parsons.com+>


Re: Zookeeper Issue when Creating a Simple Twill Application

Posted by Terence Yim <ch...@gmail.com>.
Hi,

Can you try to replace the "System.err.println" with "Logger" from slf4j?
Twill only collect logs emitted via the slf4j API.

Terence

On Mon, Feb 6, 2017 at 3:18 PM, Martin Serrano <ma...@attivio.com> wrote:

> Caleb,
>
> If you want to see your output, you should be using a logger within your
> runnable instead of the System.err.println.  The zookeeper ERROR you see on
> exit is happening after your application has shutdown I believe.  I have
> seen this error myself after the application stops.
>
> -Martin
>
>
> On 02/06/2017 05:51 PM, Meier, Caleb wrote:
>
>> Hello,
>>
>> I’m trying to create my own simple HelloWorld in Twill. The run method
>> for the runnable is the following:
>>
>>              @Override
>>              public void run() {
>>                          System.err.println("Starting No-Op notification
>> task: " + new Date());
>>                          try {
>>                                      Thread.sleep(5000);
>>                          } catch (InterruptedException e) {
>>                                      e.printStackTrace();
>>                          }
>>                          System.err.println("Ending No-Op notification
>> task: " + new Date());
>>                 }
>>
>> The client has the following main:
>>
>> public static void main(String[] args) {
>>
>>                          String zkStr = args[0];
>>                          YarnConfiguration yarnConfiguration = new
>> YarnConfiguration();
>>                          TwillRunnerService runner = new
>> YarnTwillRunnerService(yarnConfiguration, zkStr);
>>                          runner.start();
>>
>>                          TwillController controller = runner.prepare(new
>> NotificationRunnable())
>>                                                  .addLogHandler(new
>> PrinterLogHandler(new PrintWriter(System.out)))
>>                                                  .start();
>>
>>                          Runtime.getRuntime().addShutdownHook(new
>> Thread() {
>>                                      @Override
>>                                      public void run() {
>>                                                  try {
>>
>>  Futures.getUnchecked(controller.terminate());
>>                                                  } finally {
>>
>>  runner.stop();
>>                                                  }
>>                                      }
>>                          });
>>
>>                          try {
>>                                      controller.awaitTerminated();
>>                          } catch (ExecutionException e) {
>>                                      e.printStackTrace();
>>                          }
>>                 }
>>
>> I’m running the application from a shaded jar on my cluster (I’ve tried
>> running it using both the BundledJarRunner and directly). The application
>> is getting accepted by Yarn and appears to run, but I’m not seeing my
>> output and I’m seeing the following stacktrace in the logs for the
>> container that my runnable is running in.  Any suggestions?
>>
>>
>> Exception in thread "CompositeService STOPPING"
>> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>> NoNode for /TimerRunnable/discoverable
>>          at com.google.common.base.Throwables.propagate(Throwables.java:
>> 160)
>>          at com.google.common.util.concurrent.AbstractIdleService$1$2.
>> run(AbstractIdleService.java:61)
>>          at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.util.concurrent.ExecutionException:
>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>> NoNode for /TimerRunnable/discoverable
>>          at com.google.common.util.concurrent.AbstractFuture$Sync.
>> getValue(AbstractFuture.java:294)
>>          at com.google.common.util.concurrent.AbstractFuture$Sync.get(
>> AbstractFuture.java:267)
>>          at com.google.common.util.concurrent.AbstractFuture.get(Abstrac
>> tFuture.java:96)
>>          at org.apache.twill.internal.appmaster.ApplicationMasterMain$Ap
>> pMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
>>          at com.google.common.util.concurrent.AbstractIdleService$1$2.
>> run(AbstractIdleService.java:57)
>>          ... 1 more
>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>>          at org.apache.zookeeper.KeeperException.create(KeeperException.
>> java:111)
>>          at org.apache.zookeeper.KeeperException.create(KeeperException.
>> java:51)
>>          at org.apache.twill.internal.zookeeper.DefaultZKClientService$C
>> allbacks$4.processResult(DefaultZKClientService.java:619)
>>          at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(
>> ClientCnxn.java:590)
>>          at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.
>> java:495)
>> Exception in thread "main" java.lang.reflect.InvocationTargetException
>>          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:62)
>>          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>>          at java.lang.reflect.Method.invoke(Method.java:497)
>>          at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.
>> java:89)
>> Caused by: com.google.common.util.concurrent.UncheckedExecutionException:
>> java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>>          at com.google.common.util.concurrent.Futures.wrapAndThrowUnchec
>> ked(Futures.java:1015)
>>          at com.google.common.util.concurrent.Futures.getUnchecked(
>> Futures.java:1001)
>>          at com.google.common.util.concurrent.AbstractService.stopAndWai
>> t(AbstractService.java:225)
>>          at com.google.common.util.concurrent.AbstractIdleService.stopAn
>> dWait(AbstractIdleService.java:122)
>>          at org.apache.twill.internal.ServiceMain.doMain(ServiceMain.
>> java:108)
>>          at org.apache.twill.internal.appmaster.ApplicationMasterMain.
>> main(ApplicationMasterMain.java:85)
>>          ... 5 more
>> Caused by: java.util.concurrent.ExecutionException:
>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>> NoNode for /TimerRunnable/discoverable
>>          at com.google.common.util.concurrent.AbstractFuture$Sync.
>> getValue(AbstractFuture.java:294)
>>          at com.google.common.util.concurrent.AbstractFuture$Sync.get(
>> AbstractFuture.java:267)
>>          at com.google.common.util.concurrent.AbstractFuture.get(Abstrac
>> tFuture.java:96)
>>          at org.apache.twill.internal.appmaster.ApplicationMasterMain$Ap
>> pMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
>>          at com.google.common.util.concurrent.AbstractIdleService$1$2.
>> run(AbstractIdleService.java:57)
>>          at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>>          at org.apache.zookeeper.KeeperException.create(KeeperException.
>> java:111)
>>          at org.apache.zookeeper.KeeperException.create(KeeperException.
>> java:51)
>>          at org.apache.twill.internal.zookeeper.DefaultZKClientService$C
>> allbacks$4.processResult(DefaultZKClientService.java:619)
>>          at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(
>> ClientCnxn.java:590)
>>          at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.
>> java:495)
>>
>>
>> Caleb A. Meier, Ph.D.
>> Software Engineer II ♦ Analyst
>> Parsons Corporation
>> 1911 N. Fort Myer Drive, Suite 800 ♦ Arlington, VA 22209
>> Office:  (703)797-3066
>> Caleb.Meier@Parsons.com<ma...@Parsons.com> ♦ www.parsons.com
>> <https://webportal.parsons.com/,DanaInfo=www.parsons.com+>
>>
>>
>

Re: Zookeeper Issue when Creating a Simple Twill Application

Posted by Martin Serrano <ma...@attivio.com>.
Caleb,

If you want to see your output, you should be using a logger within your 
runnable instead of the System.err.println.  The zookeeper ERROR you see 
on exit is happening after your application has shutdown I believe.  I 
have seen this error myself after the application stops.

-Martin

On 02/06/2017 05:51 PM, Meier, Caleb wrote:
> Hello,
>
> I\u2019m trying to create my own simple HelloWorld in Twill. The run method for the runnable is the following:
>
>              @Override
>              public void run() {
>                          System.err.println("Starting No-Op notification task: " + new Date());
>                          try {
>                                      Thread.sleep(5000);
>                          } catch (InterruptedException e) {
>                                      e.printStackTrace();
>                          }
>                          System.err.println("Ending No-Op notification task: " + new Date());
>                 }
>
> The client has the following main:
>
> public static void main(String[] args) {
>
>                          String zkStr = args[0];
>                          YarnConfiguration yarnConfiguration = new YarnConfiguration();
>                          TwillRunnerService runner = new YarnTwillRunnerService(yarnConfiguration, zkStr);
>                          runner.start();
>
>                          TwillController controller = runner.prepare(new NotificationRunnable())
>                                                  .addLogHandler(new PrinterLogHandler(new PrintWriter(System.out)))
>                                                  .start();
>
>                          Runtime.getRuntime().addShutdownHook(new Thread() {
>                                      @Override
>                                      public void run() {
>                                                  try {
>                                                              Futures.getUnchecked(controller.terminate());
>                                                  } finally {
>                                                              runner.stop();
>                                                  }
>                                      }
>                          });
>
>                          try {
>                                      controller.awaitTerminated();
>                          } catch (ExecutionException e) {
>                                      e.printStackTrace();
>                          }
>                 }
>
> I\u2019m running the application from a shaded jar on my cluster (I\u2019ve tried running it using both the BundledJarRunner and directly). The application is getting accepted by Yarn and appears to run, but I\u2019m not seeing my output and I\u2019m seeing the following stacktrace in the logs for the container that my runnable is running in.  Any suggestions?
>
>
> Exception in thread "CompositeService STOPPING" java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>          at com.google.common.base.Throwables.propagate(Throwables.java:160)
>          at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:61)
>          at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>          at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
>          at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:267)
>          at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96)
>          at org.apache.twill.internal.appmaster.ApplicationMasterMain$AppMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
>          at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:57)
>          ... 1 more
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>          at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
>          at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>          at org.apache.twill.internal.zookeeper.DefaultZKClientService$Callbacks$4.processResult(DefaultZKClientService.java:619)
>          at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>          at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>          at java.lang.reflect.Method.invoke(Method.java:497)
>          at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:89)
> Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>          at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1015)
>          at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001)
>          at com.google.common.util.concurrent.AbstractService.stopAndWait(AbstractService.java:225)
>          at com.google.common.util.concurrent.AbstractIdleService.stopAndWait(AbstractIdleService.java:122)
>          at org.apache.twill.internal.ServiceMain.doMain(ServiceMain.java:108)
>          at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:85)
>          ... 5 more
> Caused by: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>          at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
>          at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:267)
>          at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96)
>          at org.apache.twill.internal.appmaster.ApplicationMasterMain$AppMasterTwillZKPathService.shutDown(ApplicationMasterMain.java:262)
>          at com.google.common.util.concurrent.AbstractIdleService$1$2.run(AbstractIdleService.java:57)
>          at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /TimerRunnable/discoverable
>          at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
>          at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>          at org.apache.twill.internal.zookeeper.DefaultZKClientService$Callbacks$4.processResult(DefaultZKClientService.java:619)
>          at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>          at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>
>
> Caleb A. Meier, Ph.D.
> Software Engineer II \u2666 Analyst
> Parsons Corporation
> 1911 N. Fort Myer Drive, Suite 800 \u2666 Arlington, VA 22209
> Office:  (703)797-3066
> Caleb.Meier@Parsons.com<ma...@Parsons.com> \u2666 www.parsons.com<https://webportal.parsons.com/,DanaInfo=www.parsons.com+>
>