[jira] [Updated] (STORM-204) Local topologies that throw exceptions can crash the REPL

["Rick Kellogg (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/STORM-204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rick Kellogg updated STORM-204:
-------------------------------
    Component/s: storm-core

> Local topologies that throw exceptions can crash the REPL
> ---------------------------------------------------------
>
>                 Key: STORM-204
>                 URL: https://issues.apache.org/jira/browse/STORM-204
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-core
>         Environment: $ java -version
> java version "1.7.0_45"
> Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
> Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
>            Reporter: Andrew Montalenti
>            Priority: Minor
>              Labels: local, local-job-runner, testing
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> We've been testing some Storm topologies using "lein repl". Based on the storm-starter project, our local testing environment offers a Clojure function (run-local!), which builts up the topology using the Clojure DSL and runs it. The issue is that there are conditions that can cause topology to fail; the exception thrown then crashes the Storm worker, which, in turn, crashes the lein repl itself. This is frustrating since part of the purpose of running a local cluster is to test it from something like a REPL.
> I think I've narrowed down what's going wrong. You can see an example session with stacktrace here:
> https://gist.github.com/amontalenti/8677464#file-example_storm_crashing_lein-txt-L142-L173
> The way I created this error is by renaming my Python module before running the topology, so that the Python file could not be found.
> I think what's going on is that the ShellBolt is throwing a RuntimeException, which is uncaught by whatever is running the ShellBolt. This, in turn, crashes the worker: "Error when launching multilang subprocess". The executor notices that the worker dies, but is a bit zealous and decides to call (halt-process!) on it. Under the hood, halt-process! uses Runtime.getRuntime#halt, which is a forcible kill of the running JVM. Since the JVM, in this case, is "lein repl", I think this is what ultimately kills the REPL.
> http://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html#halt(int)
> I suppose there are two possible fixes here. One is to make the Storm worker a little more resilient to a misconfigured ShellBolt. The other is to make the halt-process! call not run when a REPL environment is detected.
> I am glad to work on either of these fixes once the issue is confirmed and a path forward is suggested.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)