You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by amjad khan <am...@gmail.com> on 2014/10/26 14:21:15 UTC

Trouble running topology in Storm cluster but the same topology works fine in local cluster.

I have a topology where spout takes data from mysql and bolt dumps the
processed data into mongodb. The topology runs fine in local cluster but
when deployed in storm cluster it does nothing just the topology is running
and in UI there is no stats of tuple emitted transferred or processed. In
short nothing happens no tuple comes into storm cluster and none is dumped
into mongodb.

In worker logs it shows the following error:
2014-10-24 01:03:15 worker [ERROR] Error on initialization of server
mk-worker
java.io.IOException: No such file or directory
        at java.io.UnixFileSystem.createFileExclusively(Native Method)
        at java.io.File.createNewFile(File.java:900)
        at backtype.storm.util$touch.invoke(util.clj:432)
        at
backtype.storm.daemon.worker$fn__4348$exec_fn__1228__auto____4349.invoke(worker.clj:331)
        at clojure.lang.AFn.applyToHelper(AFn.java:185)
        at clojure.lang.AFn.applyTo(AFn.java:151)
        at clojure.core$apply.invoke(core.clj:601)
        at
backtype.storm.daemon.worker$fn__4348$mk_worker__4404.doInvoke(worker.clj:323)
        at clojure.lang.RestFn.invoke(RestFn.java:512)
        at backtype.storm.daemon.worker$_main.invoke(worker.clj:433)
        at clojure.lang.AFn.applyToHelper(AFn.java:172)
        at clojure.lang.AFn.applyTo(AFn.java:151)
        at backtype.storm.daemon.worker.main(Unknown Source)
2013-04-26 01:03:15 util [INFO] Halting process: ("Error on initialization")

In supervisor logs it shows:
2014-10-24 00:54:35 supervisor [INFO] cd4b0c4c-2e46-4ae7-b3b7-782231605b4f
still hasn't started


While the topology was running i saw the worker dir. in which for a
particular worker pid dir was created but there was no file in pid dir. I
guess the file is not created which is why the topology is not running.I
checked the permission for storm,local,dir it is full permission for the
user though which i am running the storm topology so why the file is not
created i am not getting because in the same cluster file is created in pid
dir for Wordcount topology.

I am able to run the WordCount topology in Storm Cluster.

I found one post related to this which says its a zookeeper problem but
what exactly is the zookeeper problem it wasnt mentioned. Beacause what i
think if its a zookeeper problem then i should not be able to run WordCount
topology too.

I also re-install the zookeeper by deleting first all the directories of
zookeeper. I also re-installed the storm cluster but the problem still
persists.

Please help me on this in any manner you can.

Thanks in advance.

Re: Trouble running topology in Storm cluster but the same topology works fine in local cluster.

Posted by Nick Beenham <ni...@gmail.com>.
I had a similar problem when it couldn't create the local files directory.

On Sun Oct 26 2014 at 9:22:18 AM amjad khan <am...@gmail.com>
wrote:

> I have a topology where spout takes data from mysql and bolt dumps the
> processed data into mongodb. The topology runs fine in local cluster but
> when deployed in storm cluster it does nothing just the topology is running
> and in UI there is no stats of tuple emitted transferred or processed. In
> short nothing happens no tuple comes into storm cluster and none is dumped
> into mongodb.
>
> In worker logs it shows the following error:
> 2014-10-24 01:03:15 worker [ERROR] Error on initialization of server
> mk-worker
> java.io.IOException: No such file or directory
>         at java.io.UnixFileSystem.createFileExclusively(Native Method)
>         at java.io.File.createNewFile(File.java:900)
>         at backtype.storm.util$touch.invoke(util.clj:432)
>         at
> backtype.storm.daemon.worker$fn__4348$exec_fn__1228__auto____4349.invoke(worker.clj:331)
>         at clojure.lang.AFn.applyToHelper(AFn.java:185)
>         at clojure.lang.AFn.applyTo(AFn.java:151)
>         at clojure.core$apply.invoke(core.clj:601)
>         at
> backtype.storm.daemon.worker$fn__4348$mk_worker__4404.doInvoke(worker.clj:323)
>         at clojure.lang.RestFn.invoke(RestFn.java:512)
>         at backtype.storm.daemon.worker$_main.invoke(worker.clj:433)
>         at clojure.lang.AFn.applyToHelper(AFn.java:172)
>         at clojure.lang.AFn.applyTo(AFn.java:151)
>         at backtype.storm.daemon.worker.main(Unknown Source)
> 2013-04-26 01:03:15 util [INFO] Halting process: ("Error on
> initialization")
>
> In supervisor logs it shows:
> 2014-10-24 00:54:35 supervisor [INFO] cd4b0c4c-2e46-4ae7-b3b7-782231605b4f
> still hasn't started
>
>
> While the topology was running i saw the worker dir. in which for a
> particular worker pid dir was created but there was no file in pid dir. I
> guess the file is not created which is why the topology is not running.I
> checked the permission for storm,local,dir it is full permission for the
> user though which i am running the storm topology so why the file is not
> created i am not getting because in the same cluster file is created in pid
> dir for Wordcount topology.
>
> I am able to run the WordCount topology in Storm Cluster.
>
> I found one post related to this which says its a zookeeper problem but
> what exactly is the zookeeper problem it wasnt mentioned. Beacause what i
> think if its a zookeeper problem then i should not be able to run WordCount
> topology too.
>
> I also re-install the zookeeper by deleting first all the directories of
> zookeeper. I also re-installed the storm cluster but the problem still
> persists.
>
> Please help me on this in any manner you can.
>
> Thanks in advance.
>
>