You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by vibha goyal <vi...@gmail.com> on 2016/03/13 21:43:53 UTC
Issues in Storm deployment
Hi,
I am grad student, and I am working on Storm as a part of my course project.
I have cloned the source code from git, and followed the instructions.
I am using branch : 0.10.0
After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
home,
untar it and try to run nimbus.
I get error:
Exception in thread "main" java.lang.ExceptionInInitializerError
at
org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
at org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
Caused by: java.lang.RuntimeException: java.io.IOException: Found multiple
storm.yaml resources. You're probably bundling the Storm jars with your
topology jar.
[jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
... 3 more
I deleted the "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml"
file and again tried.
This time I was able to connect the nimbus and supervisors and could see
the connections in UI.
As soon as I tried to submit the topology with the command:
storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
org.apache.storm.starter.ExclamationTopology exclamation-topology
I got error:
org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find leader
nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did you specify
a valid list of nimbus hosts for config nimbus.seeds?
First ,I am not able to resolve this error,
and second, I cannot make changes in storm.yaml as I deleted the
conf/storm.yaml
I am a beginner in Storm. I have been trying this for a day now. Any help
would be appreciated!!
Thanks!
Vibha Goyal
Re: Issues in Storm deployment
Posted by vibha goyal <vi...@gmail.com>.
Hi,
I am grad student, and I am working on Storm as a part of my course project.
I have added some code in bolt-emit function, which has increased the
latency.
Is this expected?
(fast-list-iter [^TupleImpl a anchors]
(let
[ root-ids (-> a .getMessageId .getAnchorsToIds .keySet); D1-messageId
ack-val (.getAckVal a)]
(when (pos? (count root-ids))
(.updateAckVal a 1)
(log-message
"CurrTupleCount: " (.getCurrTupleCount a))
(if (= 1 (.get task_counter))
(.incrementCurrTupleCount a))
(log-message "CurrTupleCount: " (.getCurrTupleCount a))
(fast-map-iter [[root-id weight] (.. a getMessageId getAnchorsToIds)]
(log-message "HAHA1: root id" root-id " CurrTupleCount: "
(.getCurrTupleCount a))
(let [curr-task-count (.get task_counter)
bIsLastTask? (= curr-task-count (count out-tasks))
tokens_tobe_sent (.calTokens a weight tuple_count (count
out-tasks) bIsLastTask?)]
(put-add! anchors-to-ids root-id tokens_tobe_sent)
(.updateLeftWeightsMap a root-id tokens_tobe_sent)
(log-message "VIBHA-DEBUG: Bolt[2] Component id: " component-id
" weight debug: " root-id " Input weight: " weight " Output weight:"
tokens_tobe_sent "task_counter:" curr-task-count)))
))) ;fast-list-iter ends.
(let [tuple
(TupleImpl. worker-context
values
task-id
stream
(MessageId/makeId anchors-to-ids))]
(transfer-fn t
tuple))))
In the above code, calTokens and updateLeftWeightsMap are java functions.
calTokens do a '/' and '%', while updateLeftWeightsMap updates the
calculated value in the hash.
Can anybody please tell if there is any optimized way that I am not aware
of?
Thanks already!!
Vibha Goyal
On Mon, Mar 14, 2016 at 11:23 AM, Harsha <st...@harsha.io> wrote:
> Do not package storm-core in your topology. Make sure the scope is set to
> provided.
>
>
> On Mon, Mar 14, 2016, at 07:36 AM, vibha goyal wrote:
>
>
> Thank you all!!
>
> How do I exclude this from topology? I have not made any change in any
> file.
>
>
> Just using the default files, and creating the jar files using "mvn clean
> install' in storm-starter.
>
> Regards,
> Vibha Goyal
>
> On Mon, Mar 14, 2016 at 2:58 AM, 马哲超 <ma...@gmail.com> wrote:
>
> *stom.yaml* is uesed when starting the storm cluster, not your topology.
> Do not package this file into your topology.
>
> 2016-03-14 15:15 GMT+08:00 vibha goyal <vi...@gmail.com>:
>
> Thanks, but as I mentioned in the original e-mail, I cannot keep
> storm.yaml in /home/vgoyal5/apache-storm-1.0.0-SNAPSHOT/conf/storm.yaml" .
>
> If i keep this, it give me error.
>
> How to make changes to yaml file?
>
> On Mon, Mar 14, 2016 at 1:30 AM, Harsha <ma...@harsha.io> wrote:
>
>
> its for Nimbus HA
> http://hortonworks.com/blog/fault-tolerant-nimbus-in-apache-storm/
>
>
> On Sun, Mar 13, 2016, at 11:27 PM, Sai Dilip Reddy Kiralam wrote:
>
>
> Hi Harsha,
> can you explain why nimbus seeds are used - [nimbus.seeds: ["host1",
> "host2", "host3"];]
>
>
>
>
>
> On Mon, Mar 14, 2016 at 9:49 AM, Harsha <ma...@harsha.io> wrote:
>
>
> Vibha,
> For multi-node cluster the main config we need are
> nimbus.seeds: ["nimbus_host_name"]
> storm.zookeeper.servers:
> - "zookeeper_host_name"
> - "zookeeper_host_name"
>
> You can look at the example here
> https://github.com/apache/storm/blob/master/conf/storm.yaml.example
>
> make sure you copy the same storm.yaml in all the nodes in the storm
> cluster.
>
>
> -Harsha
>
>
>
> On Sun, Mar 13, 2016, at 05:07 PM, Xiang Wang wrote:
>
> Hi,
>
> I guess you are in the wrong directory. Do "mvn package" under
> "$STORM_HOME/examples/storm-starter/". Then you can find a jar like
> "storm-starter-0.10.0.jar" in the target directory, which should be
> uploaded to storm cluster.
>
>
>
>
> -------------------------------
> Xiang Wang PhD Candidate
> Database Research Group
>
> School of Computer Science and Engineering
>
> The University of New South Wales
>
> Sydney, Australia
>
> On Mon, Mar 14, 2016 at 7:43 AM, vibha goyal <vi...@gmail.com> wrote:
>
> Hi,
>
> I am grad student, and I am working on Storm as a part of my course
> project.
>
> I have cloned the source code from git, and followed the instructions.
>
> I am using branch : 0.10.0
>
> After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
> home,
> untar it and try to run nimbus.
>
> I get error:
>
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at
> org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
> at org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
> Caused by: java.lang.RuntimeException: java.io.IOException: Found multiple
> storm.yaml resources. You're probably bundling the Storm jars with your
> topology jar.
> [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
> file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
> ... 3 more
>
>
> I deleted the "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml"
> file and again tried.
>
> This time I was able to connect the nimbus and supervisors and could see
> the connections in UI.
>
> As soon as I tried to submit the topology with the command:
>
>
>
> storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
> org.apache.storm.starter.ExclamationTopology exclamation-topology
>
>
> I got error:
>
> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find
> leader nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did
> you specify a valid list of nimbus hosts for config nimbus.seeds?
>
> First ,I am not able to resolve this error,
>
> and second, I cannot make changes in storm.yaml as I deleted the
> conf/storm.yaml
>
> I am a beginner in Storm. I have been trying this for a day now. Any help
> would be appreciated!!
>
>
>
> Thanks!
>
>
> Vibha Goyal
>
>
>
>
>
>
>
>
Re: Issues in Storm deployment
Posted by vibha goyal <vi...@gmail.com>.
Hi,
I am grad student, and I am working on Storm as a part of my course project.
I have added some code in bolt-emit function, which has increased the
latency.
Is this expected?
(fast-list-iter [^TupleImpl a anchors]
(let
[ root-ids (-> a .getMessageId .getAnchorsToIds .keySet); D1-messageId
ack-val (.getAckVal a)]
(when (pos? (count root-ids))
(.updateAckVal a 1)
(log-message
"CurrTupleCount: " (.getCurrTupleCount a))
(if (= 1 (.get task_counter))
(.incrementCurrTupleCount a))
(log-message "CurrTupleCount: " (.getCurrTupleCount a))
(fast-map-iter [[root-id weight] (.. a getMessageId getAnchorsToIds)]
(log-message "HAHA1: root id" root-id " CurrTupleCount: "
(.getCurrTupleCount a))
(let [curr-task-count (.get task_counter)
bIsLastTask? (= curr-task-count (count out-tasks))
tokens_tobe_sent (.calTokens a weight tuple_count (count
out-tasks) bIsLastTask?)]
(put-add! anchors-to-ids root-id tokens_tobe_sent)
(.updateLeftWeightsMap a root-id tokens_tobe_sent)
(log-message "VIBHA-DEBUG: Bolt[2] Component id: " component-id
" weight debug: " root-id " Input weight: " weight " Output weight:"
tokens_tobe_sent "task_counter:" curr-task-count)))
))) ;fast-list-iter ends.
(let [tuple
(TupleImpl. worker-context
values
task-id
stream
(MessageId/makeId anchors-to-ids))]
(transfer-fn t
tuple))))
In the above code, calTokens and updateLeftWeightsMap are java functions.
calTokens do a '/' and '%', while updateLeftWeightsMap updates the
calculated value in the hash.
Can anybody please tell if there is any optimized way that I am not aware
of?
Thanks already!!
Vibha Goyal
On Mon, Mar 14, 2016 at 11:23 AM, Harsha <st...@harsha.io> wrote:
> Do not package storm-core in your topology. Make sure the scope is set to
> provided.
>
>
> On Mon, Mar 14, 2016, at 07:36 AM, vibha goyal wrote:
>
>
> Thank you all!!
>
> How do I exclude this from topology? I have not made any change in any
> file.
>
>
> Just using the default files, and creating the jar files using "mvn clean
> install' in storm-starter.
>
> Regards,
> Vibha Goyal
>
> On Mon, Mar 14, 2016 at 2:58 AM, 马哲超 <ma...@gmail.com> wrote:
>
> *stom.yaml* is uesed when starting the storm cluster, not your topology.
> Do not package this file into your topology.
>
> 2016-03-14 15:15 GMT+08:00 vibha goyal <vi...@gmail.com>:
>
> Thanks, but as I mentioned in the original e-mail, I cannot keep
> storm.yaml in /home/vgoyal5/apache-storm-1.0.0-SNAPSHOT/conf/storm.yaml" .
>
> If i keep this, it give me error.
>
> How to make changes to yaml file?
>
> On Mon, Mar 14, 2016 at 1:30 AM, Harsha <ma...@harsha.io> wrote:
>
>
> its for Nimbus HA
> http://hortonworks.com/blog/fault-tolerant-nimbus-in-apache-storm/
>
>
> On Sun, Mar 13, 2016, at 11:27 PM, Sai Dilip Reddy Kiralam wrote:
>
>
> Hi Harsha,
> can you explain why nimbus seeds are used - [nimbus.seeds: ["host1",
> "host2", "host3"];]
>
>
>
>
>
> On Mon, Mar 14, 2016 at 9:49 AM, Harsha <ma...@harsha.io> wrote:
>
>
> Vibha,
> For multi-node cluster the main config we need are
> nimbus.seeds: ["nimbus_host_name"]
> storm.zookeeper.servers:
> - "zookeeper_host_name"
> - "zookeeper_host_name"
>
> You can look at the example here
> https://github.com/apache/storm/blob/master/conf/storm.yaml.example
>
> make sure you copy the same storm.yaml in all the nodes in the storm
> cluster.
>
>
> -Harsha
>
>
>
> On Sun, Mar 13, 2016, at 05:07 PM, Xiang Wang wrote:
>
> Hi,
>
> I guess you are in the wrong directory. Do "mvn package" under
> "$STORM_HOME/examples/storm-starter/". Then you can find a jar like
> "storm-starter-0.10.0.jar" in the target directory, which should be
> uploaded to storm cluster.
>
>
>
>
> -------------------------------
> Xiang Wang PhD Candidate
> Database Research Group
>
> School of Computer Science and Engineering
>
> The University of New South Wales
>
> Sydney, Australia
>
> On Mon, Mar 14, 2016 at 7:43 AM, vibha goyal <vi...@gmail.com> wrote:
>
> Hi,
>
> I am grad student, and I am working on Storm as a part of my course
> project.
>
> I have cloned the source code from git, and followed the instructions.
>
> I am using branch : 0.10.0
>
> After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
> home,
> untar it and try to run nimbus.
>
> I get error:
>
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at
> org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
> at org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
> Caused by: java.lang.RuntimeException: java.io.IOException: Found multiple
> storm.yaml resources. You're probably bundling the Storm jars with your
> topology jar.
> [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
> file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
> ... 3 more
>
>
> I deleted the "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml"
> file and again tried.
>
> This time I was able to connect the nimbus and supervisors and could see
> the connections in UI.
>
> As soon as I tried to submit the topology with the command:
>
>
>
> storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
> org.apache.storm.starter.ExclamationTopology exclamation-topology
>
>
> I got error:
>
> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find
> leader nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did
> you specify a valid list of nimbus hosts for config nimbus.seeds?
>
> First ,I am not able to resolve this error,
>
> and second, I cannot make changes in storm.yaml as I deleted the
> conf/storm.yaml
>
> I am a beginner in Storm. I have been trying this for a day now. Any help
> would be appreciated!!
>
>
>
> Thanks!
>
>
> Vibha Goyal
>
>
>
>
>
>
>
>
Re: Issues in Storm deployment
Posted by Harsha <st...@harsha.io>.
Do not package storm-core in your topology. Make sure the scope is set
to provided.
On Mon, Mar 14, 2016, at 07:36 AM, vibha goyal wrote:
>
> Thank you all!!
>
> How do I exclude this from topology? I have not made any change in
> any file.
>
>
> Just using the default files, and creating the jar files using "mvn
> clean install' in storm-starter.
>
> Regards,
> Vibha Goyal
>
> On Mon, Mar 14, 2016 at 2:58 AM, 马哲超 <ma...@gmail.com> wrote:
>> *stom.yaml* is uesed when starting the storm cluster, not your
>> topology. Do not package this file into your topology.
>>
>> 2016-03-14 15:15 GMT+08:00 vibha goyal <vi...@gmail.com>:
>>> Thanks, but as I mentioned in the original e-mail, I cannot keep
>>> storm.yaml in /home/vgoyal5/apache-storm-1.0.0-
>>> SNAPSHOT/conf/storm.yaml" .
>>>
>>> If i keep this, it give me error.
>>>
>>> How to make changes to yaml file?
>>>
>>> On Mon, Mar 14, 2016 at 1:30 AM, Harsha <ma...@harsha.io> wrote:
>>>> __
>>>> its for Nimbus HA http://hortonworks.com/blog/fault-tolerant-nimbus-in-apache-storm/
>>>>
>>>>
>>>> On Sun, Mar 13, 2016, at 11:27 PM, Sai Dilip Reddy Kiralam wrote:
>>>>>
>>>>> Hi Harsha,
>>>>> can you explain why nimbus seeds are used - [nimbus.seeds: ["host1", "host2", "host3"];]
>>>>>
>>>>> ** **
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Mar 14, 2016 at 9:49 AM, Harsha <ma...@harsha.io> wrote:
>>>>>> __
>>>>>> Vibha,
>>>>>> For multi-node cluster the main config we need are
>>>>>> nimbus.seeds: ["nimbus_host_name"]
>>>>>> storm.zookeeper.servers:
>>>>>> - "zookeeper_host_name"
>>>>>> - "zookeeper_host_name"
>>>>>>
>>>>>> You can look at the example here https://github.com/apache/storm/blob/master/conf/storm.yaml.example
>>>>>>
>>>>>> make sure you copy the same storm.yaml in all the nodes in the
>>>>>> storm cluster.
>>>>>>
>>>>>>
>>>>>> -Harsha
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Mar 13, 2016, at 05:07 PM, Xiang Wang wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I guess you are in the wrong directory. Do "mvn package" under
>>>>>>> "$STORM_HOME/examples/storm-starter/". Then you can find a jar
>>>>>>> like "storm-starter-0.10.0.jar" in the target directory, which
>>>>>>> should be uploaded to storm cluster.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -------------------------------
>>>>>>> Xiang Wang PhD Candidate
>>>>>>> Database Research Group
>>>>>>>
>>>>>>> School of Computer Science and Engineering
>>>>>>>
>>>>>>> The University of New South Wales
>>>>>>>
>>>>>>> Sydney, Australia
>>>>>>>
>>>>>>> On Mon, Mar 14, 2016 at 7:43 AM, vibha goyal <vi...@gmail.com> wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I am grad student, and I am working on Storm as a part of my
>>>>>>>> course project.
>>>>>>>>
>>>>>>>> I have cloned the source code from git, and followed the
>>>>>>>> instructions.
>>>>>>>>
>>>>>>>> I am using branch : 0.10.0
>>>>>>>>
>>>>>>>> After "mvn package", when I copy the apache-storm-
>>>>>>>> <version>.tar.gz in my home,
>>>>>>>> untar it and try to run nimbus.
>>>>>>>>
>>>>>>>> I get error:
>>>>>>>>
>>>>>>>> Exception in thread "main"
>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>> at org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
>>>>>>>> at org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
>>>>>>>> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
>>>>>>>> Caused by: java.lang.RuntimeException: java.io.IOException: Found multiple storm.yaml resources. You're probably bundling the Storm jars with your topology jar. [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml, file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
>>>>>>>> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
>>>>>>>> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
>>>>>>>> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
>>>>>>>> ... 3 more
>>>>>>>>
>>>>>>>>
>>>>>>>> I deleted the "/home/vgoyal5/apache-storm-2.0.0-
>>>>>>>> SNAPSHOT/conf/storm.yaml" file and again tried.
>>>>>>>>
>>>>>>>> This time I was able to connect the nimbus and supervisors and
>>>>>>>> could see the connections in UI.
>>>>>>>>
>>>>>>>> As soon as I tried to submit the topology with the command:
>>>>>>>>
>>>>>>>>
>>>>>>>> storm jar examples/storm-starter/target/storm-starter-2.0.0-
>>>>>>>> SNAPSHOT.jar org.apache.storm.starter.ExclamationTopology exclamation-
>>>>>>>> topology
>>>>>>>>
>>>>>>>>
>>>>>>>> I got error:
>>>>>>>>
>>>>>>>> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find leader nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did you specify a valid list of nimbus hosts for config nimbus.seeds?
>>>>>>>>
>>>>>>>> First ,I am not able to resolve this error,
>>>>>>>>
>>>>>>>> and second, I cannot make changes in storm.yaml as I deleted
>>>>>>>> the conf/storm.yaml
>>>>>>>>
>>>>>>>> I am a beginner in Storm. I have been trying this for a day
>>>>>>>> now. Any help would be appreciated!!
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>> Vibha Goyal
>>>>>>>>
>>>>>>
>>>>
Re: Issues in Storm deployment
Posted by vibha goyal <vi...@gmail.com>.
Thank you all!!
How do I exclude this from topology? I have not made any change in any
file.
Just using the default files, and creating the jar files using "mvn clean
install' in storm-starter.
Regards,
Vibha Goyal
On Mon, Mar 14, 2016 at 2:58 AM, 马哲超 <ma...@gmail.com> wrote:
> *stom.yaml* is uesed when starting the storm cluster, not your topology.
> Do not package this file into your topology.
>
> 2016-03-14 15:15 GMT+08:00 vibha goyal <vi...@gmail.com>:
>
>> Thanks, but as I mentioned in the original e-mail, I cannot keep
>> storm.yaml in /home/vgoyal5/apache-storm-1.0.0-SNAPSHOT/conf/storm.yaml"
>> .
>>
>> If i keep this, it give me error.
>>
>> How to make changes to yaml file?
>>
>> On Mon, Mar 14, 2016 at 1:30 AM, Harsha <ma...@harsha.io> wrote:
>>
>>> its for Nimbus HA
>>> http://hortonworks.com/blog/fault-tolerant-nimbus-in-apache-storm/
>>>
>>>
>>> On Sun, Mar 13, 2016, at 11:27 PM, Sai Dilip Reddy Kiralam wrote:
>>>
>>>
>>> Hi Harsha,
>>> can you explain why nimbus seeds are used - [nimbus.seeds: ["host1",
>>> "host2", "host3"];]
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Mar 14, 2016 at 9:49 AM, Harsha <ma...@harsha.io> wrote:
>>>
>>>
>>> Vibha,
>>> For multi-node cluster the main config we need are
>>> nimbus.seeds: ["nimbus_host_name"]
>>> storm.zookeeper.servers:
>>> - "zookeeper_host_name"
>>> - "zookeeper_host_name"
>>>
>>> You can look at the example here
>>> https://github.com/apache/storm/blob/master/conf/storm.yaml.example
>>>
>>> make sure you copy the same storm.yaml in all the nodes in the storm
>>> cluster.
>>>
>>>
>>> -Harsha
>>>
>>>
>>>
>>> On Sun, Mar 13, 2016, at 05:07 PM, Xiang Wang wrote:
>>>
>>> Hi,
>>>
>>> I guess you are in the wrong directory. Do "mvn package" under
>>> "$STORM_HOME/examples/storm-starter/". Then you can find a jar like
>>> "storm-starter-0.10.0.jar" in the target directory, which should be
>>> uploaded to storm cluster.
>>>
>>>
>>>
>>>
>>> -------------------------------
>>> Xiang Wang PhD Candidate
>>> Database Research Group
>>>
>>> School of Computer Science and Engineering
>>>
>>> The University of New South Wales
>>>
>>> Sydney, Australia
>>>
>>> On Mon, Mar 14, 2016 at 7:43 AM, vibha goyal <vi...@gmail.com>
>>> wrote:
>>>
>>> Hi,
>>>
>>> I am grad student, and I am working on Storm as a part of my course
>>> project.
>>>
>>> I have cloned the source code from git, and followed the instructions.
>>>
>>> I am using branch : 0.10.0
>>>
>>> After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
>>> home,
>>> untar it and try to run nimbus.
>>>
>>> I get error:
>>>
>>> Exception in thread "main" java.lang.ExceptionInInitializerError
>>> at
>>> org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
>>> at
>>> org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
>>> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
>>> Caused by: java.lang.RuntimeException: java.io.IOException: Found
>>> multiple storm.yaml resources. You're probably bundling the Storm jars with
>>> your topology jar.
>>> [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
>>> file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
>>> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
>>> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
>>> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
>>> ... 3 more
>>>
>>>
>>> I deleted the
>>> "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml" file and again
>>> tried.
>>>
>>> This time I was able to connect the nimbus and supervisors and could see
>>> the connections in UI.
>>>
>>> As soon as I tried to submit the topology with the command:
>>>
>>>
>>>
>>> storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
>>> org.apache.storm.starter.ExclamationTopology exclamation-topology
>>>
>>>
>>> I got error:
>>>
>>> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find
>>> leader nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did
>>> you specify a valid list of nimbus hosts for config nimbus.seeds?
>>>
>>> First ,I am not able to resolve this error,
>>>
>>> and second, I cannot make changes in storm.yaml as I deleted the
>>> conf/storm.yaml
>>>
>>> I am a beginner in Storm. I have been trying this for a day now. Any
>>> help would be appreciated!!
>>>
>>>
>>>
>>> Thanks!
>>>
>>>
>>> Vibha Goyal
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>
Re: Issues in Storm deployment
Posted by 马哲超 <ma...@gmail.com>.
*stom.yaml* is uesed when starting the storm cluster, not your topology. Do
not package this file into your topology.
2016-03-14 15:15 GMT+08:00 vibha goyal <vi...@gmail.com>:
> Thanks, but as I mentioned in the original e-mail, I cannot keep
> storm.yaml in /home/vgoyal5/apache-storm-1.0.0-SNAPSHOT/conf/storm.yaml" .
>
> If i keep this, it give me error.
>
> How to make changes to yaml file?
>
> On Mon, Mar 14, 2016 at 1:30 AM, Harsha <ma...@harsha.io> wrote:
>
>> its for Nimbus HA
>> http://hortonworks.com/blog/fault-tolerant-nimbus-in-apache-storm/
>>
>>
>> On Sun, Mar 13, 2016, at 11:27 PM, Sai Dilip Reddy Kiralam wrote:
>>
>>
>> Hi Harsha,
>> can you explain why nimbus seeds are used - [nimbus.seeds: ["host1",
>> "host2", "host3"];]
>>
>>
>>
>>
>>
>> On Mon, Mar 14, 2016 at 9:49 AM, Harsha <ma...@harsha.io> wrote:
>>
>>
>> Vibha,
>> For multi-node cluster the main config we need are
>> nimbus.seeds: ["nimbus_host_name"]
>> storm.zookeeper.servers:
>> - "zookeeper_host_name"
>> - "zookeeper_host_name"
>>
>> You can look at the example here
>> https://github.com/apache/storm/blob/master/conf/storm.yaml.example
>>
>> make sure you copy the same storm.yaml in all the nodes in the storm
>> cluster.
>>
>>
>> -Harsha
>>
>>
>>
>> On Sun, Mar 13, 2016, at 05:07 PM, Xiang Wang wrote:
>>
>> Hi,
>>
>> I guess you are in the wrong directory. Do "mvn package" under
>> "$STORM_HOME/examples/storm-starter/". Then you can find a jar like
>> "storm-starter-0.10.0.jar" in the target directory, which should be
>> uploaded to storm cluster.
>>
>>
>>
>>
>> -------------------------------
>> Xiang Wang PhD Candidate
>> Database Research Group
>>
>> School of Computer Science and Engineering
>>
>> The University of New South Wales
>>
>> Sydney, Australia
>>
>> On Mon, Mar 14, 2016 at 7:43 AM, vibha goyal <vi...@gmail.com>
>> wrote:
>>
>> Hi,
>>
>> I am grad student, and I am working on Storm as a part of my course
>> project.
>>
>> I have cloned the source code from git, and followed the instructions.
>>
>> I am using branch : 0.10.0
>>
>> After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
>> home,
>> untar it and try to run nimbus.
>>
>> I get error:
>>
>> Exception in thread "main" java.lang.ExceptionInInitializerError
>> at
>> org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
>> at
>> org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
>> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
>> Caused by: java.lang.RuntimeException: java.io.IOException: Found
>> multiple storm.yaml resources. You're probably bundling the Storm jars with
>> your topology jar.
>> [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
>> file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
>> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
>> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
>> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
>> ... 3 more
>>
>>
>> I deleted the "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml"
>> file and again tried.
>>
>> This time I was able to connect the nimbus and supervisors and could see
>> the connections in UI.
>>
>> As soon as I tried to submit the topology with the command:
>>
>>
>>
>> storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
>> org.apache.storm.starter.ExclamationTopology exclamation-topology
>>
>>
>> I got error:
>>
>> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find
>> leader nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did
>> you specify a valid list of nimbus hosts for config nimbus.seeds?
>>
>> First ,I am not able to resolve this error,
>>
>> and second, I cannot make changes in storm.yaml as I deleted the
>> conf/storm.yaml
>>
>> I am a beginner in Storm. I have been trying this for a day now. Any help
>> would be appreciated!!
>>
>>
>>
>> Thanks!
>>
>>
>> Vibha Goyal
>>
>>
>>
>>
>>
>>
>
>
Re: Issues in Storm deployment
Posted by vibha goyal <vi...@gmail.com>.
Thanks, but as I mentioned in the original e-mail, I cannot keep storm.yaml
in /home/vgoyal5/apache-storm-1.0.0-SNAPSHOT/conf/storm.yaml" .
If i keep this, it give me error.
How to make changes to yaml file?
On Mon, Mar 14, 2016 at 1:30 AM, Harsha <ma...@harsha.io> wrote:
> its for Nimbus HA
> http://hortonworks.com/blog/fault-tolerant-nimbus-in-apache-storm/
>
>
> On Sun, Mar 13, 2016, at 11:27 PM, Sai Dilip Reddy Kiralam wrote:
>
>
> Hi Harsha,
> can you explain why nimbus seeds are used - [nimbus.seeds: ["host1",
> "host2", "host3"];]
>
>
>
>
>
> On Mon, Mar 14, 2016 at 9:49 AM, Harsha <ma...@harsha.io> wrote:
>
>
> Vibha,
> For multi-node cluster the main config we need are
> nimbus.seeds: ["nimbus_host_name"]
> storm.zookeeper.servers:
> - "zookeeper_host_name"
> - "zookeeper_host_name"
>
> You can look at the example here
> https://github.com/apache/storm/blob/master/conf/storm.yaml.example
>
> make sure you copy the same storm.yaml in all the nodes in the storm
> cluster.
>
>
> -Harsha
>
>
>
> On Sun, Mar 13, 2016, at 05:07 PM, Xiang Wang wrote:
>
> Hi,
>
> I guess you are in the wrong directory. Do "mvn package" under
> "$STORM_HOME/examples/storm-starter/". Then you can find a jar like
> "storm-starter-0.10.0.jar" in the target directory, which should be
> uploaded to storm cluster.
>
>
>
>
> -------------------------------
> Xiang Wang PhD Candidate
> Database Research Group
>
> School of Computer Science and Engineering
>
> The University of New South Wales
>
> Sydney, Australia
>
> On Mon, Mar 14, 2016 at 7:43 AM, vibha goyal <vi...@gmail.com> wrote:
>
> Hi,
>
> I am grad student, and I am working on Storm as a part of my course
> project.
>
> I have cloned the source code from git, and followed the instructions.
>
> I am using branch : 0.10.0
>
> After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
> home,
> untar it and try to run nimbus.
>
> I get error:
>
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at
> org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
> at org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
> Caused by: java.lang.RuntimeException: java.io.IOException: Found multiple
> storm.yaml resources. You're probably bundling the Storm jars with your
> topology jar.
> [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
> file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
> ... 3 more
>
>
> I deleted the "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml"
> file and again tried.
>
> This time I was able to connect the nimbus and supervisors and could see
> the connections in UI.
>
> As soon as I tried to submit the topology with the command:
>
>
>
> storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
> org.apache.storm.starter.ExclamationTopology exclamation-topology
>
>
> I got error:
>
> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find
> leader nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did
> you specify a valid list of nimbus hosts for config nimbus.seeds?
>
> First ,I am not able to resolve this error,
>
> and second, I cannot make changes in storm.yaml as I deleted the
> conf/storm.yaml
>
> I am a beginner in Storm. I have been trying this for a day now. Any help
> would be appreciated!!
>
>
>
> Thanks!
>
>
> Vibha Goyal
>
>
>
>
>
>
Re: Issues in Storm deployment
Posted by Harsha <ma...@harsha.io>.
its for Nimbus HA�http://hortonworks.com/blog/fault-tolerant-nimbus-in-apache-storm/
On Sun, Mar 13, 2016, at 11:27 PM, Sai Dilip Reddy Kiralam wrote:
>
> Hi Harsha,
> can you explain why nimbus seeds are used - [nimbus.seeds: ["host1", "host2", "host3"];]
>
> ** **
>
>
>
> On Mon, Mar 14, 2016 at 9:49 AM, Harsha <ma...@harsha.io> wrote:
>> __
>> Vibha,
>> For multi-node cluster the main config we need are
>> nimbus.seeds: ["nimbus_host_name"]
>> storm.zookeeper.servers:
>> - "zookeeper_host_name"
>> - "zookeeper_host_name"
>>
>> You can look at the example here https://github.com/apache/storm/blob/master/conf/storm.yaml.example
>>
>> make sure you copy the same storm.yaml in all the nodes in the storm
>> cluster.
>>
>>
>> -Harsha
>>
>>
>>
>> On Sun, Mar 13, 2016, at 05:07 PM, Xiang Wang wrote:
>>> Hi,
>>>
>>> I guess you are in the wrong directory. Do "mvn package" under "$STORM_HOME/examples/storm-
>>> starter/". Then you can find a jar like "storm-starter-0.10.0.jar"
>>> in the target directory, which should be uploaded to storm cluster.
>>>
>>>
>>>
>>>
>>> -------------------------------
>>> Xiang Wang PhD Candidate
>>> Database Research Group
>>>
>>> School of Computer Science and Engineering
>>>
>>> The University of New South Wales
>>>
>>> Sydney, Australia
>>>
>>> On Mon, Mar 14, 2016 at 7:43 AM, vibha goyal <vi...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I am grad student, and I am working on Storm as a part of my course
>>>> project.
>>>>
>>>> I have cloned the source code from git, and followed the
>>>> instructions.
>>>>
>>>> I am using branch :�0.10.0
>>>>
>>>> After "mvn package", when I copy the apache-storm-<version>.tar.gz
>>>> in my home,
>>>> untar it and try to run nimbus.
>>>>
>>>> I get error:�
>>>>
>>>> Exception in thread "main" java.lang.ExceptionInInitializerError
>>>> at org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
>>>> at org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
>>>> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
>>>> Caused by: java.lang.RuntimeException: java.io.IOException: Found multiple storm.yaml resources. You're probably bundling the Storm jars with your topology jar. [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml, file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
>>>> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
>>>> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
>>>> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
>>>> ... 3 more
>>>>
>>>>
>>>> I deleted the "/home/vgoyal5/apache-storm-2.0.0-
>>>> SNAPSHOT/conf/storm.yaml" file and again tried.
>>>>
>>>> This time I was able to connect the nimbus and supervisors and
>>>> could see the connections in UI.
>>>>
>>>> As soon as I tried to submit the topology with the command:
>>>>
>>>>
>>>> storm jar examples/storm-starter/target/storm-starter-2.0.0-
>>>> SNAPSHOT.jar org.apache.storm.starter.ExclamationTopology exclamation-
>>>> topology
>>>>
>>>>
>>>> I got error:
>>>>
>>>> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find leader nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did you specify a valid list of nimbus hosts for config nimbus.seeds?
>>>>
>>>> First ,I am not able to resolve this error,
>>>>
>>>> and second, I cannot make changes in storm.yaml as I deleted the
>>>> conf/storm.yaml
>>>>
>>>> I am a beginner in Storm. I have been trying this for a day now.
>>>> Any help would be appreciated!!
>>>>
>>>>
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> Vibha Goyal
>>>>
>>
Re: Issues in Storm deployment
Posted by Sai Dilip Reddy Kiralam <dk...@aadhya-analytics.com>.
Hi Harsha,
can you explain why nimbus seeds are used - [nimbus.seeds: ["host1",
"host2", "host3"];]
On Mon, Mar 14, 2016 at 9:49 AM, Harsha <ma...@harsha.io> wrote:
> Vibha,
> For multi-node cluster the main config we need are
> nimbus.seeds: ["nimbus_host_name"]
> storm.zookeeper.servers:
> - "zookeeper_host_name"
> - "zookeeper_host_name"
>
> You can look at the example here
> https://github.com/apache/storm/blob/master/conf/storm.yaml.example
>
> make sure you copy the same storm.yaml in all the nodes in the storm
> cluster.
>
> -Harsha
>
>
> On Sun, Mar 13, 2016, at 05:07 PM, Xiang Wang wrote:
>
> Hi,
>
> I guess you are in the wrong directory. Do "mvn package" under
> "$STORM_HOME/examples/storm-starter/". Then you can find a jar like
> "storm-starter-0.10.0.jar" in the target directory, which should be
> uploaded to storm cluster.
>
>
>
>
> -------------------------------
> Xiang Wang PhD Candidate
> Database Research Group
>
> School of Computer Science and Engineering
>
> The University of New South Wales
>
> Sydney, Australia
>
> On Mon, Mar 14, 2016 at 7:43 AM, vibha goyal <vi...@gmail.com> wrote:
>
> Hi,
>
> I am grad student, and I am working on Storm as a part of my course
> project.
>
> I have cloned the source code from git, and followed the instructions.
>
> I am using branch : 0.10.0
>
> After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
> home,
> untar it and try to run nimbus.
>
> I get error:
>
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at
> org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
> at org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
> Caused by: java.lang.RuntimeException: java.io.IOException: Found multiple
> storm.yaml resources. You're probably bundling the Storm jars with your
> topology jar.
> [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
> file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
> ... 3 more
>
>
> I deleted the "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml"
> file and again tried.
>
> This time I was able to connect the nimbus and supervisors and could see
> the connections in UI.
>
> As soon as I tried to submit the topology with the command:
>
>
>
> storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
> org.apache.storm.starter.ExclamationTopology exclamation-topology
>
>
> I got error:
>
> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find
> leader nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did
> you specify a valid list of nimbus hosts for config nimbus.seeds?
>
> First ,I am not able to resolve this error,
>
> and second, I cannot make changes in storm.yaml as I deleted the
> conf/storm.yaml
>
> I am a beginner in Storm. I have been trying this for a day now. Any help
> would be appreciated!!
>
>
>
> Thanks!
>
>
> Vibha Goyal
>
>
>
>
Re: Issues in Storm deployment
Posted by Harsha <ma...@harsha.io>.
Vibha,
For multi-node cluster the main config we need are
nimbus.seeds: ["nimbus_host_name"]
storm.zookeeper.servers:
- "zookeeper_host_name"
- "zookeeper_host_name"
You can look at the example here https://github.com/apache/storm/blob/master/conf/storm.yaml.example
make sure you copy the same storm.yaml in all the nodes in the
storm cluster.
-Harsha
On Sun, Mar 13, 2016, at 05:07 PM, Xiang Wang wrote:
> Hi,
>
> I guess you are in the wrong directory. Do "mvn package" under "$STORM_HOME/examples/storm-
> starter/". Then you can find a jar like "storm-starter-0.10.0.jar" in
> the target directory, which should be uploaded to storm cluster.
>
>
>
>
> -------------------------------
> Xiang Wang PhD Candidate
> Database Research Group
>
> School of Computer Science and Engineering
>
> The University of New South Wales
>
> Sydney, Australia
>
> On Mon, Mar 14, 2016 at 7:43 AM, vibha goyal <vi...@gmail.com> wrote:
>> Hi,
>>
>> I am grad student, and I am working on Storm as a part of my course
>> project.
>>
>> I have cloned the source code from git, and followed the
>> instructions.
>>
>> I am using branch :�0.10.0
>>
>> After "mvn package", when I copy the apache-storm-<version>.tar.gz�in
>> my home,
>> untar it and try to run nimbus.
>>
>> I get error:�
>>
>> Exception in thread "main" java.lang.ExceptionInInitializerError
>> at org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
>> at org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
>> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
>> Caused by: java.lang.RuntimeException: java.io.IOException: Found multiple storm.yaml resources. You're probably bundling the Storm jars with your topology jar. [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml, file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
>> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
>> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
>> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
>> ... 3 more
>>
>>
>> I deleted the "/home/vgoyal5/apache-storm-2.0.0-
>> SNAPSHOT/conf/storm.yaml" file and again tried.
>>
>> This time I was able to connect the nimbus and supervisors and could
>> see the connections in UI.
>>
>> As soon as I tried to submit the topology with the command:
>>
>>
>> storm jar examples/storm-starter/target/storm-starter-2.0.0-
>> SNAPSHOT.jar org.apache.storm.starter.ExclamationTopology exclamation-
>> topology
>>
>>
>> I got error:
>>
>> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find leader nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did you specify a valid list of nimbus hosts for config nimbus.seeds?
>>
>> First ,I am not able to resolve this error,
>>
>> and second, I cannot make changes in storm.yaml as I deleted the
>> conf/storm.yaml
>>
>> I am a beginner in Storm. I have been trying this for a day now. Any
>> help would be appreciated!!
>>
>>
>>
>> Thanks!
>>
>>
>> Vibha Goyal
>>
Re: Issues in Storm deployment
Posted by Xiang Wang <xi...@gmail.com>.
Hi,
I guess you are in the wrong directory. Do "mvn package" under
"$STORM_HOME/examples/storm-starter/". Then you can find a jar like
"storm-starter-0.10.0.jar" in the target directory, which should be
uploaded to storm cluster.
-------------------------------
Xiang Wang PhD Candidate
Database Research Group
School of Computer Science and Engineering
The University of New South Wales
Sydney, Australia
On Mon, Mar 14, 2016 at 7:43 AM, vibha goyal <vi...@gmail.com> wrote:
> Hi,
>
> I am grad student, and I am working on Storm as a part of my course
> project.
>
> I have cloned the source code from git, and followed the instructions.
>
> I am using branch : 0.10.0
>
> After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
> home,
> untar it and try to run nimbus.
>
> I get error:
>
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at
> org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
> at org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
> Caused by: java.lang.RuntimeException: java.io.IOException: Found multiple
> storm.yaml resources. You're probably bundling the Storm jars with your
> topology jar.
> [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
> file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
> ... 3 more
>
>
> I deleted the "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml"
> file and again tried.
>
> This time I was able to connect the nimbus and supervisors and could see
> the connections in UI.
>
> As soon as I tried to submit the topology with the command:
>
> storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
> org.apache.storm.starter.ExclamationTopology exclamation-topology
>
> I got error:
>
> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find
> leader nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did
> you specify a valid list of nimbus hosts for config nimbus.seeds?
>
> First ,I am not able to resolve this error,
>
> and second, I cannot make changes in storm.yaml as I deleted the
> conf/storm.yaml
>
> I am a beginner in Storm. I have been trying this for a day now. Any help
> would be appreciated!!
>
>
>
> Thanks!
>
> Vibha Goyal
>
Re: Issues in Storm deployment
Posted by vibha goyal <vi...@gmail.com>.
It worked!!
Can you please help me about how to change the storm.yaml file ?
Thanks already!!
On Sun, Mar 13, 2016 at 5:55 PM, vibha goyal <vi...@gmail.com> wrote:
> Hi ,
>
> Thanks for prompt reply.
>
> But, I have to use on multi-node cluster, and changes in storm.yaml will
> be required.
>
> Will the below settings work?
>
>
>
> On Sun, Mar 13, 2016 at 4:15 PM, Harsha <st...@harsha.io> wrote:
>
>> Vibhay,
>> You are using master branch which is under development to
>> move to Java. This might be very unstable. I suggest you to
>> use 1.x-branch in general as its getting ready for storm
>> 1.0 release. You don't need to add anything to storm.yaml ,
>> defaults should be fine for a single node cluster.
>>
>> 1. git checkout 1.x-branch
>> 2. mvn -DskipTests clean install
>> 3. cd storm-dist/binary
>> 4. mvn clean package
>> 5. cd target
>> 6. find and unzip apache-storm-1.0.0-SNAPSHOT
>> 7. run required services
>> bin/storm dev-zookeeper, bin/storm nimbus, bin/storm supervisor,
>> bin/storm ui
>> 8. ./bin/storm jar
>> examples/storm-starter/storm-starter-topologies-1.0.0-SNAPSHOT.jar
>> org.apache.storm.starter.WordCountTopology wordcount
>>
>> For bit more details you can refer here
>> http://blog.harsha.io/setting-up-a-single-node-apache-storm-cluster/
>> its on a older version of storm but relevant for the latest as well.
>>
>> -Harsha
>>
>>
>>
>>
>> On Sun, Mar 13, 2016, at 01:43 PM, vibha goyal wrote:
>> > Hi,
>> >
>> > I am grad student, and I am working on Storm as a part of my course
>> > project.
>> >
>> > I have cloned the source code from git, and followed the instructions.
>> >
>> > I am using branch : 0.10.0
>> >
>> > After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
>> > home,
>> > untar it and try to run nimbus.
>> >
>> > I get error:
>> >
>> > Exception in thread "main" java.lang.ExceptionInInitializerError
>> > at
>> >
>> org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
>> > at
>> > org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
>> > at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
>> > Caused by: java.lang.RuntimeException: java.io.IOException: Found
>> > multiple
>> > storm.yaml resources. You're probably bundling the Storm jars with your
>> > topology jar.
>> >
>> [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
>> > file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
>> > at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
>> > at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
>> > at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
>> > ... 3 more
>> >
>> >
>> > I deleted the
>> "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml"
>> > file and again tried.
>> >
>> > This time I was able to connect the nimbus and supervisors and could see
>> > the connections in UI.
>> >
>> > As soon as I tried to submit the topology with the command:
>> >
>> > storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
>> > org.apache.storm.starter.ExclamationTopology exclamation-topology
>> >
>> > I got error:
>> >
>> > org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find
>> > leader
>> > nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did you
>> > specify
>> > a valid list of nimbus hosts for config nimbus.seeds?
>> >
>> > First ,I am not able to resolve this error,
>> >
>> > and second, I cannot make changes in storm.yaml as I deleted the
>> > conf/storm.yaml
>> >
>> > I am a beginner in Storm. I have been trying this for a day now. Any
>> help
>> > would be appreciated!!
>> >
>> >
>> >
>> > Thanks!
>> >
>> > Vibha Goyal
>>
>
>
Re: Issues in Storm deployment
Posted by vibha goyal <vi...@gmail.com>.
Hi ,
Thanks for prompt reply.
But, I have to use on multi-node cluster, and changes in storm.yaml will be
required.
Will the below settings work?
On Sun, Mar 13, 2016 at 4:15 PM, Harsha <st...@harsha.io> wrote:
> Vibhay,
> You are using master branch which is under development to
> move to Java. This might be very unstable. I suggest you to
> use 1.x-branch in general as its getting ready for storm
> 1.0 release. You don't need to add anything to storm.yaml ,
> defaults should be fine for a single node cluster.
>
> 1. git checkout 1.x-branch
> 2. mvn -DskipTests clean install
> 3. cd storm-dist/binary
> 4. mvn clean package
> 5. cd target
> 6. find and unzip apache-storm-1.0.0-SNAPSHOT
> 7. run required services
> bin/storm dev-zookeeper, bin/storm nimbus, bin/storm supervisor,
> bin/storm ui
> 8. ./bin/storm jar
> examples/storm-starter/storm-starter-topologies-1.0.0-SNAPSHOT.jar
> org.apache.storm.starter.WordCountTopology wordcount
>
> For bit more details you can refer here
> http://blog.harsha.io/setting-up-a-single-node-apache-storm-cluster/
> its on a older version of storm but relevant for the latest as well.
>
> -Harsha
>
>
>
>
> On Sun, Mar 13, 2016, at 01:43 PM, vibha goyal wrote:
> > Hi,
> >
> > I am grad student, and I am working on Storm as a part of my course
> > project.
> >
> > I have cloned the source code from git, and followed the instructions.
> >
> > I am using branch : 0.10.0
> >
> > After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
> > home,
> > untar it and try to run nimbus.
> >
> > I get error:
> >
> > Exception in thread "main" java.lang.ExceptionInInitializerError
> > at
> >
> org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
> > at
> > org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
> > at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
> > Caused by: java.lang.RuntimeException: java.io.IOException: Found
> > multiple
> > storm.yaml resources. You're probably bundling the Storm jars with your
> > topology jar.
> >
> [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
> > file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
> > at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
> > at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
> > at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
> > ... 3 more
> >
> >
> > I deleted the "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml"
> > file and again tried.
> >
> > This time I was able to connect the nimbus and supervisors and could see
> > the connections in UI.
> >
> > As soon as I tried to submit the topology with the command:
> >
> > storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
> > org.apache.storm.starter.ExclamationTopology exclamation-topology
> >
> > I got error:
> >
> > org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find
> > leader
> > nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did you
> > specify
> > a valid list of nimbus hosts for config nimbus.seeds?
> >
> > First ,I am not able to resolve this error,
> >
> > and second, I cannot make changes in storm.yaml as I deleted the
> > conf/storm.yaml
> >
> > I am a beginner in Storm. I have been trying this for a day now. Any help
> > would be appreciated!!
> >
> >
> >
> > Thanks!
> >
> > Vibha Goyal
>
Re: Issues in Storm deployment
Posted by Harsha <st...@harsha.io>.
Vibhay,
You are using master branch which is under development to
move to Java. This might be very unstable. I suggest you to
use 1.x-branch in general as its getting ready for storm
1.0 release. You don't need to add anything to storm.yaml ,
defaults should be fine for a single node cluster.
1. git checkout 1.x-branch
2. mvn -DskipTests clean install
3. cd storm-dist/binary
4. mvn clean package
5. cd target
6. find and unzip apache-storm-1.0.0-SNAPSHOT
7. run required services
bin/storm dev-zookeeper, bin/storm nimbus, bin/storm supervisor,
bin/storm ui
8. ./bin/storm jar
examples/storm-starter/storm-starter-topologies-1.0.0-SNAPSHOT.jar
org.apache.storm.starter.WordCountTopology wordcount
For bit more details you can refer here
http://blog.harsha.io/setting-up-a-single-node-apache-storm-cluster/
its on a older version of storm but relevant for the latest as well.
-Harsha
On Sun, Mar 13, 2016, at 01:43 PM, vibha goyal wrote:
> Hi,
>
> I am grad student, and I am working on Storm as a part of my course
> project.
>
> I have cloned the source code from git, and followed the instructions.
>
> I am using branch : 0.10.0
>
> After "mvn package", when I copy the apache-storm-<version>.tar.gz in my
> home,
> untar it and try to run nimbus.
>
> I get error:
>
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at
> org.apache.storm.utils.ConfigUtils.readStormConfigImpl(ConfigUtils.java:138)
> at
> org.apache.storm.utils.ConfigUtils.readStormConfig(ConfigUtils.java:134)
> at org.apache.storm.command.ConfigValue.main(ConfigValue.java:27)
> Caused by: java.lang.RuntimeException: java.io.IOException: Found
> multiple
> storm.yaml resources. You're probably bundling the Storm jars with your
> topology jar.
> [jar:file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/lib/storm-core-2.0.0-SNAPSHOT.jar!/storm.yaml,
> file:/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml]
> at org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:372)
> at org.apache.storm.utils.Utils.readStormConfig(Utils.java:456)
> at org.apache.storm.utils.Utils.<clinit>(Utils.java:173)
> ... 3 more
>
>
> I deleted the "/home/vgoyal5/apache-storm-2.0.0-SNAPSHOT/conf/storm.yaml"
> file and again tried.
>
> This time I was able to connect the nimbus and supervisors and could see
> the connections in UI.
>
> As soon as I tried to submit the topology with the command:
>
> storm jar examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar
> org.apache.storm.starter.ExclamationTopology exclamation-topology
>
> I got error:
>
> org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find
> leader
> nimbus from seed hosts [sp16-cs525-g05-01.cs.illinois.edu]. Did you
> specify
> a valid list of nimbus hosts for config nimbus.seeds?
>
> First ,I am not able to resolve this error,
>
> and second, I cannot make changes in storm.yaml as I deleted the
> conf/storm.yaml
>
> I am a beginner in Storm. I have been trying this for a day now. Any help
> would be appreciated!!
>
>
>
> Thanks!
>
> Vibha Goyal