You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by David Crossland <da...@elastacloud.com> on 2014/04/01 17:41:24 UTC
Curiosity
I'm just wondering about the storm.yaml settings, should these be replicated to all nodes in the topology or need it only be configured on the “master” node where I deploy the topology?
thanks
David
Re: Curiosity
Posted by Svend Vanderveken <sv...@gmail.com>.
David,
Keep in mind that there is a scope difference between putting configuration
in your code/deployed topology and putting it in storm.yaml on each node:
in the former case the parameters will be global for all your topologies
whereas in the later case they only apply to one specific topology.
As Patricio correctly mention, it is recommended to use deployment
automation tools to ensure conherence of deployed configuration files.
Tools like Storm are meant to be used "at scale" => on many hosts => it is
assumed that we have some kind of deployment automation system to assist us
in the management of that IT landscape.
Cheers,
S
On Tue, Apr 1, 2014 at 12:20 PM, David Crossland <da...@elastacloud.com>wrote:
> That's a pain, thanks. Its either that then or configure it in code...
> either way I either have to recompile or touch each node..
>
> Still thanks for the info xD
>
> D
>
> *From:* Jon Logan <jm...@buffalo.edu>
> *Sent:* Tuesday, 1 April 2014 16:43
> *To:* user@storm.incubator.apache.org
>
> All nodes.
>
>
> On Tue, Apr 1, 2014 at 11:41 AM, David Crossland <da...@elastacloud.com>wrote:
>
>> I'm just wondering about the storm.yaml settings, should these be
>> replicated to all nodes in the topology or need it only be configured on
>> the "master" node where I deploy the topology?
>>
>> thanks
>> David
>>
>>
>
Re: Curiosity
Posted by Spico Florin <sp...@gmail.com>.
Hello!
If you'd like to deploy your storm topology on Amazon cloud you can use
the https://github.com/KasperMadsen/storm-deploy-alternative. Kasper has
helped me a lot with the installation.
I try it and it works fine with this configuration:
cluster:
- t1.micro {MASTER,UI}
- t1.micro {WORKER}
- t1.micro {WORKER}
- t1.micro {ZK}
- storm-version "0.9.0.1"
- zk-version "3.4.5"
- image "us-east-1/ami-35dbde5c"
- region "us-east-1"
- remote-exec-preconfig {cd ~, echo hey > hey.txt}
- remote-exec-postconfig {}
Also, you can use Nathan's Marz storm-deploy as described here:
https://github.com/nathanmarz/storm and also in this message:
http://qnalist.com/questions/4792578/can-be-storm-deploy-script-used-for-deploying-storm-0-9-x-version-on-amazon
The only thing that I could not succeeded is to start my topology on the
cluster from my Windows 7 client. Have anyone try it? If yes, where the
storm.yaml should be located.. In my case was pu in
C:\Users\myuser\.storm. You can see my entire discussion with Kasper here:
https://github.com/KasperMadsen/storm-deploy-alternative/issues/1
I'll appreciate if anyone can help me to understand how to deploy my
topology from Windows 7 on cloud (or cluster).
I'll look forward for your answers .
Regards,
Florin
On Tue, Apr 1, 2014 at 11:05 PM, David Crossland <da...@elastacloud.com>wrote:
> Appreciated, we will be looking at tools like this I the future. Atm,
> I'm just trying to tune to topology, as I'll be modifying code anyway I'll
> just stick the config mods in there.
>
> D
>
> *From:* Patricio Echagüe <pa...@gmail.com>
> *Sent:* Tuesday, 1 April 2014 17:22
> *To:* user@storm.incubator.apache.org
>
> That's where chef or puppet come handy.
>
> Sent from my Nexus 4.
> On Apr 1, 2014 9:20 AM, "David Crossland" <da...@elastacloud.com> wrote:
>
>> That's a pain, thanks. Its either that then or configure it in code...
>> either way I either have to recompile or touch each node..
>>
>> Still thanks for the info xD
>>
>> D
>>
>> *From:* Jon Logan <jm...@buffalo.edu>
>> *Sent:* Tuesday, 1 April 2014 16:43
>> *To:* user@storm.incubator.apache.org
>>
>> All nodes.
>>
>>
>> On Tue, Apr 1, 2014 at 11:41 AM, David Crossland <da...@elastacloud.com>wrote:
>>
>>> I'm just wondering about the storm.yaml settings, should these be
>>> replicated to all nodes in the topology or need it only be configured on
>>> the "master" node where I deploy the topology?
>>>
>>> thanks
>>> David
>>>
>>>
>>
Re: Curiosity
Posted by David Crossland <da...@elastacloud.com>.
Appreciated, we will be looking at tools like this I the future. Atm, I'm just trying to tune to topology, as I'll be modifying code anyway I'll just stick the config mods in there.
D
From: Patricio Echag?e<ma...@gmail.com>
Sent: ?Tuesday?, ?1? ?April? ?2014 ?17?:?22
To: user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>
That's where chef or puppet come handy.
Sent from my Nexus 4.
On Apr 1, 2014 9:20 AM, "David Crossland" <da...@elastacloud.com>> wrote:
That's a pain, thanks. Its either that then or configure it in code... either way I either have to recompile or touch each node..
Still thanks for the info xD
D
From: Jon Logan<ma...@buffalo.edu>
Sent: Tuesday, 1 April 2014 16:43
To: user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>
All nodes.
On Tue, Apr 1, 2014 at 11:41 AM, David Crossland <da...@elastacloud.com>> wrote:
I'm just wondering about the storm.yaml settings, should these be replicated to all nodes in the topology or need it only be configured on the "master" node where I deploy the topology?
thanks
David
Re: Curiosity
Posted by Patricio Echagüe <pa...@gmail.com>.
That's where chef or puppet come handy.
Sent from my Nexus 4.
On Apr 1, 2014 9:20 AM, "David Crossland" <da...@elastacloud.com> wrote:
> That's a pain, thanks. Its either that then or configure it in code...
> either way I either have to recompile or touch each node..
>
> Still thanks for the info xD
>
> D
>
> *From:* Jon Logan <jm...@buffalo.edu>
> *Sent:* Tuesday, 1 April 2014 16:43
> *To:* user@storm.incubator.apache.org
>
> All nodes.
>
>
> On Tue, Apr 1, 2014 at 11:41 AM, David Crossland <da...@elastacloud.com>wrote:
>
>> I'm just wondering about the storm.yaml settings, should these be
>> replicated to all nodes in the topology or need it only be configured on
>> the "master" node where I deploy the topology?
>>
>> thanks
>> David
>>
>>
>
Re: Curiosity
Posted by David Crossland <da...@elastacloud.com>.
That's a pain, thanks. Its either that then or configure it in code... either way I either have to recompile or touch each node..
Still thanks for the info xD
D
From: Jon Logan<ma...@buffalo.edu>
Sent: ?Tuesday?, ?1? ?April? ?2014 ?16?:?43
To: user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>
All nodes.
On Tue, Apr 1, 2014 at 11:41 AM, David Crossland <da...@elastacloud.com>> wrote:
I'm just wondering about the storm.yaml settings, should these be replicated to all nodes in the topology or need it only be configured on the "master" node where I deploy the topology?
thanks
David
Re: Curiosity
Posted by Jon Logan <jm...@buffalo.edu>.
All nodes.
On Tue, Apr 1, 2014 at 11:41 AM, David Crossland <da...@elastacloud.com>wrote:
> I'm just wondering about the storm.yaml settings, should these be
> replicated to all nodes in the topology or need it only be configured on
> the "master" node where I deploy the topology?
>
> thanks
> David
>
>