You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by sam mohel <sa...@gmail.com> on 2016/04/16 06:34:31 UTC
how can i increase size of heap
i want to increase size of heap for worker to -Xms4G how can i write it in
storm.yaml ?
and how can i set the number of CPU consuming bolts to the amount of cores
?
Thanks for any help
Re: Set worker Xmx
Posted by sam mohel <sa...@gmail.com>.
sorry message sent by mistake
On Sat, Apr 16, 2016 at 11:10 PM, sam mohel <sa...@gmail.com> wrote:
> same problem i tried it in code still read another
>
Re: Set worker Xmx
Posted by sam mohel <sa...@gmail.com>.
same problem i tried it in code still read another
Re: Set worker Xmx
Posted by "Matthias J. Sax" <mj...@apache.org>.
Do not add it to the yaml file, but set the parameter in the Config
object you hand over when submitting the topology. (Config values
overwrite yaml values)
Config c = new Config();
c.put("topology.worker.childopts", "-Xmx4g");
StormSubmitter.submitTopology("t-name", c, builder.createTopology());
-Matthias
On 04/16/2016 03:22 PM, Benjamin Cuthbert wrote:
> But does that not set it for all workers? What If I want to the specific?
>
> Regards
>> On 16 Apr 2016, at 14:19, Matthias J. Sax <mj...@apache.org> wrote:
>>
>> topology.worker.childopts
>>
>> On 04/16/2016 02:38 PM, Benjamin Cuthbert wrote:
>>> All,
>>>
>>> How do you set the individual worker Xmx settings for different topologies.
>>>
>>
>
Re: Set worker Xmx
Posted by Benjamin Cuthbert <cu...@gmail.com>.
But does that not set it for all workers? What If I want to the specific?
Regards
> On 16 Apr 2016, at 14:19, Matthias J. Sax <mj...@apache.org> wrote:
>
> topology.worker.childopts
>
> On 04/16/2016 02:38 PM, Benjamin Cuthbert wrote:
>> All,
>>
>> How do you set the individual worker Xmx settings for different topologies.
>>
>
Re: Set worker Xmx
Posted by "Matthias J. Sax" <mj...@apache.org>.
topology.worker.childopts
On 04/16/2016 02:38 PM, Benjamin Cuthbert wrote:
> All,
>
> How do you set the individual worker Xmx settings for different topologies.
>
Set worker Xmx
Posted by Benjamin Cuthbert <cu...@gmail.com>.
All,
How do you set the individual worker Xmx settings for different topologies.
Re: how can i increase size of heap
Posted by sam mohel <sa...@gmail.com>.
i restarted but still not work
On Sun, Apr 17, 2016 at 3:21 AM, Erik Weathers <ew...@groupon.com>
wrote:
> No, I don't think there's any facility for the storm daemons to re-read
> their storm.yaml config. Just restart it...
>
> On Sat, Apr 16, 2016 at 6:10 PM, sam mohel <sa...@gmail.com> wrote:
>
>> I set my configuration then run code in local using maven command . i
>> think I don't need to restart or should I ?
>>
>> When I need to change any configuration in storm. Yaml . I closed every
>> thing then change what I need then run the code . is it wrong way ? I mean
>> can I make change in configuration while the process running ?
>>
>>
>> On Sunday, April 17, 2016, Erik Weathers <ew...@groupon.com> wrote:
>>
>>> When you are running in local mode, everything runs within the same
>>> process, so you just restart the local-mode process.
>>>
>>> - Erik
>>>
>>> On Sat, Apr 16, 2016 at 5:24 PM, sam mohel <sa...@gmail.com> wrote:
>>>
>>>> thank for your replying , sorry for my questions , i'm new try to learn
>>>> , i ran the code first in local mode . it loading configurations that i can
>>>> see Terminal window , in this case how can i restart supervisor
>>>>
>>>> On Sun, Apr 17, 2016 at 1:53 AM, Erik Weathers <ew...@groupon.com>
>>>> wrote:
>>>>
>>>>> (1) You need to make sure the changes are in storm.yaml on the host(s)
>>>>> running the workers. I *assume* that you need to restart the supervisors
>>>>> to pick up the new config also, since the supervisor constructs the worker
>>>>> command. The Nimbus normally [1] doesn't send all the configs out to the
>>>>> supervisor and workers.
>>>>> (2) If you want to set it via code in your topology, then you should
>>>>> set "topology.worker.childopts", instead of "worker.childopts".
>>>>>
>>>>> [1] For storm-mesos <https://github.com/mesos/storm>, the Nimbus
>>>>> *does* send the configs to the Supervisor and thus workers.
>>>>>
>>>>> On Sat, Apr 16, 2016 at 2:10 PM, sam mohel <sa...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks for you both
>>>>>> but it didn't read what i wrote
>>>>>> i tried im storm.yaml
>>>>>> worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
>>>>>> but it read "worker.childopts" "-Xmx768m" and i noticed
>>>>>> topology.worker.childopts" nil
>>>>>>
>>>>>> what is the difference between them ?
>>>>>>
>>>>>> i tried to set it in the code by
>>>>>> conf.put("worker.childopts","-Xms4g"); but still read same size
>>>>>>
>>>>>> How can i make my change be readen as i set ?
>>>>>>
>>>>>> On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <do...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> -Xms<size> set initial Java heap size
>>>>>>> -Xmx<size> set maximum Java heap size
>>>>>>>
>>>>>>> Now you have maximum heap size 2048m. If you need to increase heap
>>>>>>> size, delete -Xmx2048m and add -Xms4G
>>>>>>>
>>>>>>> 2016-04-16 14:03 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>>>>>
>>>>>>>> i have this in storm.yaml
>>>>>>>> worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m
>>>>>>>> -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -verbose:gc
>>>>>>>> -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
>>>>>>>>
>>>>>>>> what should i write in this statement ?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Saturday, April 16, 2016, Andrey Dudin <do...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> >i want to increase size of heap for worker to -Xms4G how can i
>>>>>>>>> write it in storm.yaml ?
>>>>>>>>>
>>>>>>>>> add this option to worker.childopts
>>>>>>>>>
>>>>>>>>> 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> i want to increase size of heap for worker to -Xms4G how can i
>>>>>>>>>> write it in storm.yaml ?
>>>>>>>>>>
>>>>>>>>>> and how can i set the number of CPU consuming bolts to the
>>>>>>>>>> amount of cores ?
>>>>>>>>>>
>>>>>>>>>> Thanks for any help
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> С уважением Дудин Андрей
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> С уважением Дудин Андрей
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>
Re: how can i increase size of heap
Posted by Erik Weathers <ew...@groupon.com>.
No, I don't think there's any facility for the storm daemons to re-read
their storm.yaml config. Just restart it...
On Sat, Apr 16, 2016 at 6:10 PM, sam mohel <sa...@gmail.com> wrote:
> I set my configuration then run code in local using maven command . i
> think I don't need to restart or should I ?
>
> When I need to change any configuration in storm. Yaml . I closed every
> thing then change what I need then run the code . is it wrong way ? I mean
> can I make change in configuration while the process running ?
>
>
> On Sunday, April 17, 2016, Erik Weathers <ew...@groupon.com> wrote:
>
>> When you are running in local mode, everything runs within the same
>> process, so you just restart the local-mode process.
>>
>> - Erik
>>
>> On Sat, Apr 16, 2016 at 5:24 PM, sam mohel <sa...@gmail.com> wrote:
>>
>>> thank for your replying , sorry for my questions , i'm new try to learn
>>> , i ran the code first in local mode . it loading configurations that i can
>>> see Terminal window , in this case how can i restart supervisor
>>>
>>> On Sun, Apr 17, 2016 at 1:53 AM, Erik Weathers <ew...@groupon.com>
>>> wrote:
>>>
>>>> (1) You need to make sure the changes are in storm.yaml on the host(s)
>>>> running the workers. I *assume* that you need to restart the supervisors
>>>> to pick up the new config also, since the supervisor constructs the worker
>>>> command. The Nimbus normally [1] doesn't send all the configs out to the
>>>> supervisor and workers.
>>>> (2) If you want to set it via code in your topology, then you should
>>>> set "topology.worker.childopts", instead of "worker.childopts".
>>>>
>>>> [1] For storm-mesos <https://github.com/mesos/storm>, the Nimbus
>>>> *does* send the configs to the Supervisor and thus workers.
>>>>
>>>> On Sat, Apr 16, 2016 at 2:10 PM, sam mohel <sa...@gmail.com> wrote:
>>>>
>>>>> Thanks for you both
>>>>> but it didn't read what i wrote
>>>>> i tried im storm.yaml
>>>>> worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
>>>>> but it read "worker.childopts" "-Xmx768m" and i noticed
>>>>> topology.worker.childopts" nil
>>>>>
>>>>> what is the difference between them ?
>>>>>
>>>>> i tried to set it in the code by
>>>>> conf.put("worker.childopts","-Xms4g"); but still read same size
>>>>>
>>>>> How can i make my change be readen as i set ?
>>>>>
>>>>> On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <do...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> -Xms<size> set initial Java heap size
>>>>>> -Xmx<size> set maximum Java heap size
>>>>>>
>>>>>> Now you have maximum heap size 2048m. If you need to increase heap
>>>>>> size, delete -Xmx2048m and add -Xms4G
>>>>>>
>>>>>> 2016-04-16 14:03 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>>>>
>>>>>>> i have this in storm.yaml
>>>>>>> worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m
>>>>>>> -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -verbose:gc
>>>>>>> -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
>>>>>>>
>>>>>>> what should i write in this statement ?
>>>>>>>
>>>>>>>
>>>>>>> On Saturday, April 16, 2016, Andrey Dudin <do...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> >i want to increase size of heap for worker to -Xms4G how can i
>>>>>>>> write it in storm.yaml ?
>>>>>>>>
>>>>>>>> add this option to worker.childopts
>>>>>>>>
>>>>>>>> 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>>>>>>
>>>>>>>>> i want to increase size of heap for worker to -Xms4G how can i
>>>>>>>>> write it in storm.yaml ?
>>>>>>>>>
>>>>>>>>> and how can i set the number of CPU consuming bolts to the amount
>>>>>>>>> of cores ?
>>>>>>>>>
>>>>>>>>> Thanks for any help
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> С уважением Дудин Андрей
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> С уважением Дудин Андрей
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
Re: how can i increase size of heap
Posted by sam mohel <sa...@gmail.com>.
I set my configuration then run code in local using maven command . i think
I don't need to restart or should I ?
When I need to change any configuration in storm. Yaml . I closed every
thing then change what I need then run the code . is it wrong way ? I mean
can I make change in configuration while the process running ?
On Sunday, April 17, 2016, Erik Weathers <ew...@groupon.com> wrote:
> When you are running in local mode, everything runs within the same
> process, so you just restart the local-mode process.
>
> - Erik
>
> On Sat, Apr 16, 2016 at 5:24 PM, sam mohel <sammohel5@gmail.com
> <javascript:_e(%7B%7D,'cvml','sammohel5@gmail.com');>> wrote:
>
>> thank for your replying , sorry for my questions , i'm new try to learn ,
>> i ran the code first in local mode . it loading configurations that i can
>> see Terminal window , in this case how can i restart supervisor
>>
>> On Sun, Apr 17, 2016 at 1:53 AM, Erik Weathers <eweathers@groupon.com
>> <javascript:_e(%7B%7D,'cvml','eweathers@groupon.com');>> wrote:
>>
>>> (1) You need to make sure the changes are in storm.yaml on the host(s)
>>> running the workers. I *assume* that you need to restart the supervisors
>>> to pick up the new config also, since the supervisor constructs the worker
>>> command. The Nimbus normally [1] doesn't send all the configs out to the
>>> supervisor and workers.
>>> (2) If you want to set it via code in your topology, then you should set
>>> "topology.worker.childopts", instead of "worker.childopts".
>>>
>>> [1] For storm-mesos <https://github.com/mesos/storm>, the Nimbus *does*
>>> send the configs to the Supervisor and thus workers.
>>>
>>> On Sat, Apr 16, 2016 at 2:10 PM, sam mohel <sammohel5@gmail.com
>>> <javascript:_e(%7B%7D,'cvml','sammohel5@gmail.com');>> wrote:
>>>
>>>> Thanks for you both
>>>> but it didn't read what i wrote
>>>> i tried im storm.yaml
>>>> worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
>>>> but it read "worker.childopts" "-Xmx768m" and i noticed
>>>> topology.worker.childopts" nil
>>>>
>>>> what is the difference between them ?
>>>>
>>>> i tried to set it in the code by
>>>> conf.put("worker.childopts","-Xms4g"); but still read same size
>>>>
>>>> How can i make my change be readen as i set ?
>>>>
>>>> On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <doodin201@gmail.com
>>>> <javascript:_e(%7B%7D,'cvml','doodin201@gmail.com');>> wrote:
>>>>
>>>>> -Xms<size> set initial Java heap size
>>>>> -Xmx<size> set maximum Java heap size
>>>>>
>>>>> Now you have maximum heap size 2048m. If you need to increase heap
>>>>> size, delete -Xmx2048m and add -Xms4G
>>>>>
>>>>> 2016-04-16 14:03 GMT+03:00 sam mohel <sammohel5@gmail.com
>>>>> <javascript:_e(%7B%7D,'cvml','sammohel5@gmail.com');>>:
>>>>>
>>>>>> i have this in storm.yaml
>>>>>> worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m
>>>>>> -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -verbose:gc
>>>>>> -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
>>>>>>
>>>>>> what should i write in this statement ?
>>>>>>
>>>>>>
>>>>>> On Saturday, April 16, 2016, Andrey Dudin <doodin201@gmail.com
>>>>>> <javascript:_e(%7B%7D,'cvml','doodin201@gmail.com');>> wrote:
>>>>>>
>>>>>>> >i want to increase size of heap for worker to -Xms4G how can i
>>>>>>> write it in storm.yaml ?
>>>>>>>
>>>>>>> add this option to worker.childopts
>>>>>>>
>>>>>>> 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>>>>>
>>>>>>>> i want to increase size of heap for worker to -Xms4G how can i
>>>>>>>> write it in storm.yaml ?
>>>>>>>>
>>>>>>>> and how can i set the number of CPU consuming bolts to the amount
>>>>>>>> of cores ?
>>>>>>>>
>>>>>>>> Thanks for any help
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> С уважением Дудин Андрей
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> С уважением Дудин Андрей
>>>>>
>>>>
>>>>
>>>
>>
>
Re: how can i increase size of heap
Posted by Erik Weathers <ew...@groupon.com>.
When you are running in local mode, everything runs within the same
process, so you just restart the local-mode process.
- Erik
On Sat, Apr 16, 2016 at 5:24 PM, sam mohel <sa...@gmail.com> wrote:
> thank for your replying , sorry for my questions , i'm new try to learn ,
> i ran the code first in local mode . it loading configurations that i can
> see Terminal window , in this case how can i restart supervisor
>
> On Sun, Apr 17, 2016 at 1:53 AM, Erik Weathers <ew...@groupon.com>
> wrote:
>
>> (1) You need to make sure the changes are in storm.yaml on the host(s)
>> running the workers. I *assume* that you need to restart the supervisors
>> to pick up the new config also, since the supervisor constructs the worker
>> command. The Nimbus normally [1] doesn't send all the configs out to the
>> supervisor and workers.
>> (2) If you want to set it via code in your topology, then you should set
>> "topology.worker.childopts", instead of "worker.childopts".
>>
>> [1] For storm-mesos <https://github.com/mesos/storm>, the Nimbus *does*
>> send the configs to the Supervisor and thus workers.
>>
>> On Sat, Apr 16, 2016 at 2:10 PM, sam mohel <sa...@gmail.com> wrote:
>>
>>> Thanks for you both
>>> but it didn't read what i wrote
>>> i tried im storm.yaml
>>> worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
>>> but it read "worker.childopts" "-Xmx768m" and i noticed
>>> topology.worker.childopts" nil
>>>
>>> what is the difference between them ?
>>>
>>> i tried to set it in the code by
>>> conf.put("worker.childopts","-Xms4g"); but still read same size
>>>
>>> How can i make my change be readen as i set ?
>>>
>>> On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <do...@gmail.com>
>>> wrote:
>>>
>>>> -Xms<size> set initial Java heap size
>>>> -Xmx<size> set maximum Java heap size
>>>>
>>>> Now you have maximum heap size 2048m. If you need to increase heap
>>>> size, delete -Xmx2048m and add -Xms4G
>>>>
>>>> 2016-04-16 14:03 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>>
>>>>> i have this in storm.yaml
>>>>> worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m -XX:+PrintGCDetails
>>>>> -XX:+PrintGCTimeStamps -verbose:gc
>>>>> -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
>>>>>
>>>>> what should i write in this statement ?
>>>>>
>>>>>
>>>>> On Saturday, April 16, 2016, Andrey Dudin <do...@gmail.com> wrote:
>>>>>
>>>>>> >i want to increase size of heap for worker to -Xms4G how can i write
>>>>>> it in storm.yaml ?
>>>>>>
>>>>>> add this option to worker.childopts
>>>>>>
>>>>>> 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>>>>
>>>>>>> i want to increase size of heap for worker to -Xms4G how can i write
>>>>>>> it in storm.yaml ?
>>>>>>>
>>>>>>> and how can i set the number of CPU consuming bolts to the amount
>>>>>>> of cores ?
>>>>>>>
>>>>>>> Thanks for any help
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> С уважением Дудин Андрей
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> С уважением Дудин Андрей
>>>>
>>>
>>>
>>
>
Re: how can i increase size of heap
Posted by sam mohel <sa...@gmail.com>.
thank for your replying , sorry for my questions , i'm new try to learn , i
ran the code first in local mode . it loading configurations that i can see
Terminal window , in this case how can i restart supervisor
On Sun, Apr 17, 2016 at 1:53 AM, Erik Weathers <ew...@groupon.com>
wrote:
> (1) You need to make sure the changes are in storm.yaml on the host(s)
> running the workers. I *assume* that you need to restart the supervisors
> to pick up the new config also, since the supervisor constructs the worker
> command. The Nimbus normally [1] doesn't send all the configs out to the
> supervisor and workers.
> (2) If you want to set it via code in your topology, then you should set
> "topology.worker.childopts", instead of "worker.childopts".
>
> [1] For storm-mesos <https://github.com/mesos/storm>, the Nimbus *does*
> send the configs to the Supervisor and thus workers.
>
> On Sat, Apr 16, 2016 at 2:10 PM, sam mohel <sa...@gmail.com> wrote:
>
>> Thanks for you both
>> but it didn't read what i wrote
>> i tried im storm.yaml
>> worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
>> but it read "worker.childopts" "-Xmx768m" and i noticed
>> topology.worker.childopts" nil
>>
>> what is the difference between them ?
>>
>> i tried to set it in the code by
>> conf.put("worker.childopts","-Xms4g"); but still read same size
>>
>> How can i make my change be readen as i set ?
>>
>> On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <do...@gmail.com>
>> wrote:
>>
>>> -Xms<size> set initial Java heap size
>>> -Xmx<size> set maximum Java heap size
>>>
>>> Now you have maximum heap size 2048m. If you need to increase heap size,
>>> delete -Xmx2048m and add -Xms4G
>>>
>>> 2016-04-16 14:03 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>
>>>> i have this in storm.yaml
>>>> worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m -XX:+PrintGCDetails
>>>> -XX:+PrintGCTimeStamps -verbose:gc
>>>> -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
>>>>
>>>> what should i write in this statement ?
>>>>
>>>>
>>>> On Saturday, April 16, 2016, Andrey Dudin <do...@gmail.com> wrote:
>>>>
>>>>> >i want to increase size of heap for worker to -Xms4G how can i write
>>>>> it in storm.yaml ?
>>>>>
>>>>> add this option to worker.childopts
>>>>>
>>>>> 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>>>
>>>>>> i want to increase size of heap for worker to -Xms4G how can i write
>>>>>> it in storm.yaml ?
>>>>>>
>>>>>> and how can i set the number of CPU consuming bolts to the amount of
>>>>>> cores ?
>>>>>>
>>>>>> Thanks for any help
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> С уважением Дудин Андрей
>>>>>
>>>>
>>>
>>>
>>> --
>>> С уважением Дудин Андрей
>>>
>>
>>
>
Re: how can i increase size of heap
Posted by Erik Weathers <ew...@groupon.com>.
(1) You need to make sure the changes are in storm.yaml on the host(s)
running the workers. I *assume* that you need to restart the supervisors
to pick up the new config also, since the supervisor constructs the worker
command. The Nimbus normally [1] doesn't send all the configs out to the
supervisor and workers.
(2) If you want to set it via code in your topology, then you should set
"topology.worker.childopts", instead of "worker.childopts".
[1] For storm-mesos <https://github.com/mesos/storm>, the Nimbus *does*
send the configs to the Supervisor and thus workers.
On Sat, Apr 16, 2016 at 2:10 PM, sam mohel <sa...@gmail.com> wrote:
> Thanks for you both
> but it didn't read what i wrote
> i tried im storm.yaml
> worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
> but it read "worker.childopts" "-Xmx768m" and i noticed
> topology.worker.childopts" nil
>
> what is the difference between them ?
>
> i tried to set it in the code by
> conf.put("worker.childopts","-Xms4g"); but still read same size
>
> How can i make my change be readen as i set ?
>
> On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <do...@gmail.com> wrote:
>
>> -Xms<size> set initial Java heap size
>> -Xmx<size> set maximum Java heap size
>>
>> Now you have maximum heap size 2048m. If you need to increase heap size,
>> delete -Xmx2048m and add -Xms4G
>>
>> 2016-04-16 14:03 GMT+03:00 sam mohel <sa...@gmail.com>:
>>
>>> i have this in storm.yaml
>>> worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m -XX:+PrintGCDetails
>>> -XX:+PrintGCTimeStamps -verbose:gc
>>> -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
>>>
>>> what should i write in this statement ?
>>>
>>>
>>> On Saturday, April 16, 2016, Andrey Dudin <do...@gmail.com> wrote:
>>>
>>>> >i want to increase size of heap for worker to -Xms4G how can i write
>>>> it in storm.yaml ?
>>>>
>>>> add this option to worker.childopts
>>>>
>>>> 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>>
>>>>> i want to increase size of heap for worker to -Xms4G how can i write
>>>>> it in storm.yaml ?
>>>>>
>>>>> and how can i set the number of CPU consuming bolts to the amount of
>>>>> cores ?
>>>>>
>>>>> Thanks for any help
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> С уважением Дудин Андрей
>>>>
>>>
>>
>>
>> --
>> С уважением Дудин Андрей
>>
>
>
Re: how can i increase size of heap
Posted by sam mohel <sa...@gmail.com>.
thanks i got it , i restarted again and got the right number but from the
code .
thanks alot for you
On Sun, Apr 17, 2016 at 4:34 PM, Matthias J. Sax <mj...@apache.org> wrote:
> Do you use LocalCluster? For this, storm.yaml is ignored... If you want
> to set JVM arguments, you need to do this the same way as for any other
> regular Java program. storm.yaml is only effective for cluster setup.
>
> -Matthias
>
> On 04/17/2016 03:42 PM, sam mohel wrote:
> > Just i have one storm.yaml as i ran code in local mode "one machine" , i
> > tried also in the code but didn't work well
> >
> > On Sun, Apr 17, 2016 at 12:45 PM, Matthias J. Sax <mjsax@apache.org
> > <ma...@apache.org>> wrote:
> >
> > Which "storm.yaml" file did you change? Each supervisor has it's own
> > copy of it (at least usually; or do you have a NFS and storm.yaml is
> > shared over all nodes in the cluster?), and you need to change it in
> all
> > of them.
> >
> > Not sure, why the value of your Config object did not get picked
> up...
> > Did you try "topology.worker.childopts", too?
> >
> > -Matthias
> >
> >
> >
> > On 04/16/2016 11:10 PM, sam mohel wrote:
> > > Thanks for you both
> > > but it didn't read what i wrote
> > > i tried im storm.yaml
> > > worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
> > > but it read "worker.childopts" "-Xmx768m" and i noticed
> > > topology.worker.childopts" nil
> > >
> > > what is the difference between them ?
> > >
> > > i tried to set it in the code by
> > > conf.put("worker.childopts","-Xms4g"); but still read same size
> > >
> > > How can i make my change be readen as i set ?
> > >
> > > On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <doodin201@gmail.com
> <ma...@gmail.com>
> > > <mailto:doodin201@gmail.com <ma...@gmail.com>>> wrote:
> > >
> > > |-Xms<size>set initial Javaheap size -Xmx<size>set maximum
> > Javaheap size|
> > >
> > > Now you have maximum heap size 2048m. If you need to increase
> heap
> > > size, delete -Xmx2048m and add -Xms4G
> > >
> > > 2016-04-16 14:03 GMT+03:00 sam mohel <sammohel5@gmail.com
> <ma...@gmail.com>
> > > <mailto:sammohel5@gmail.com <ma...@gmail.com>>>:
> > >
> > > i have this in storm.yaml
> > > worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m
> > > -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -verbose:gc
> > > -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
> > >
> > > what should i write in this statement ?
> > >
> > >
> > > On Saturday, April 16, 2016, Andrey Dudin <
> doodin201@gmail.com <ma...@gmail.com>
> > > <mailto:doodin201@gmail.com <ma...@gmail.com>>>
> > wrote:
> > >
> > > >i want to increase size of heap for worker to -Xms4G
> > how can i write it in storm.yaml ?
> > >
> > > add this option to worker.childopts
> > >
> > > 2016-04-16 7:34 GMT+03:00 sam mohel
> > <sammohel5@gmail.com <ma...@gmail.com>>:
> > >
> > > i want to increase size of heap for worker to
> > -Xms4G how
> > > can i write it in storm.yaml ?
> > >
> > > and how can i set the number of CPU consuming
> > bolts to
> > > the amount of cores ?
> > >
> > > Thanks for any help
> > >
> > >
> > >
> > >
> > > --
> > > С уважением Дудин Андрей
> > >
> > >
> > >
> > >
> > > --
> > > С уважением Дудин Андрей
> > >
> > >
> >
> >
>
>
Re: how can i increase size of heap
Posted by "Matthias J. Sax" <mj...@apache.org>.
Do you use LocalCluster? For this, storm.yaml is ignored... If you want
to set JVM arguments, you need to do this the same way as for any other
regular Java program. storm.yaml is only effective for cluster setup.
-Matthias
On 04/17/2016 03:42 PM, sam mohel wrote:
> Just i have one storm.yaml as i ran code in local mode "one machine" , i
> tried also in the code but didn't work well
>
> On Sun, Apr 17, 2016 at 12:45 PM, Matthias J. Sax <mjsax@apache.org
> <ma...@apache.org>> wrote:
>
> Which "storm.yaml" file did you change? Each supervisor has it's own
> copy of it (at least usually; or do you have a NFS and storm.yaml is
> shared over all nodes in the cluster?), and you need to change it in all
> of them.
>
> Not sure, why the value of your Config object did not get picked up...
> Did you try "topology.worker.childopts", too?
>
> -Matthias
>
>
>
> On 04/16/2016 11:10 PM, sam mohel wrote:
> > Thanks for you both
> > but it didn't read what i wrote
> > i tried im storm.yaml
> > worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
> > but it read "worker.childopts" "-Xmx768m" and i noticed
> > topology.worker.childopts" nil
> >
> > what is the difference between them ?
> >
> > i tried to set it in the code by
> > conf.put("worker.childopts","-Xms4g"); but still read same size
> >
> > How can i make my change be readen as i set ?
> >
> > On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <doodin201@gmail.com <ma...@gmail.com>
> > <mailto:doodin201@gmail.com <ma...@gmail.com>>> wrote:
> >
> > |-Xms<size>set initial Javaheap size -Xmx<size>set maximum
> Javaheap size|
> >
> > Now you have maximum heap size 2048m. If you need to increase heap
> > size, delete -Xmx2048m and add -Xms4G
> >
> > 2016-04-16 14:03 GMT+03:00 sam mohel <sammohel5@gmail.com <ma...@gmail.com>
> > <mailto:sammohel5@gmail.com <ma...@gmail.com>>>:
> >
> > i have this in storm.yaml
> > worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m
> > -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -verbose:gc
> > -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
> >
> > what should i write in this statement ?
> >
> >
> > On Saturday, April 16, 2016, Andrey Dudin <doodin201@gmail.com <ma...@gmail.com>
> > <mailto:doodin201@gmail.com <ma...@gmail.com>>>
> wrote:
> >
> > >i want to increase size of heap for worker to -Xms4G
> how can i write it in storm.yaml ?
> >
> > add this option to worker.childopts
> >
> > 2016-04-16 7:34 GMT+03:00 sam mohel
> <sammohel5@gmail.com <ma...@gmail.com>>:
> >
> > i want to increase size of heap for worker to
> -Xms4G how
> > can i write it in storm.yaml ?
> >
> > and how can i set the number of CPU consuming
> bolts to
> > the amount of cores ?
> >
> > Thanks for any help
> >
> >
> >
> >
> > --
> > С уважением Дудин Андрей
> >
> >
> >
> >
> > --
> > С уважением Дудин Андрей
> >
> >
>
>
Re: how can i increase size of heap
Posted by sam mohel <sa...@gmail.com>.
Just i have one storm.yaml as i ran code in local mode "one machine" , i
tried also in the code but didn't work well
On Sun, Apr 17, 2016 at 12:45 PM, Matthias J. Sax <mj...@apache.org> wrote:
> Which "storm.yaml" file did you change? Each supervisor has it's own
> copy of it (at least usually; or do you have a NFS and storm.yaml is
> shared over all nodes in the cluster?), and you need to change it in all
> of them.
>
> Not sure, why the value of your Config object did not get picked up...
> Did you try "topology.worker.childopts", too?
>
> -Matthias
>
>
>
> On 04/16/2016 11:10 PM, sam mohel wrote:
> > Thanks for you both
> > but it didn't read what i wrote
> > i tried im storm.yaml
> > worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
> > but it read "worker.childopts" "-Xmx768m" and i noticed
> > topology.worker.childopts" nil
> >
> > what is the difference between them ?
> >
> > i tried to set it in the code by
> > conf.put("worker.childopts","-Xms4g"); but still read same size
> >
> > How can i make my change be readen as i set ?
> >
> > On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <doodin201@gmail.com
> > <ma...@gmail.com>> wrote:
> >
> > |-Xms<size>set initial Javaheap size -Xmx<size>set maximum Javaheap
> size|
> >
> > Now you have maximum heap size 2048m. If you need to increase heap
> > size, delete -Xmx2048m and add -Xms4G
> >
> > 2016-04-16 14:03 GMT+03:00 sam mohel <sammohel5@gmail.com
> > <ma...@gmail.com>>:
> >
> > i have this in storm.yaml
> > worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m
> > -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -verbose:gc
> > -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
> >
> > what should i write in this statement ?
> >
> >
> > On Saturday, April 16, 2016, Andrey Dudin <doodin201@gmail.com
> > <ma...@gmail.com>> wrote:
> >
> > >i want to increase size of heap for worker to -Xms4G how
> can i write it in storm.yaml ?
> >
> > add this option to worker.childopts
> >
> > 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
> >
> > i want to increase size of heap for worker to -Xms4G how
> > can i write it in storm.yaml ?
> >
> > and how can i set the number of CPU consuming bolts to
> > the amount of cores ?
> >
> > Thanks for any help
> >
> >
> >
> >
> > --
> > С уважением Дудин Андрей
> >
> >
> >
> >
> > --
> > С уважением Дудин Андрей
> >
> >
>
>
Re: how can i increase size of heap
Posted by "Matthias J. Sax" <mj...@apache.org>.
Which "storm.yaml" file did you change? Each supervisor has it's own
copy of it (at least usually; or do you have a NFS and storm.yaml is
shared over all nodes in the cluster?), and you need to change it in all
of them.
Not sure, why the value of your Config object did not get picked up...
Did you try "topology.worker.childopts", too?
-Matthias
On 04/16/2016 11:10 PM, sam mohel wrote:
> Thanks for you both
> but it didn't read what i wrote
> i tried im storm.yaml
> worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
> but it read "worker.childopts" "-Xmx768m" and i noticed
> topology.worker.childopts" nil
>
> what is the difference between them ?
>
> i tried to set it in the code by
> conf.put("worker.childopts","-Xms4g"); but still read same size
>
> How can i make my change be readen as i set ?
>
> On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <doodin201@gmail.com
> <ma...@gmail.com>> wrote:
>
> |-Xms<size>set initial Javaheap size -Xmx<size>set maximum Javaheap size|
>
> Now you have maximum heap size 2048m. If you need to increase heap
> size, delete -Xmx2048m and add -Xms4G
>
> 2016-04-16 14:03 GMT+03:00 sam mohel <sammohel5@gmail.com
> <ma...@gmail.com>>:
>
> i have this in storm.yaml
> worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m
> -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -verbose:gc
> -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
>
> what should i write in this statement ?
>
>
> On Saturday, April 16, 2016, Andrey Dudin <doodin201@gmail.com
> <ma...@gmail.com>> wrote:
>
> >i want to increase size of heap for worker to -Xms4G how can i write it in storm.yaml ?
>
> add this option to worker.childopts
>
> 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
>
> i want to increase size of heap for worker to -Xms4G how
> can i write it in storm.yaml ?
>
> and how can i set the number of CPU consuming bolts to
> the amount of cores ?
>
> Thanks for any help
>
>
>
>
> --
> С уважением Дудин Андрей
>
>
>
>
> --
> С уважением Дудин Андрей
>
>
Re: how can i increase size of heap
Posted by sam mohel <sa...@gmail.com>.
Thanks for you both
but it didn't read what i wrote
i tried im storm.yaml
worker.childopts: "-Xms4g -Djava.net.preferIPv4Stack=true"
but it read "worker.childopts" "-Xmx768m" and i noticed
topology.worker.childopts" nil
what is the difference between them ?
i tried to set it in the code by
conf.put("worker.childopts","-Xms4g"); but still read same size
How can i make my change be readen as i set ?
On Sat, Apr 16, 2016 at 2:37 PM, Andrey Dudin <do...@gmail.com> wrote:
> -Xms<size> set initial Java heap size
> -Xmx<size> set maximum Java heap size
>
> Now you have maximum heap size 2048m. If you need to increase heap size,
> delete -Xmx2048m and add -Xms4G
>
> 2016-04-16 14:03 GMT+03:00 sam mohel <sa...@gmail.com>:
>
>> i have this in storm.yaml
>> worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m -XX:+PrintGCDetails
>> -XX:+PrintGCTimeStamps -verbose:gc
>> -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
>>
>> what should i write in this statement ?
>>
>>
>> On Saturday, April 16, 2016, Andrey Dudin <do...@gmail.com> wrote:
>>
>>> >i want to increase size of heap for worker to -Xms4G how can i write it
>>> in storm.yaml ?
>>>
>>> add this option to worker.childopts
>>>
>>> 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
>>>
>>>> i want to increase size of heap for worker to -Xms4G how can i write it
>>>> in storm.yaml ?
>>>>
>>>> and how can i set the number of CPU consuming bolts to the amount of
>>>> cores ?
>>>>
>>>> Thanks for any help
>>>>
>>>
>>>
>>>
>>> --
>>> С уважением Дудин Андрей
>>>
>>
>
>
> --
> С уважением Дудин Андрей
>
Re: how can i increase size of heap
Posted by Andrey Dudin <do...@gmail.com>.
-Xms<size> set initial Java heap size
-Xmx<size> set maximum Java heap size
Now you have maximum heap size 2048m. If you need to increase heap size,
delete -Xmx2048m and add -Xms4G
2016-04-16 14:03 GMT+03:00 sam mohel <sa...@gmail.com>:
> i have this in storm.yaml
> worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m -XX:+PrintGCDetails
> -XX:+PrintGCTimeStamps -verbose:gc
> -Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
>
> what should i write in this statement ?
>
>
> On Saturday, April 16, 2016, Andrey Dudin <do...@gmail.com> wrote:
>
>> >i want to increase size of heap for worker to -Xms4G how can i write it
>> in storm.yaml ?
>>
>> add this option to worker.childopts
>>
>> 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
>>
>>> i want to increase size of heap for worker to -Xms4G how can i write it
>>> in storm.yaml ?
>>>
>>> and how can i set the number of CPU consuming bolts to the amount of
>>> cores ?
>>>
>>> Thanks for any help
>>>
>>
>>
>>
>> --
>> С уважением Дудин Андрей
>>
>
--
С уважением Дудин Андрей
Re: how can i increase size of heap
Posted by sam mohel <sa...@gmail.com>.
i have this in storm.yaml
worker.childopts: "-Xmx2048m -XX:MaxPermSize=256m -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -verbose:gc
-Xloggc:/usr/local/storm/logs/gc-storm-worker-%6700%.log"
what should i write in this statement ?
On Saturday, April 16, 2016, Andrey Dudin <do...@gmail.com> wrote:
> >i want to increase size of heap for worker to -Xms4G how can i write it
> in storm.yaml ?
>
> add this option to worker.childopts
>
> 2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
>
>> i want to increase size of heap for worker to -Xms4G how can i write it
>> in storm.yaml ?
>>
>> and how can i set the number of CPU consuming bolts to the amount of
>> cores ?
>>
>> Thanks for any help
>>
>
>
>
> --
> С уважением Дудин Андрей
>
Re: how can i increase size of heap
Posted by Andrey Dudin <do...@gmail.com>.
>i want to increase size of heap for worker to -Xms4G how can i write it in
storm.yaml ?
add this option to worker.childopts
2016-04-16 7:34 GMT+03:00 sam mohel <sa...@gmail.com>:
> i want to increase size of heap for worker to -Xms4G how can i write it in
> storm.yaml ?
>
> and how can i set the number of CPU consuming bolts to the amount of
> cores ?
>
> Thanks for any help
>
--
С уважением Дудин Андрей
Re: how can i increase size of heap
Posted by "Matthias J. Sax" <mj...@apache.org>.
1) yes, this is correct (just as you do it on command line, ie, multiple
flags separated by blank)
worker childopts : "-Xms4g -Djava.net.preferIPv4Stack=true"
2) still nor sure what you mean... if you get GC overhead limit exceed
it means that the GC tries to clean up objects to free memory, but no
objects could be deleted (and GC tried multiple time without success).
This is a Java thing and not directly related to Storm. You should be
able to tackle it, by increasing the JVM memory, as you already do by (1) :)
-Matthias
On 04/16/2016 01:29 PM, sam mohel wrote:
> thanks but what if i want to write this also in worker childopts :
> "-Djava.net.preferIPv4Stack=true"
>
> is it will be like
>
> worker childopts : "-Xms4g -Djava.net.preferIPv4Stack=true" ?
>
> for Second question i got this statemnent by searching for my problem with
> GC overhead limited exceed but didn't know how can i do it ?
>
>
> On Sat, Apr 16, 2016 at 1:22 PM, Matthias J. Sax <mj...@apache.org> wrote:
>
>> use parameter worker.childopts
>>
>> worker.childopts: "-Xms4g"
>>
>> Not sure what you mean by your second question...
>>
>> -Matthias
>>
>> On 04/16/2016 06:34 AM, sam mohel wrote:
>>> i want to increase size of heap for worker to -Xms4G how can i write it
>> in
>>> storm.yaml ?
>>>
>>> and how can i set the number of CPU consuming bolts to the amount of
>> cores
>>> ?
>>>
>>> Thanks for any help
>>>
>>
>>
>
Re: how can i increase size of heap
Posted by sam mohel <sa...@gmail.com>.
thanks but what if i want to write this also in worker childopts :
"-Djava.net.preferIPv4Stack=true"
is it will be like
worker childopts : "-Xms4g -Djava.net.preferIPv4Stack=true" ?
for Second question i got this statemnent by searching for my problem with
GC overhead limited exceed but didn't know how can i do it ?
On Sat, Apr 16, 2016 at 1:22 PM, Matthias J. Sax <mj...@apache.org> wrote:
> use parameter worker.childopts
>
> worker.childopts: "-Xms4g"
>
> Not sure what you mean by your second question...
>
> -Matthias
>
> On 04/16/2016 06:34 AM, sam mohel wrote:
> > i want to increase size of heap for worker to -Xms4G how can i write it
> in
> > storm.yaml ?
> >
> > and how can i set the number of CPU consuming bolts to the amount of
> cores
> > ?
> >
> > Thanks for any help
> >
>
>
Re: how can i increase size of heap
Posted by "Matthias J. Sax" <mj...@apache.org>.
use parameter worker.childopts
worker.childopts: "-Xms4g"
Not sure what you mean by your second question...
-Matthias
On 04/16/2016 06:34 AM, sam mohel wrote:
> i want to increase size of heap for worker to -Xms4G how can i write it in
> storm.yaml ?
>
> and how can i set the number of CPU consuming bolts to the amount of cores
> ?
>
> Thanks for any help
>