You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Wuyang Zhang <wu...@winlab.rutgers.edu> on 2018/02/10 03:44:13 UTC
Resource Aware Scheduler Kills Nimbus
Hi,
I have the difficulty with the resource aware scheduler from the version
1.1.1.
As suggested by the website, I add the following lines to storm.yaml:
*storm.scheduler:
“org.apache.storm.scheduler.resource.ResourceAwareScheduler”*
* supervisor.memory.capacity.mb: 2048.0*
* supervisor.cpu.capacity: 400.0*
Afterwards, I rerun the storm and the nimbus process has been killed.
The last log I found from nimbus.log is:
o.a.s.d.nimbus main [INFO] Using custom scheduler:
“org.apache.storm.scheduler.resource.ResourceAwareScheduler”.
I suppose something goes wrong when it runs the scheduler.
Do I miss anything to enable this scheduler?
Best,
Wuyang
ᐧ
Re: Resource Aware Scheduler Kills Nimbus
Posted by Wuyang Zhang <wu...@winlab.rutgers.edu>.
Hi Alessio,
Thank you for your reply. I think the storm community can consider to
introduce some automatic resource analysis mechanisms to set up those
values. The resource-aware scheduling algorithm itself is great, while it
may not perform well without accurately setting the task resource demands.
Cheers,
Wuyang
ᐧ
On Mon, Feb 19, 2018 at 11:43 AM, Alessio Pagliari <pa...@i3s.unice.fr>
wrote:
> Hi Wuyang,
>
> This is a nice question, I didn’t find anything that computes it
> automatically, yet. By now you have to test your topology tuning the
> resource requirements until you don’t find pleasing values.
>
> Cheers,
>
> Alessio
>
> On 10 Feb 2018, at 21:42, Wuyang Zhang <wu...@winlab.rutgers.edu> wrote:
>
> Hi Jerry,
>
> I am not familiar with the pull operation on the Storm website... Can you
> do it when you get a chance?
>
> Also, I wonder how can I accurately specify the resource requirement,
> especially the cpu demand, for each task?
>
> Best,
> Wuyang
> ᐧ
>
> On Sat, Feb 10, 2018 at 3:21 PM, Jerry Peng <je...@gmail.com>
> wrote:
>
>> Yup, an error in the documentation
>>
>> Feel free to submit a pull request to fix the documentation.
>>
>> Best,
>>
>> Jerry
>>
>> On Sat, Feb 10, 2018 at 1:42 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
>> wrote:
>>
>>> Ok, I find it works by adding
>>> storm.scheduler: org.apache.storm.scheduler.res
>>> ource.ResourceAwareScheduler
>>>
>>> instead of adding
>>>
>>> storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
>>>
>>>
>>>
>>> suggested by the website..
>>>
>>> ᐧ
>>>
>>> On Sat, Feb 10, 2018 at 2:22 PM, Wuyang Zhang <wuyang@winlab.rutgers.edu
>>> > wrote:
>>>
>>>> When I redo the procedure, here is the more log
>>>>
>>>> *2018-02-10 14:10:32.791 o.a.s.d.nimbus main [INFO] Using custom
>>>> scheduler: “org.apache.storm.scheduler.re
>>>> <http://org.apache.storm.scheduler.re/>source.ResourceAwareScheduler”*
>>>> *2018-02-10 14:10:32.793 o.a.s.d.nimbus main [ERROR] Error on
>>>> initialization of server service-handler*
>>>> *java.lang.ClassNotFoundException: “org.apache.storm.scheduler.re
>>>> <http://org.apache.storm.scheduler.re/>source.ResourceAwareScheduler”*
>>>> * at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>> ~[?:1.8.0_151]*
>>>> * at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>> ~[?:1.8.0_151]*
>>>> * at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>> ~[?:1.8.0_151]*
>>>> * at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>> ~[?:1.8.0_151]*
>>>> * at java.lang.Class.forName0(Native Method) ~[?:1.8.0_151]*
>>>> * at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_151]*
>>>> * at org.apache.storm.util$new_instance.invoke(util.clj:1027)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at org.apache.storm.daemon.nimbus$mk_scheduler.invoke(nimbus.cl
>>>> <http://nimbus.cl/>j:127) ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at org.apache.storm.daemon.nimbus$nimbus_data.invoke(nimbus.clj:215)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at
>>>> org.apache.storm.daemon.nimbus$fn__11007$exec_fn__1370__auto____11008.invoke(nimbus.clj:2451)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at clojure.lang.AFn.applyToHelper(AFn.java:156)
>>>> ~[clojure-1.7.0.jar:?]*
>>>> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
>>>> * at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]*
>>>> * at
>>>> org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
>>>> * at
>>>> org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at clojure.lang.AFn.applyToHelper(AFn.java:152)
>>>> ~[clojure-1.7.0.jar:?]*
>>>> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
>>>> * at org.apache.storm.daemon.nimbus.main(Unknown Source)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> *2018-02-10 14:10:32.806 o.a.s.util main [ERROR] Halting process:
>>>> ("Error on initialization")*
>>>> *java.lang.RuntimeException: ("Error on initialization")*
>>>> * at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at clojure.lang.RestFn.invoke(RestFn.java:423) ~[clojure-1.7.0.jar:?]*
>>>> * at
>>>> org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
>>>> * at
>>>> org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>> * at clojure.lang.AFn.applyToHelper(AFn.java:152)
>>>> ~[clojure-1.7.0.jar:?]*
>>>> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
>>>> * at org.apache.storm.daemon.nimbus.main(Unknown Source)
>>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>>
>>>> I download a binary version of Storm 1.1.1 from
>>>> http://www.apache.org/dyn/closer.lua/storm/apache-storm
>>>> -1.1.1/apache-storm-1.1.1.tar.gz.
>>>> ᐧ
>>>>
>>>> On Sat, Feb 10, 2018 at 2:02 PM, Wuyang Zhang <
>>>> wuyang@winlab.rutgers.edu> wrote:
>>>>
>>>>> Hi Jerry,
>>>>>
>>>>> I just retry to use the resource aware scheduler in a single node with
>>>>> 4 slots without any further settings. The scheduler still kills the nimbus
>>>>> process. Please find nimbus.log and storm.yaml attached.
>>>>>
>>>>> Without add the line
>>>>>
>>>>> storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
>>>>>
>>>>> in storm.yaml, everything works fine.
>>>>>
>>>>> I also tried
>>>>>
>>>>> conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class);
>>>>>
>>>>> The cluster can run, but I cannot see any log related to
>>>>> DefaultResourceAware. It seems like that it doesn't really run.
>>>>>
>>>>> Please take a look at the log and see if I miss anything to enable
>>>>> them to work?
>>>>>
>>>>> Best,
>>>>> Wuyang
>>>>> ᐧ
>>>>>
>>>>> On Sat, Feb 10, 2018 at 12:23 PM, Zhang Wuyang <
>>>>> qingshanyouyou@gmail.com> wrote:
>>>>>
>>>>>> Hi Jerry,
>>>>>>
>>>>>> Thank you for the reply.
>>>>>>
>>>>>> Please find the storm.yaml below:
>>>>>>
>>>>>> ########### These MUST be filled in for a storm configuration
>>>>>> storm.zookeeper.servers:
>>>>>> - "10.10.21.1"
>>>>>> - "10.10.21.2"
>>>>>> - "10.10.21.3"
>>>>>> - "10.10.21.4"
>>>>>> - "10.10.21.5"
>>>>>> - "10.10.21.6"
>>>>>> - "10.10.21.7"
>>>>>>
>>>>>> nimbus.seeds: ["10.10.21.1"]
>>>>>>
>>>>>> storm.zookeeper.port: 2181
>>>>>> ###### These may optionally be filled in:
>>>>>>
>>>>>> worker.heap.memory.mb: 8192
>>>>>>
>>>>>> supervisor.scheduler.meta:
>>>>>> tag: 10.10.20.1
>>>>>>
>>>>>> #storm.scheduler: storm.EmptyScheduler
>>>>>> storm.scheduler: “org.apache.storm.scheduler.re
>>>>>> source.ResourceAwareScheduler”
>>>>>> supervisor.memory.capacity.mb: 2048.0
>>>>>> supervisor.cpu.capacity: 4000.0
>>>>>> supervisor.slots.ports:
>>>>>> - 6700
>>>>>> - 6701
>>>>>> - 6702
>>>>>> - 6703
>>>>>>
>>>>>> worker.childopts: "-Xmx%HEAP-MEM%m -Xms6000m -XX:+PrintGCDetails
>>>>>> -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps
>>>>>> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M
>>>>>> -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump"
>>>>>>
>>>>>> On Sat, Feb 10, 2018 at 12:16 PM, Jerry Peng <
>>>>>> jerry.boyang.peng@gmail.com> wrote:
>>>>>>
>>>>>>> Wuyang,
>>>>>>>
>>>>>>> Can you provide your whole storm.yaml ?
>>>>>>>
>>>>>>> Best,
>>>>>>>
>>>>>>> Jerry
>>>>>>>
>>>>>>> On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <
>>>>>>> wuyang@winlab.rutgers.edu> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have the difficulty with the resource aware scheduler from the
>>>>>>>> version 1.1.1.
>>>>>>>>
>>>>>>>> As suggested by the website, I add the following lines to
>>>>>>>> storm.yaml:
>>>>>>>>
>>>>>>>> *storm.scheduler: “org.apache.storm.scheduler.re
>>>>>>>> <http://org.apache.storm.scheduler.re/>source.ResourceAwareScheduler”*
>>>>>>>> * supervisor.memory.capacity.mb: 2048.0*
>>>>>>>> * supervisor.cpu.capacity: 400.0*
>>>>>>>>
>>>>>>>> Afterwards, I rerun the storm and the nimbus process has been
>>>>>>>> killed.
>>>>>>>> The last log I found from nimbus.log is:
>>>>>>>> o.a.s.d.nimbus main [INFO] Using custom scheduler: “
>>>>>>>> org.apache.storm.scheduler.resource.ResourceAwareScheduler”.
>>>>>>>>
>>>>>>>> I suppose something goes wrong when it runs the scheduler.
>>>>>>>>
>>>>>>>> Do I miss anything to enable this scheduler?
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Wuyang
>>>>>>>> ᐧ
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
Re: Resource Aware Scheduler Kills Nimbus
Posted by Alessio Pagliari <pa...@i3s.unice.fr>.
Hi Wuyang,
This is a nice question, I didn’t find anything that computes it automatically, yet. By now you have to test your topology tuning the resource requirements until you don’t find pleasing values.
Cheers,
Alessio
> On 10 Feb 2018, at 21:42, Wuyang Zhang <wu...@winlab.rutgers.edu> wrote:
>
> Hi Jerry,
>
> I am not familiar with the pull operation on the Storm website... Can you do it when you get a chance?
>
> Also, I wonder how can I accurately specify the resource requirement, especially the cpu demand, for each task?
>
> Best,
> Wuyang
> ᐧ
>
> On Sat, Feb 10, 2018 at 3:21 PM, Jerry Peng <jerry.boyang.peng@gmail.com <ma...@gmail.com>> wrote:
> Yup, an error in the documentation
>
> Feel free to submit a pull request to fix the documentation.
>
> Best,
>
> Jerry
>
> On Sat, Feb 10, 2018 at 1:42 PM, Wuyang Zhang <wuyang@winlab.rutgers.edu <ma...@winlab.rutgers.edu>> wrote:
> Ok, I find it works by adding
> storm.scheduler: org.apache.storm.scheduler.resource.ResourceAwareScheduler
>
> instead of adding
>
> storm.scheduler: “org.apache.storm.scheduler.re <http://org.apache.storm.scheduler.re/>source.ResourceAwareScheduler”
>
>
> suggested by the website..
>
> ᐧ
>
> On Sat, Feb 10, 2018 at 2:22 PM, Wuyang Zhang <wuyang@winlab.rutgers.edu <ma...@winlab.rutgers.edu>> wrote:
> When I redo the procedure, here is the more log
>
> 2018-02-10 14:10:32.791 o.a.s.d.nimbus main [INFO] Using custom scheduler: “org.apache.storm.scheduler.re <http://org.apache.storm.scheduler.re/>source.ResourceAwareScheduler”
> 2018-02-10 14:10:32.793 o.a.s.d.nimbus main [ERROR] Error on initialization of server service-handler
> java.lang.ClassNotFoundException: “org.apache.storm.scheduler.re <http://org.apache.storm.scheduler.re/>source.ResourceAwareScheduler”
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_151]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_151]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[?:1.8.0_151]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_151]
> at java.lang.Class.forName0(Native Method) ~[?:1.8.0_151]
> at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_151]
> at org.apache.storm.util$new_instance.invoke(util.clj:1027) ~[storm-core-1.1.1.jar:1.1.1]
> at org.apache.storm.daemon.nimbus$mk_scheduler.invoke(nimbus.cl <http://nimbus.cl/>j:127) ~[storm-core-1.1.1.jar:1.1.1]
> at org.apache.storm.daemon.nimbus$nimbus_data.invoke(nimbus.clj:215) ~[storm-core-1.1.1.jar:1.1.1]
> at org.apache.storm.daemon.nimbus$fn__11007$exec_fn__1370__auto____11008.invoke(nimbus.clj:2451) ~[storm-core-1.1.1.jar:1.1.1]
> at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[clojure-1.7.0.jar:?]
> at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
> at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448) ~[storm-core-1.1.1.jar:1.1.1]
> at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536) ~[storm-core-1.1.1.jar:1.1.1]
> at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569) ~[storm-core-1.1.1.jar:1.1.1]
> at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592) ~[storm-core-1.1.1.jar:1.1.1]
> at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]
> at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.nimbus.main(Unknown Source) ~[storm-core-1.1.1.jar:1.1.1]
> 2018-02-10 14:10:32.806 o.a.s.util main [ERROR] Halting process: ("Error on initialization")
> java.lang.RuntimeException: ("Error on initialization")
> at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) ~[storm-core-1.1.1.jar:1.1.1]
> at clojure.lang.RestFn.invoke(RestFn.java:423) ~[clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448) ~[storm-core-1.1.1.jar:1.1.1]
> at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536) ~[storm-core-1.1.1.jar:1.1.1]
> at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569) ~[storm-core-1.1.1.jar:1.1.1]
> at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592) ~[storm-core-1.1.1.jar:1.1.1]
> at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]
> at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.nimbus.main(Unknown Source) ~[storm-core-1.1.1.jar:1.1.1]
>
> I download a binary version of Storm 1.1.1 from http://www.apache.org/dyn/closer.lua/storm/apache-storm-1.1.1/apache-storm-1.1.1.tar.gz <http://www.apache.org/dyn/closer.lua/storm/apache-storm-1.1.1/apache-storm-1.1.1.tar.gz>.
> ᐧ
>
> On Sat, Feb 10, 2018 at 2:02 PM, Wuyang Zhang <wuyang@winlab.rutgers.edu <ma...@winlab.rutgers.edu>> wrote:
> Hi Jerry,
>
> I just retry to use the resource aware scheduler in a single node with 4 slots without any further settings. The scheduler still kills the nimbus process. Please find nimbus.log and storm.yaml attached.
>
> Without add the line
> storm.scheduler: “org.apache.storm.scheduler.re <http://org.apache.storm.scheduler.re/>source.ResourceAwareScheduler”
> in storm.yaml, everything works fine.
>
> I also tried
> conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class);
> The cluster can run, but I cannot see any log related to DefaultResourceAware. It seems like that it doesn't really run.
>
> Please take a look at the log and see if I miss anything to enable them to work?
>
> Best,
> Wuyang
> ᐧ
>
> On Sat, Feb 10, 2018 at 12:23 PM, Zhang Wuyang <qingshanyouyou@gmail.com <ma...@gmail.com>> wrote:
> Hi Jerry,
>
> Thank you for the reply.
>
> Please find the storm.yaml below:
>
> ########### These MUST be filled in for a storm configuration
> storm.zookeeper.servers:
> - "10.10.21.1"
> - "10.10.21.2"
> - "10.10.21.3"
> - "10.10.21.4"
> - "10.10.21.5"
> - "10.10.21.6"
> - "10.10.21.7"
>
> nimbus.seeds: ["10.10.21.1"]
>
> storm.zookeeper.port: 2181
> ###### These may optionally be filled in:
>
> worker.heap.memory.mb: 8192
>
> supervisor.scheduler.meta:
> tag: 10.10.20.1
>
> #storm.scheduler: storm.EmptyScheduler
> storm.scheduler: “org.apache.storm.scheduler.re <http://org.apache.storm.scheduler.re/>source.ResourceAwareScheduler”
> supervisor.memory.capacity.mb: 2048.0
> supervisor.cpu.capacity: 4000.0
> supervisor.slots.ports:
> - 6700
> - 6701
> - 6702
> - 6703
>
> worker.childopts: "-Xmx%HEAP-MEM%m -Xms6000m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump"
>
> On Sat, Feb 10, 2018 at 12:16 PM, Jerry Peng <jerry.boyang.peng@gmail.com <ma...@gmail.com>> wrote:
> Wuyang,
>
> Can you provide your whole storm.yaml ?
>
> Best,
>
> Jerry
>
> On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <wuyang@winlab.rutgers.edu <ma...@winlab.rutgers.edu>> wrote:
> Hi,
>
> I have the difficulty with the resource aware scheduler from the version 1.1.1.
>
> As suggested by the website, I add the following lines to storm.yaml:
>
> storm.scheduler: “org.apache.storm.scheduler.re <http://org.apache.storm.scheduler.re/>source.ResourceAwareScheduler”
> supervisor.memory.capacity.mb: 2048.0
> supervisor.cpu.capacity: 400.0
>
> Afterwards, I rerun the storm and the nimbus process has been killed.
> The last log I found from nimbus.log is:
> o.a.s.d.nimbus main [INFO] Using custom scheduler: “org.apache.storm.scheduler.re <http://org.apache.storm.scheduler.re/>source.ResourceAwareScheduler”.
>
> I suppose something goes wrong when it runs the scheduler.
>
> Do I miss anything to enable this scheduler?
>
> Best,
> Wuyang
> ᐧ
>
>
>
>
>
>
>
Re: Resource Aware Scheduler Kills Nimbus
Posted by Wuyang Zhang <wu...@winlab.rutgers.edu>.
Hi Jerry,
I am not familiar with the pull operation on the Storm website... Can you
do it when you get a chance?
Also, I wonder how can I accurately specify the resource requirement,
especially the cpu demand, for each task?
Best,
Wuyang
ᐧ
On Sat, Feb 10, 2018 at 3:21 PM, Jerry Peng <je...@gmail.com>
wrote:
> Yup, an error in the documentation
>
> Feel free to submit a pull request to fix the documentation.
>
> Best,
>
> Jerry
>
> On Sat, Feb 10, 2018 at 1:42 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
> wrote:
>
>> Ok, I find it works by adding
>> storm.scheduler: org.apache.storm.scheduler.res
>> ource.ResourceAwareScheduler
>>
>> instead of adding
>>
>> storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
>>
>> suggested by the website..
>>
>> ᐧ
>>
>> On Sat, Feb 10, 2018 at 2:22 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
>> wrote:
>>
>>> When I redo the procedure, here is the more log
>>>
>>> *2018-02-10 14:10:32.791 o.a.s.d.nimbus main [INFO] Using custom
>>> scheduler: “org.apache.storm.scheduler.re
>>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>>> *2018-02-10 14:10:32.793 o.a.s.d.nimbus main [ERROR] Error on
>>> initialization of server service-handler*
>>> *java.lang.ClassNotFoundException: “org.apache.storm.scheduler.re
>>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>>> * at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>> ~[?:1.8.0_151]*
>>> * at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>> ~[?:1.8.0_151]*
>>> * at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>> ~[?:1.8.0_151]*
>>> * at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>> ~[?:1.8.0_151]*
>>> * at java.lang.Class.forName0(Native Method) ~[?:1.8.0_151]*
>>> * at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_151]*
>>> * at org.apache.storm.util$new_instance.invoke(util.clj:1027)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at org.apache.storm.daemon.nimbus$mk_scheduler.invoke(nimbus.cl
>>> <http://nimbus.cl>j:127) ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at org.apache.storm.daemon.nimbus$nimbus_data.invoke(nimbus.clj:215)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at
>>> org.apache.storm.daemon.nimbus$fn__11007$exec_fn__1370__auto____11008.invoke(nimbus.clj:2451)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[clojure-1.7.0.jar:?]*
>>> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
>>> * at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]*
>>> * at
>>> org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
>>> * at
>>> org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]*
>>> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
>>> * at org.apache.storm.daemon.nimbus.main(Unknown Source)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> *2018-02-10 14:10:32.806 o.a.s.util main [ERROR] Halting process:
>>> ("Error on initialization")*
>>> *java.lang.RuntimeException: ("Error on initialization")*
>>> * at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at clojure.lang.RestFn.invoke(RestFn.java:423) ~[clojure-1.7.0.jar:?]*
>>> * at
>>> org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
>>> * at
>>> org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>> * at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]*
>>> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
>>> * at org.apache.storm.daemon.nimbus.main(Unknown Source)
>>> ~[storm-core-1.1.1.jar:1.1.1]*
>>>
>>> I download a binary version of Storm 1.1.1 from
>>> http://www.apache.org/dyn/closer.lua/storm/apache-storm
>>> -1.1.1/apache-storm-1.1.1.tar.gz.
>>> ᐧ
>>>
>>> On Sat, Feb 10, 2018 at 2:02 PM, Wuyang Zhang <wuyang@winlab.rutgers.edu
>>> > wrote:
>>>
>>>> Hi Jerry,
>>>>
>>>> I just retry to use the resource aware scheduler in a single node with
>>>> 4 slots without any further settings. The scheduler still kills the nimbus
>>>> process. Please find nimbus.log and storm.yaml attached.
>>>>
>>>> Without add the line
>>>>
>>>> storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
>>>>
>>>> in storm.yaml, everything works fine.
>>>>
>>>> I also tried
>>>>
>>>> conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class);
>>>>
>>>> The cluster can run, but I cannot see any log related to
>>>> DefaultResourceAware. It seems like that it doesn't really run.
>>>>
>>>> Please take a look at the log and see if I miss anything to enable them
>>>> to work?
>>>>
>>>> Best,
>>>> Wuyang
>>>> ᐧ
>>>>
>>>> On Sat, Feb 10, 2018 at 12:23 PM, Zhang Wuyang <
>>>> qingshanyouyou@gmail.com> wrote:
>>>>
>>>>> Hi Jerry,
>>>>>
>>>>> Thank you for the reply.
>>>>>
>>>>> Please find the storm.yaml below:
>>>>>
>>>>> ########### These MUST be filled in for a storm configuration
>>>>> storm.zookeeper.servers:
>>>>> - "10.10.21.1"
>>>>> - "10.10.21.2"
>>>>> - "10.10.21.3"
>>>>> - "10.10.21.4"
>>>>> - "10.10.21.5"
>>>>> - "10.10.21.6"
>>>>> - "10.10.21.7"
>>>>>
>>>>> nimbus.seeds: ["10.10.21.1"]
>>>>>
>>>>> storm.zookeeper.port: 2181
>>>>> ###### These may optionally be filled in:
>>>>>
>>>>> worker.heap.memory.mb: 8192
>>>>>
>>>>> supervisor.scheduler.meta:
>>>>> tag: 10.10.20.1
>>>>>
>>>>> #storm.scheduler: storm.EmptyScheduler
>>>>> storm.scheduler: “org.apache.storm.scheduler.re
>>>>> source.ResourceAwareScheduler”
>>>>> supervisor.memory.capacity.mb: 2048.0
>>>>> supervisor.cpu.capacity: 4000.0
>>>>> supervisor.slots.ports:
>>>>> - 6700
>>>>> - 6701
>>>>> - 6702
>>>>> - 6703
>>>>>
>>>>> worker.childopts: "-Xmx%HEAP-MEM%m -Xms6000m -XX:+PrintGCDetails
>>>>> -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps
>>>>> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M
>>>>> -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump"
>>>>>
>>>>> On Sat, Feb 10, 2018 at 12:16 PM, Jerry Peng <
>>>>> jerry.boyang.peng@gmail.com> wrote:
>>>>>
>>>>>> Wuyang,
>>>>>>
>>>>>> Can you provide your whole storm.yaml ?
>>>>>>
>>>>>> Best,
>>>>>>
>>>>>> Jerry
>>>>>>
>>>>>> On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <
>>>>>> wuyang@winlab.rutgers.edu> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have the difficulty with the resource aware scheduler from the
>>>>>>> version 1.1.1.
>>>>>>>
>>>>>>> As suggested by the website, I add the following lines to storm.yaml:
>>>>>>>
>>>>>>> *storm.scheduler: “org.apache.storm.scheduler.re
>>>>>>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>>>>>>> * supervisor.memory.capacity.mb: 2048.0*
>>>>>>> * supervisor.cpu.capacity: 400.0*
>>>>>>>
>>>>>>> Afterwards, I rerun the storm and the nimbus process has been
>>>>>>> killed.
>>>>>>> The last log I found from nimbus.log is:
>>>>>>> o.a.s.d.nimbus main [INFO] Using custom scheduler: “
>>>>>>> org.apache.storm.scheduler.resource.ResourceAwareScheduler”.
>>>>>>>
>>>>>>> I suppose something goes wrong when it runs the scheduler.
>>>>>>>
>>>>>>> Do I miss anything to enable this scheduler?
>>>>>>>
>>>>>>> Best,
>>>>>>> Wuyang
>>>>>>> ᐧ
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Resource Aware Scheduler Kills Nimbus
Posted by Jerry Peng <je...@gmail.com>.
Yup, an error in the documentation
Feel free to submit a pull request to fix the documentation.
Best,
Jerry
On Sat, Feb 10, 2018 at 1:42 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
wrote:
> Ok, I find it works by adding
> storm.scheduler: org.apache.storm.scheduler.resource.
> ResourceAwareScheduler
>
> instead of adding
>
> storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
>
> suggested by the website..
>
> ᐧ
>
> On Sat, Feb 10, 2018 at 2:22 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
> wrote:
>
>> When I redo the procedure, here is the more log
>>
>> *2018-02-10 14:10:32.791 o.a.s.d.nimbus main [INFO] Using custom
>> scheduler: “org.apache.storm.scheduler.re
>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>> *2018-02-10 14:10:32.793 o.a.s.d.nimbus main [ERROR] Error on
>> initialization of server service-handler*
>> *java.lang.ClassNotFoundException: “org.apache.storm.scheduler.re
>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>> * at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> ~[?:1.8.0_151]*
>> * at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_151]*
>> * at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>> ~[?:1.8.0_151]*
>> * at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_151]*
>> * at java.lang.Class.forName0(Native Method) ~[?:1.8.0_151]*
>> * at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_151]*
>> * at org.apache.storm.util$new_instance.invoke(util.clj:1027)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at org.apache.storm.daemon.nimbus$mk_scheduler.invoke(nimbus.clj:127)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at org.apache.storm.daemon.nimbus$nimbus_data.invoke(nimbus.clj:215)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at
>> org.apache.storm.daemon.nimbus$fn__11007$exec_fn__1370__auto____11008.invoke(nimbus.clj:2451)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[clojure-1.7.0.jar:?]*
>> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
>> * at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]*
>> * at
>> org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
>> * at
>> org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]*
>> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
>> * at org.apache.storm.daemon.nimbus.main(Unknown Source)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> *2018-02-10 14:10:32.806 o.a.s.util main [ERROR] Halting process: ("Error
>> on initialization")*
>> *java.lang.RuntimeException: ("Error on initialization")*
>> * at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at clojure.lang.RestFn.invoke(RestFn.java:423) ~[clojure-1.7.0.jar:?]*
>> * at
>> org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
>> * at
>> org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>> * at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]*
>> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
>> * at org.apache.storm.daemon.nimbus.main(Unknown Source)
>> ~[storm-core-1.1.1.jar:1.1.1]*
>>
>> I download a binary version of Storm 1.1.1 from
>> http://www.apache.org/dyn/closer.lua/storm/apache-storm
>> -1.1.1/apache-storm-1.1.1.tar.gz.
>> ᐧ
>>
>> On Sat, Feb 10, 2018 at 2:02 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
>> wrote:
>>
>>> Hi Jerry,
>>>
>>> I just retry to use the resource aware scheduler in a single node with 4
>>> slots without any further settings. The scheduler still kills the nimbus
>>> process. Please find nimbus.log and storm.yaml attached.
>>>
>>> Without add the line
>>>
>>> storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
>>>
>>> in storm.yaml, everything works fine.
>>>
>>> I also tried
>>>
>>> conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class);
>>>
>>> The cluster can run, but I cannot see any log related to
>>> DefaultResourceAware. It seems like that it doesn't really run.
>>>
>>> Please take a look at the log and see if I miss anything to enable them
>>> to work?
>>>
>>> Best,
>>> Wuyang
>>> ᐧ
>>>
>>> On Sat, Feb 10, 2018 at 12:23 PM, Zhang Wuyang <qingshanyouyou@gmail.com
>>> > wrote:
>>>
>>>> Hi Jerry,
>>>>
>>>> Thank you for the reply.
>>>>
>>>> Please find the storm.yaml below:
>>>>
>>>> ########### These MUST be filled in for a storm configuration
>>>> storm.zookeeper.servers:
>>>> - "10.10.21.1"
>>>> - "10.10.21.2"
>>>> - "10.10.21.3"
>>>> - "10.10.21.4"
>>>> - "10.10.21.5"
>>>> - "10.10.21.6"
>>>> - "10.10.21.7"
>>>>
>>>> nimbus.seeds: ["10.10.21.1"]
>>>>
>>>> storm.zookeeper.port: 2181
>>>> ###### These may optionally be filled in:
>>>>
>>>> worker.heap.memory.mb: 8192
>>>>
>>>> supervisor.scheduler.meta:
>>>> tag: 10.10.20.1
>>>>
>>>> #storm.scheduler: storm.EmptyScheduler
>>>> storm.scheduler: “org.apache.storm.scheduler.re
>>>> source.ResourceAwareScheduler”
>>>> supervisor.memory.capacity.mb: 2048.0
>>>> supervisor.cpu.capacity: 4000.0
>>>> supervisor.slots.ports:
>>>> - 6700
>>>> - 6701
>>>> - 6702
>>>> - 6703
>>>>
>>>> worker.childopts: "-Xmx%HEAP-MEM%m -Xms6000m -XX:+PrintGCDetails
>>>> -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps
>>>> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M
>>>> -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump"
>>>>
>>>> On Sat, Feb 10, 2018 at 12:16 PM, Jerry Peng <
>>>> jerry.boyang.peng@gmail.com> wrote:
>>>>
>>>>> Wuyang,
>>>>>
>>>>> Can you provide your whole storm.yaml ?
>>>>>
>>>>> Best,
>>>>>
>>>>> Jerry
>>>>>
>>>>> On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <
>>>>> wuyang@winlab.rutgers.edu> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have the difficulty with the resource aware scheduler from the
>>>>>> version 1.1.1.
>>>>>>
>>>>>> As suggested by the website, I add the following lines to storm.yaml:
>>>>>>
>>>>>> *storm.scheduler: “org.apache.storm.scheduler.re
>>>>>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>>>>>> * supervisor.memory.capacity.mb: 2048.0*
>>>>>> * supervisor.cpu.capacity: 400.0*
>>>>>>
>>>>>> Afterwards, I rerun the storm and the nimbus process has been
>>>>>> killed.
>>>>>> The last log I found from nimbus.log is:
>>>>>> o.a.s.d.nimbus main [INFO] Using custom scheduler: “
>>>>>> org.apache.storm.scheduler.resource.ResourceAwareScheduler”.
>>>>>>
>>>>>> I suppose something goes wrong when it runs the scheduler.
>>>>>>
>>>>>> Do I miss anything to enable this scheduler?
>>>>>>
>>>>>> Best,
>>>>>> Wuyang
>>>>>> ᐧ
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Resource Aware Scheduler Kills Nimbus
Posted by Wuyang Zhang <wu...@winlab.rutgers.edu>.
Ok, I find it works by adding
storm.scheduler: org.apache.storm.scheduler.resource.ResourceAwareScheduler
instead of adding
storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
suggested by the website..
ᐧ
On Sat, Feb 10, 2018 at 2:22 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
wrote:
> When I redo the procedure, here is the more log
>
> *2018-02-10 14:10:32.791 o.a.s.d.nimbus main [INFO] Using custom
> scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”*
> *2018-02-10 14:10:32.793 o.a.s.d.nimbus main [ERROR] Error on
> initialization of server service-handler*
> *java.lang.ClassNotFoundException:
> “org.apache.storm.scheduler.resource.ResourceAwareScheduler”*
> * at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> ~[?:1.8.0_151]*
> * at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_151]*
> * at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> ~[?:1.8.0_151]*
> * at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_151]*
> * at java.lang.Class.forName0(Native Method) ~[?:1.8.0_151]*
> * at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_151]*
> * at org.apache.storm.util$new_instance.invoke(util.clj:1027)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$mk_scheduler.invoke(nimbus.clj:127)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$nimbus_data.invoke(nimbus.clj:215)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at
> org.apache.storm.daemon.nimbus$fn__11007$exec_fn__1370__auto____11008.invoke(nimbus.clj:2451)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[clojure-1.7.0.jar:?]*
> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
> * at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]*
> * at
> org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
> * at
> org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]*
> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
> * at org.apache.storm.daemon.nimbus.main(Unknown Source)
> ~[storm-core-1.1.1.jar:1.1.1]*
> *2018-02-10 14:10:32.806 o.a.s.util main [ERROR] Halting process: ("Error
> on initialization")*
> *java.lang.RuntimeException: ("Error on initialization")*
> * at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.RestFn.invoke(RestFn.java:423) ~[clojure-1.7.0.jar:?]*
> * at
> org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
> * at
> org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]*
> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
> * at org.apache.storm.daemon.nimbus.main(Unknown Source)
> ~[storm-core-1.1.1.jar:1.1.1]*
>
> I download a binary version of Storm 1.1.1 from http://www.apache.org/
> dyn/closer.lua/storm/apache-storm-1.1.1/apache-storm-1.1.1.tar.gz.
> ᐧ
>
> On Sat, Feb 10, 2018 at 2:02 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
> wrote:
>
>> Hi Jerry,
>>
>> I just retry to use the resource aware scheduler in a single node with 4
>> slots without any further settings. The scheduler still kills the nimbus
>> process. Please find nimbus.log and storm.yaml attached.
>>
>> Without add the line
>>
>> storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
>>
>> in storm.yaml, everything works fine.
>>
>> I also tried
>>
>> conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class);
>>
>> The cluster can run, but I cannot see any log related to
>> DefaultResourceAware. It seems like that it doesn't really run.
>>
>> Please take a look at the log and see if I miss anything to enable them
>> to work?
>>
>> Best,
>> Wuyang
>> ᐧ
>>
>> On Sat, Feb 10, 2018 at 12:23 PM, Zhang Wuyang <qi...@gmail.com>
>> wrote:
>>
>>> Hi Jerry,
>>>
>>> Thank you for the reply.
>>>
>>> Please find the storm.yaml below:
>>>
>>> ########### These MUST be filled in for a storm configuration
>>> storm.zookeeper.servers:
>>> - "10.10.21.1"
>>> - "10.10.21.2"
>>> - "10.10.21.3"
>>> - "10.10.21.4"
>>> - "10.10.21.5"
>>> - "10.10.21.6"
>>> - "10.10.21.7"
>>>
>>> nimbus.seeds: ["10.10.21.1"]
>>>
>>> storm.zookeeper.port: 2181
>>> ###### These may optionally be filled in:
>>>
>>> worker.heap.memory.mb: 8192
>>>
>>> supervisor.scheduler.meta:
>>> tag: 10.10.20.1
>>>
>>> #storm.scheduler: storm.EmptyScheduler
>>> storm.scheduler: “org.apache.storm.scheduler.re
>>> source.ResourceAwareScheduler”
>>> supervisor.memory.capacity.mb: 2048.0
>>> supervisor.cpu.capacity: 4000.0
>>> supervisor.slots.ports:
>>> - 6700
>>> - 6701
>>> - 6702
>>> - 6703
>>>
>>> worker.childopts: "-Xmx%HEAP-MEM%m -Xms6000m -XX:+PrintGCDetails
>>> -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps
>>> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M
>>> -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump"
>>>
>>> On Sat, Feb 10, 2018 at 12:16 PM, Jerry Peng <
>>> jerry.boyang.peng@gmail.com> wrote:
>>>
>>>> Wuyang,
>>>>
>>>> Can you provide your whole storm.yaml ?
>>>>
>>>> Best,
>>>>
>>>> Jerry
>>>>
>>>> On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <wuyang@winlab.rutgers.edu
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have the difficulty with the resource aware scheduler from the
>>>>> version 1.1.1.
>>>>>
>>>>> As suggested by the website, I add the following lines to storm.yaml:
>>>>>
>>>>> *storm.scheduler: “org.apache.storm.scheduler.re
>>>>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>>>>> * supervisor.memory.capacity.mb: 2048.0*
>>>>> * supervisor.cpu.capacity: 400.0*
>>>>>
>>>>> Afterwards, I rerun the storm and the nimbus process has been killed.
>>>>> The last log I found from nimbus.log is:
>>>>> o.a.s.d.nimbus main [INFO] Using custom scheduler: “
>>>>> org.apache.storm.scheduler.resource.ResourceAwareScheduler”.
>>>>>
>>>>> I suppose something goes wrong when it runs the scheduler.
>>>>>
>>>>> Do I miss anything to enable this scheduler?
>>>>>
>>>>> Best,
>>>>> Wuyang
>>>>> ᐧ
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Resource Aware Scheduler Kills Nimbus
Posted by Wuyang Zhang <wu...@winlab.rutgers.edu>.
When I redo the procedure, here is the more log
*2018-02-10 14:10:32.791 o.a.s.d.nimbus main [INFO] Using custom scheduler:
“org.apache.storm.scheduler.resource.ResourceAwareScheduler”*
*2018-02-10 14:10:32.793 o.a.s.d.nimbus main [ERROR] Error on
initialization of server service-handler*
*java.lang.ClassNotFoundException:
“org.apache.storm.scheduler.resource.ResourceAwareScheduler”*
* at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
~[?:1.8.0_151]*
* at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_151]*
* at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
~[?:1.8.0_151]*
* at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_151]*
* at java.lang.Class.forName0(Native Method) ~[?:1.8.0_151]*
* at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_151]*
* at org.apache.storm.util$new_instance.invoke(util.clj:1027)
~[storm-core-1.1.1.jar:1.1.1]*
* at org.apache.storm.daemon.nimbus$mk_scheduler.invoke(nimbus.clj:127)
~[storm-core-1.1.1.jar:1.1.1]*
* at org.apache.storm.daemon.nimbus$nimbus_data.invoke(nimbus.clj:215)
~[storm-core-1.1.1.jar:1.1.1]*
* at
org.apache.storm.daemon.nimbus$fn__11007$exec_fn__1370__auto____11008.invoke(nimbus.clj:2451)
~[storm-core-1.1.1.jar:1.1.1]*
* at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[clojure-1.7.0.jar:?]*
* at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
* at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]*
* at
org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
~[storm-core-1.1.1.jar:1.1.1]*
* at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
* at
org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
~[storm-core-1.1.1.jar:1.1.1]*
* at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
~[storm-core-1.1.1.jar:1.1.1]*
* at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
~[storm-core-1.1.1.jar:1.1.1]*
* at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]*
* at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
* at org.apache.storm.daemon.nimbus.main(Unknown Source)
~[storm-core-1.1.1.jar:1.1.1]*
*2018-02-10 14:10:32.806 o.a.s.util main [ERROR] Halting process: ("Error
on initialization")*
*java.lang.RuntimeException: ("Error on initialization")*
* at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
~[storm-core-1.1.1.jar:1.1.1]*
* at clojure.lang.RestFn.invoke(RestFn.java:423) ~[clojure-1.7.0.jar:?]*
* at
org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
~[storm-core-1.1.1.jar:1.1.1]*
* at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
* at
org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
~[storm-core-1.1.1.jar:1.1.1]*
* at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
~[storm-core-1.1.1.jar:1.1.1]*
* at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
~[storm-core-1.1.1.jar:1.1.1]*
* at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]*
* at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
* at org.apache.storm.daemon.nimbus.main(Unknown Source)
~[storm-core-1.1.1.jar:1.1.1]*
I download a binary version of Storm 1.1.1 from
http://www.apache.org/dyn/closer.lua/storm/apache-storm-1.1.1/apache-storm-1.1.1.tar.gz
.
ᐧ
On Sat, Feb 10, 2018 at 2:02 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
wrote:
> Hi Jerry,
>
> I just retry to use the resource aware scheduler in a single node with 4
> slots without any further settings. The scheduler still kills the nimbus
> process. Please find nimbus.log and storm.yaml attached.
>
> Without add the line
>
> storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
>
> in storm.yaml, everything works fine.
>
> I also tried
>
> conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class);
>
> The cluster can run, but I cannot see any log related to
> DefaultResourceAware. It seems like that it doesn't really run.
>
> Please take a look at the log and see if I miss anything to enable them to
> work?
>
> Best,
> Wuyang
> ᐧ
>
> On Sat, Feb 10, 2018 at 12:23 PM, Zhang Wuyang <qi...@gmail.com>
> wrote:
>
>> Hi Jerry,
>>
>> Thank you for the reply.
>>
>> Please find the storm.yaml below:
>>
>> ########### These MUST be filled in for a storm configuration
>> storm.zookeeper.servers:
>> - "10.10.21.1"
>> - "10.10.21.2"
>> - "10.10.21.3"
>> - "10.10.21.4"
>> - "10.10.21.5"
>> - "10.10.21.6"
>> - "10.10.21.7"
>>
>> nimbus.seeds: ["10.10.21.1"]
>>
>> storm.zookeeper.port: 2181
>> ###### These may optionally be filled in:
>>
>> worker.heap.memory.mb: 8192
>>
>> supervisor.scheduler.meta:
>> tag: 10.10.20.1
>>
>> #storm.scheduler: storm.EmptyScheduler
>> storm.scheduler: “org.apache.storm.scheduler.re
>> source.ResourceAwareScheduler”
>> supervisor.memory.capacity.mb: 2048.0
>> supervisor.cpu.capacity: 4000.0
>> supervisor.slots.ports:
>> - 6700
>> - 6701
>> - 6702
>> - 6703
>>
>> worker.childopts: "-Xmx%HEAP-MEM%m -Xms6000m -XX:+PrintGCDetails
>> -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps
>> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M
>> -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump"
>>
>> On Sat, Feb 10, 2018 at 12:16 PM, Jerry Peng <jerry.boyang.peng@gmail.com
>> > wrote:
>>
>>> Wuyang,
>>>
>>> Can you provide your whole storm.yaml ?
>>>
>>> Best,
>>>
>>> Jerry
>>>
>>> On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have the difficulty with the resource aware scheduler from the
>>>> version 1.1.1.
>>>>
>>>> As suggested by the website, I add the following lines to storm.yaml:
>>>>
>>>> *storm.scheduler: “org.apache.storm.scheduler.re
>>>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>>>> * supervisor.memory.capacity.mb: 2048.0*
>>>> * supervisor.cpu.capacity: 400.0*
>>>>
>>>> Afterwards, I rerun the storm and the nimbus process has been killed.
>>>> The last log I found from nimbus.log is:
>>>> o.a.s.d.nimbus main [INFO] Using custom scheduler: “
>>>> org.apache.storm.scheduler.resource.ResourceAwareScheduler”.
>>>>
>>>> I suppose something goes wrong when it runs the scheduler.
>>>>
>>>> Do I miss anything to enable this scheduler?
>>>>
>>>> Best,
>>>> Wuyang
>>>> ᐧ
>>>>
>>>
>>>
>>
>
Re: Resource Aware Scheduler Kills Nimbus
Posted by Wuyang Zhang <wu...@winlab.rutgers.edu>.
Hi Jerry,
I just retry to use the resource aware scheduler in a single node with 4
slots without any further settings. The scheduler still kills the nimbus
process. Please find nimbus.log and storm.yaml attached.
Without add the line
storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
in storm.yaml, everything works fine.
I also tried
conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class);
The cluster can run, but I cannot see any log related to
DefaultResourceAware. It seems like that it doesn't really run.
Please take a look at the log and see if I miss anything to enable them to
work?
Best,
Wuyang
ᐧ
On Sat, Feb 10, 2018 at 12:23 PM, Zhang Wuyang <qi...@gmail.com>
wrote:
> Hi Jerry,
>
> Thank you for the reply.
>
> Please find the storm.yaml below:
>
> ########### These MUST be filled in for a storm configuration
> storm.zookeeper.servers:
> - "10.10.21.1"
> - "10.10.21.2"
> - "10.10.21.3"
> - "10.10.21.4"
> - "10.10.21.5"
> - "10.10.21.6"
> - "10.10.21.7"
>
> nimbus.seeds: ["10.10.21.1"]
>
> storm.zookeeper.port: 2181
> ###### These may optionally be filled in:
>
> worker.heap.memory.mb: 8192
>
> supervisor.scheduler.meta:
> tag: 10.10.20.1
>
> #storm.scheduler: storm.EmptyScheduler
> storm.scheduler: “org.apache.storm.scheduler.resource.
> ResourceAwareScheduler”
> supervisor.memory.capacity.mb: 2048.0
> supervisor.cpu.capacity: 4000.0
> supervisor.slots.ports:
> - 6700
> - 6701
> - 6702
> - 6703
>
> worker.childopts: "-Xmx%HEAP-MEM%m -Xms6000m -XX:+PrintGCDetails
> -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps
> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M
> -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump"
>
> On Sat, Feb 10, 2018 at 12:16 PM, Jerry Peng <je...@gmail.com>
> wrote:
>
>> Wuyang,
>>
>> Can you provide your whole storm.yaml ?
>>
>> Best,
>>
>> Jerry
>>
>> On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
>> wrote:
>>
>>> Hi,
>>>
>>> I have the difficulty with the resource aware scheduler from the version
>>> 1.1.1.
>>>
>>> As suggested by the website, I add the following lines to storm.yaml:
>>>
>>> *storm.scheduler: “org.apache.storm.scheduler.re
>>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>>> * supervisor.memory.capacity.mb: 2048.0*
>>> * supervisor.cpu.capacity: 400.0*
>>>
>>> Afterwards, I rerun the storm and the nimbus process has been killed.
>>> The last log I found from nimbus.log is:
>>> o.a.s.d.nimbus main [INFO] Using custom scheduler: “
>>> org.apache.storm.scheduler.resource.ResourceAwareScheduler”.
>>>
>>> I suppose something goes wrong when it runs the scheduler.
>>>
>>> Do I miss anything to enable this scheduler?
>>>
>>> Best,
>>> Wuyang
>>> ᐧ
>>>
>>
>>
>
Re: Resource Aware Scheduler Kills Nimbus
Posted by Zhang Wuyang <qi...@gmail.com>.
Hi Jerry,
Thank you for the reply.
Please find the storm.yaml below:
########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
- "10.10.21.1"
- "10.10.21.2"
- "10.10.21.3"
- "10.10.21.4"
- "10.10.21.5"
- "10.10.21.6"
- "10.10.21.7"
nimbus.seeds: ["10.10.21.1"]
storm.zookeeper.port: 2181
###### These may optionally be filled in:
worker.heap.memory.mb: 8192
supervisor.scheduler.meta:
tag: 10.10.20.1
#storm.scheduler: storm.EmptyScheduler
storm.scheduler:
“org.apache.storm.scheduler.resource.ResourceAwareScheduler”
supervisor.memory.capacity.mb: 2048.0
supervisor.cpu.capacity: 4000.0
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
worker.childopts: "-Xmx%HEAP-MEM%m -Xms6000m -XX:+PrintGCDetails
-Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps
-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump"
On Sat, Feb 10, 2018 at 12:16 PM, Jerry Peng <je...@gmail.com>
wrote:
> Wuyang,
>
> Can you provide your whole storm.yaml ?
>
> Best,
>
> Jerry
>
> On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
> wrote:
>
>> Hi,
>>
>> I have the difficulty with the resource aware scheduler from the version
>> 1.1.1.
>>
>> As suggested by the website, I add the following lines to storm.yaml:
>>
>> *storm.scheduler: “org.apache.storm.scheduler.re
>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>> * supervisor.memory.capacity.mb: 2048.0*
>> * supervisor.cpu.capacity: 400.0*
>>
>> Afterwards, I rerun the storm and the nimbus process has been killed.
>> The last log I found from nimbus.log is:
>> o.a.s.d.nimbus main [INFO] Using custom scheduler: “
>> org.apache.storm.scheduler.resource.ResourceAwareScheduler”.
>>
>> I suppose something goes wrong when it runs the scheduler.
>>
>> Do I miss anything to enable this scheduler?
>>
>> Best,
>> Wuyang
>> ᐧ
>>
>
>
Re: Resource Aware Scheduler Kills Nimbus
Posted by Jerry Peng <je...@gmail.com>.
Wuyang,
Can you provide your whole storm.yaml ?
Best,
Jerry
On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <wu...@winlab.rutgers.edu>
wrote:
> Hi,
>
> I have the difficulty with the resource aware scheduler from the version
> 1.1.1.
>
> As suggested by the website, I add the following lines to storm.yaml:
>
> *storm.scheduler:
> “org.apache.storm.scheduler.resource.ResourceAwareScheduler”*
> * supervisor.memory.capacity.mb: 2048.0*
> * supervisor.cpu.capacity: 400.0*
>
> Afterwards, I rerun the storm and the nimbus process has been killed.
> The last log I found from nimbus.log is:
> o.a.s.d.nimbus main [INFO] Using custom scheduler:
> “org.apache.storm.scheduler.resource.ResourceAwareScheduler”.
>
> I suppose something goes wrong when it runs the scheduler.
>
> Do I miss anything to enable this scheduler?
>
> Best,
> Wuyang
> ᐧ
>