You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@storm.apache.org by hilfi alkaff <hi...@gmail.com> on 2013/12/20 01:05:52 UTC

Storm scheduling

Hi,

I'm trying to trace how Nimbus schedules the supervisors when a new task
comes in.

I'm currently looking at the function
<http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying#>compute-new-topology->executor->node+port
in Nimbus.clj but I am having a hard time understanding the clojure code.
Could anybody explain how Nimbus does it briefly and if I'm looking at the
correct portion of the code.

Thanks a lot,

--
~Hilfi Alkaff

Re: Storm scheduling

Posted by 李家宏 <jh...@gmail.com>.

hi , Ruihao

actually, i need it. Thanks in advance

Regards


2013/12/20 Ruihao Ye <zj...@gmail.com>

> Only have a chinese blog that analysize some source code of storm. Do you
> need it?
>
>
> On Fri, Dec 20, 2013 at 8:05 AM, hilfi alkaff <hi...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I'm trying to trace how Nimbus schedules the supervisors when a new task
> > comes in.
> >
> > I'm currently looking at the function
> > <
> >
> http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying#
> > >compute-new-topology->executor->node+port
> > in Nimbus.clj but I am having a hard time understanding the clojure code.
> > Could anybody explain how Nimbus does it briefly and if I'm looking at
> the
> > correct portion of the code.
> >
> > Thanks a lot,
> >
> > --
> > ~Hilfi Alkaff
> >
>



-- 

======================================================

Gvain

Email: jh.li.em@gmail.com

Re: Storm scheduling

Posted by James Xu <xu...@gmail.com>.

Hi, 

If you just want to understand the logic, you might want to look at these two files: https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/scheduler/EvenScheduler.clj?source=cc ,  https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/scheduler/DefaultScheduler.clj

If you want to write your custom scheduler, have a look at this post: http://xumingming.sinaapp.com/885/twitter-storm-how-to-develop-a-pluggable-scheduler/

- James Xu
On 2013年12月20日, at 下午1:51, hilfi alkaff <hi...@gmail.com> wrote:

> Hi,
> 
> Unfortunately I can not read Chinese, :(. If anyone else knows the relevant
> functions/writeups for my question, that will be extremely helpful.
> 
> Thanks,
> 
> 
> On Thu, Dec 19, 2013 at 9:53 PM, Ruihao Ye <zj...@gmail.com> wrote:
> 
>> Only have a chinese blog that analysize some source code of storm. Do you
>> need it?
>> 
>> 
>> On Fri, Dec 20, 2013 at 8:05 AM, hilfi alkaff <hi...@gmail.com>
>> wrote:
>> 
>>> Hi,
>>> 
>>> I'm trying to trace how Nimbus schedules the supervisors when a new task
>>> comes in.
>>> 
>>> I'm currently looking at the function
>>> <
>>> 
>> http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying#
>>>> compute-new-topology->executor->node+port
>>> in Nimbus.clj but I am having a hard time understanding the clojure code.
>>> Could anybody explain how Nimbus does it briefly and if I'm looking at
>> the
>>> correct portion of the code.
>>> 
>>> Thanks a lot,
>>> 
>>> --
>>> ~Hilfi Alkaff
>>> 
>> 
> 
> 
> 
> -- 
> ~Hilfi Alkaff~

Re: Storm scheduling

Posted by hilfi alkaff <hi...@gmail.com>.

Hi,

Unfortunately I can not read Chinese, :(. If anyone else knows the relevant
functions/writeups for my question, that will be extremely helpful.

Thanks,


On Thu, Dec 19, 2013 at 9:53 PM, Ruihao Ye <zj...@gmail.com> wrote:

> Only have a chinese blog that analysize some source code of storm. Do you
> need it?
>
>
> On Fri, Dec 20, 2013 at 8:05 AM, hilfi alkaff <hi...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I'm trying to trace how Nimbus schedules the supervisors when a new task
> > comes in.
> >
> > I'm currently looking at the function
> > <
> >
> http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying#
> > >compute-new-topology->executor->node+port
> > in Nimbus.clj but I am having a hard time understanding the clojure code.
> > Could anybody explain how Nimbus does it briefly and if I'm looking at
> the
> > correct portion of the code.
> >
> > Thanks a lot,
> >
> > --
> > ~Hilfi Alkaff
> >
>



-- 
~Hilfi Alkaff~

Re: Storm scheduling

Posted by Ruihao Ye <zj...@gmail.com>.

Only have a chinese blog that analysize some source code of storm. Do you
need it?


On Fri, Dec 20, 2013 at 8:05 AM, hilfi alkaff <hi...@gmail.com> wrote:

> Hi,
>
> I'm trying to trace how Nimbus schedules the supervisors when a new task
> comes in.
>
> I'm currently looking at the function
> <
> http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying#
> >compute-new-topology->executor->node+port
> in Nimbus.clj but I am having a hard time understanding the clojure code.
> Could anybody explain how Nimbus does it briefly and if I'm looking at the
> correct portion of the code.
>
> Thanks a lot,
>
> --
> ~Hilfi Alkaff
>