Could Storm config dispatcher like akka?

[Renkai <ga...@gmail.com>]

Hi all:
I am a storm user for one year and now I am learning Akka. I found something interesting in Akka that it could config its dispatcher(see http://doc.akka.io/docs/akka/2.4.0/scala/dispatchers.html). I think it is useful for tuning performance and storm seems missing something like that.

Re: Could Storm config dispatcher like akka?

[Bobby Evans <ev...@yahoo-inc.com.INVALID>]

Renkai,
The execution models between storm and akka are rather different.  Storm has dedicated threads for each executor, although multiple tasks (bolt/spout instances) can share the same executor thread.  We rely on the OS to do the right thing in scheduling threads to run.  Akka is using an actor model.  The dispatcher essentially provides a thread pool for all of the actors to share.  Instead of allowing the OS to do all of the context switching they use the dispatcher to decide in many cases when one actor should stop executing and another should start.  Currently storm tends to run with latency in mind over throughput, but we are working on adding in tuning knobs to allow for more throughput at the expense of some latency.  However the ultimate goal is to get to the point where you tell storm the constraints you have in terms of latency, throughput, and cost and let storm work to optimize the topology to meet those desires.  This is still a ways out, but to me feels much more user friendly then trying to figure out what the batch size should be for a given actor manually.
 - Bobby 


     On Sunday, October 18, 2015 10:03 AM, Renkai <ga...@gmail.com> wrote:
   

 Hi all:
I am a storm user for one year and now I am learning Akka. I found something interesting in Akka that it could config its dispatcher(see http://doc.akka.io/docs/akka/2.4.0/scala/dispatchers.html). I think it is useful for tuning performance and storm seems missing something like that.