You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by #ZHANG SHUHAO# <SZ...@e.ntu.edu.sg> on 2016/06/21 14:08:49 UTC
Can storm pin executor threads to CPU cores?
Hi all,
I want to manage the binding of Storm executor thread (NOT worker process) to CPU cores (NOT host nodes).
I assume if I would know how the executor thread are initiated, I can manage the binding of it by relying on things like JNI.
I have tried my best to look into the related source code of Storm.
Unfortunately, I am very new to Clojure, and I don't get how the executor thread are initiated, in particular, I'm looking for something like "Thread thread = new Thread(obj);", which is one of the most common way of creating threads in Java.
Any helps are appreciated. Thank you!
Best regards.
Shuhao Zhang (Tony)
+65-86166722
Re: Can storm pin executor threads to CPU cores?
Posted by Bobby Evans <ev...@yahoo-inc.com.INVALID>.
There is only one thread associated with an input queue and that is the one the executor runs on. The output queue has a separate thread that all it does is pop messages off of the output queue and routes them to where they need to be. In my experience this runs fairly infrequently. I would be curious to see if it would benefit from being pined to the same core as the executor thread, as it would likely improve cache hits for the data in the queue, but if the spout/bolt is really busy it would compete with the other thread for CPU resources.
The output queue itself is created here
https://github.com/apache/storm/blob/v1.0.1/storm-core/src/clj/org/apache/storm/daemon/executor.clj#L230-L236
The thread is created here
https://github.com/apache/storm/blob/v1.0.1/storm-core/src/clj/org/apache/storm/daemon/executor.clj#L297-L311
The thread does not really run much user generated code so trying to find a simple place to insert the pinning without modifying storm would be difficult.
You do have access to the storm-conf, which is the topology config, and the executor-data to help you pick a core to pin it to there, and run similar code when the bolt/spout thread is created
https://github.com/apache/storm/blob/v1.0.1/storm-core/src/clj/org/apache/storm/daemon/executor.clj#L547
and
https://github.com/apache/storm/blob/v1.0.1/storm-core/src/clj/org/apache/storm/daemon/executor.clj#L743
- Bobby
On Thursday, June 23, 2016 3:32 AM, #ZHANG SHUHAO# <SZ...@e.ntu.edu.sg> wrote:
Hi!
Thanks for your pointers!
I want to make sure the I/O message queue also being pined together with the corresponding executor, is it possible to do in Storm?
Follow Bobby's "easier solution" suggestion, in a machine with multiple CPU sockets, if I pin the executor thread to one CPU socket in the open method, what would happen to its input&output buffer (the disruptor queue)? Would it being "pined"? That is to ask, "who" creates the internal buffers and when?
Any helps are appreciated. Thank you!
Tony.
> On 21 Jun 2016, at 10:16 PM, Bobby Evans <ev...@yahoo-inc.com.INVALID> wrote:
>
> If you really want to do this you should be able to do this is the open method of your spout or in the prepare method of the bolt. They execute on the same thread so from that you should be able to do what you need to do.
> As for the clojure in executor.clj we use the async-loop macro for creating the task threads.
>
> https://github.com/apache/storm/blob/v1.0.2/storm-core/src/clj/org/apache/storm/util.clj#L472-L514
>
> and on line 479 you can see the clojure equivalent of `new Thread(...)` `(Thread. ...)` - Bobby
>
> On Tuesday, June 21, 2016 9:09 AM, #ZHANG SHUHAO# <SZ...@e.ntu.edu.sg> wrote:
>
>
> Hi all,
>
> I want to manage the binding of Storm executor thread (NOT worker process) to CPU cores (NOT host nodes).
> I assume if I would know how the executor thread are initiated, I can manage the binding of it by relying on things like JNI.
> I have tried my best to look into the related source code of Storm.
> Unfortunately, I am very new to Clojure, and I don't get how the executor thread are initiated, in particular, I'm looking for something like "Thread thread = new Thread(obj);", which is one of the most common way of creating threads in Java.
>
> Any helps are appreciated. Thank you!
>
> Best regards.
> Shuhao Zhang (Tony)
> +65-86166722
>
>
Re: Can storm pin executor threads to CPU cores?
Posted by #ZHANG SHUHAO# <SZ...@e.ntu.edu.sg>.
Hi!
Thanks for your pointers!
I want to make sure the I/O message queue also being pined together with the corresponding executor, is it possible to do in Storm?
Follow Bobby's "easier solution" suggestion, in a machine with multiple CPU sockets, if I pin the executor thread to one CPU socket in the open method, what would happen to its input&output buffer (the disruptor queue)? Would it being "pined"? That is to ask, "who" creates the internal buffers and when?
Any helps are appreciated. Thank you!
Tony.
> On 21 Jun 2016, at 10:16 PM, Bobby Evans <ev...@yahoo-inc.com.INVALID> wrote:
>
> If you really want to do this you should be able to do this is the open method of your spout or in the prepare method of the bolt. They execute on the same thread so from that you should be able to do what you need to do.
> As for the clojure in executor.clj we use the async-loop macro for creating the task threads.
>
> https://github.com/apache/storm/blob/v1.0.2/storm-core/src/clj/org/apache/storm/util.clj#L472-L514
>
> and on line 479 you can see the clojure equivalent of `new Thread(...)` `(Thread. ...)` - Bobby
>
> On Tuesday, June 21, 2016 9:09 AM, #ZHANG SHUHAO# <SZ...@e.ntu.edu.sg> wrote:
>
>
> Hi all,
>
> I want to manage the binding of Storm executor thread (NOT worker process) to CPU cores (NOT host nodes).
> I assume if I would know how the executor thread are initiated, I can manage the binding of it by relying on things like JNI.
> I have tried my best to look into the related source code of Storm.
> Unfortunately, I am very new to Clojure, and I don't get how the executor thread are initiated, in particular, I'm looking for something like "Thread thread = new Thread(obj);", which is one of the most common way of creating threads in Java.
>
> Any helps are appreciated. Thank you!
>
> Best regards.
> Shuhao Zhang (Tony)
> +65-86166722
>
>
Re: Can storm pin executor threads to CPU cores?
Posted by Bobby Evans <ev...@yahoo-inc.com.INVALID>.
If you really want to do this you should be able to do this is the open method of your spout or in the prepare method of the bolt. They execute on the same thread so from that you should be able to do what you need to do.
As for the clojure in executor.clj we use the async-loop macro for creating the task threads.
https://github.com/apache/storm/blob/v1.0.2/storm-core/src/clj/org/apache/storm/util.clj#L472-L514
and on line 479 you can see the clojure equivalent of `new Thread(...)` `(Thread. ...)` - Bobby
On Tuesday, June 21, 2016 9:09 AM, #ZHANG SHUHAO# <SZ...@e.ntu.edu.sg> wrote:
Hi all,
I want to manage the binding of Storm executor thread (NOT worker process) to CPU cores (NOT host nodes).
I assume if I would know how the executor thread are initiated, I can manage the binding of it by relying on things like JNI.
I have tried my best to look into the related source code of Storm.
Unfortunately, I am very new to Clojure, and I don't get how the executor thread are initiated, in particular, I'm looking for something like "Thread thread = new Thread(obj);", which is one of the most common way of creating threads in Java.
Any helps are appreciated. Thank you!
Best regards.
Shuhao Zhang (Tony)
+65-86166722