You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Dollyn <do...@gmail.com> on 2015/11/26 10:54:29 UTC
Will Storm/JStorm new a separate bolt instance for every task/thread?
For example:
builder.setBolt("xxx", new XXXBolt(), 10)
Assume that there is only one worker, how many XXXBolt's java object
instance will be there? 1 or 10?
In fact, I did some test by add the following code in the execute method:
LOGGER.debug("BoltID: {}, ThreadID: {} - {}",
System.identityHashCode(this), Thread.currentThread().getName(),
Thread.currentThread().getId());
The result is:
BoltID: 558207946, ThreadID: analyze_bolt:4-BoltExecutors - 107
BoltID: 1052936813, ThreadID: analyze_bolt:5-BoltExecutors - 103
Seems that there is instance for every thread, but can't be sure that will
or not bolt instance will be reused among tasks/threads, and the document
does not make this clear either.
--
Dollyn
Re: Will Storm/JStorm new a separate bolt instance for every task/thread?
Posted by Dollyn <do...@gmail.com>.
That is very helpful,
Thank you very much yuanjun Li and Longda!
On Thu, Nov 26, 2015 at 10:34 PM, yuanjun Li <cs...@gmail.com> wrote:
> You can read the links below:
>
>
> http://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html
>
> https://groups.google.com/forum/#!topic/storm-user/IS8PijLjU8c
>
--
Dollyn
Re: Will Storm/JStorm new a separate bolt instance for every task/thread?
Posted by yuanjun Li <cs...@gmail.com>.
You can read the links below:
http://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html
https://groups.google.com/forum/#!topic/storm-user/IS8PijLjU8c
Re: Will Storm/JStorm new a separate bolt instance for every task/thread?
Posted by Dollyn <do...@gmail.com>.
If the bolt class is
class XXXBolt {
private int x;
}
builder.setBolt("xxx", new XXXBolt(), 10).setNumTasks(20);
In jstorm there is no thread-safe problem on the field x.
And if
builder.setBolt("xxx", new XXXBolt(), 10)
There is no thread-safe problem in both storm and storm. Is that right?
On Thu, Nov 26, 2015 at 6:18 PM, 封仲淹(纪君祥) <zh...@alibaba-inc.com>
wrote:
>
> Please read this page http://storm.apache.org/documentation/Concepts.html
> firstly.
>
> Let me explain several conception:
>
> In storm, one worker means one process, one executor means one thread,
> one task means one bolt or spout instance.
> In jstorm, worker means process, task also means one thread and one bolt
> or spout instance. there is no executor.
>
>
>
> builder.setBolt("xxx", new XXXBolt(), 10);
>
> In storm, this will create 10 executor, due to not set task parallel, the
> default ratio of executor and task is 1, there are 10 tasks, . Every task
> will occupy one executor (thread).
> In jstorm, this will create 10 tasks. Every task occupy one thread
>
>
> builder.setBolt("xxx", new XXXBolt(), 10).setNumTasks(20);
> In storm, this will create 10 executor, 20 tasks, Every two tasks will
> occupy one executor (thread).
> In jstorm, this will create 10 tasks. Every task will occupy one thread.
>
>
> There is a little confusion between Storm and JStorm, but later in Storm
> 2.0, we will unify them.
>
> regards
> Longda
>
>
>
>
>
>
> ------------------------------------------------------------------
> From:Dollyn <do...@gmail.com>
> Send Time:2015年11月26日(星期四) 17:54
> To:user <us...@storm.apache.org>
> Subject:Will Storm/JStorm new a separate bolt instance for every
> task/thread?
>
> For example:
>
> builder.setBolt("xxx", new XXXBolt(), 10)
>
>
> Assume that there is only one worker, how many XXXBolt's java object
> instance will be there? 1 or 10?
>
>
> In fact, I did some test by add the following code in the execute method:
>
> LOGGER.debug("BoltID: {}, ThreadID: {} - {}",
> System.identityHashCode(this), Thread.currentThread().getName(),
> Thread.currentThread().getId());
>
> The result is:
>
> BoltID: 558207946, ThreadID: analyze_bolt:4-BoltExecutors - 107
> BoltID: 1052936813, ThreadID: analyze_bolt:5-BoltExecutors - 103
>
> Seems that there is instance for every thread, but can't be sure that will
> or not bolt instance will be reused among tasks/threads, and the document
> does not make this clear either.
>
>
> --
> Dollyn
>
>
--
Dollyn
Re: Will Storm/JStorm new a separate bolt instance for every task/thread?
Posted by "封仲淹(纪君祥)" <zh...@alibaba-inc.com>.
Please read this page http://storm.apache.org/documentation/Concepts.html firstly.Let me explain several conception:In storm, one worker means one process, one executor means one thread, one task means one bolt or spout instance.In jstorm, worker means process, task also means one thread and one bolt or spout instance. there is no executor.builder.setBolt("xxx", new XXXBolt(), 10);In storm, this will create 10 executor, due to not set task parallel, the default ratio of executor and task is 1, there are 10 tasks, . Every task will occupy one executor (thread).In jstorm, this will create 10 tasks. Every task occupy one threadbuilder.setBolt("xxx", new XXXBolt(), 10).setNumTasks(20);In storm, this will create 10 executor, 20 tasks, Every two tasks will occupy one executor (thread).In jstorm, this will create 10 tasks. Every task will occupy one thread.There is a little confusion between Storm and JStorm, but later in Storm 2.0, we will unify them.regardsLongda------------------------------------------------------------------From:Dollyn <do...@gmail.com>Send Time:2015年11月26日(星期四) 17:54To:user <us...@storm.apache.org>Subject:Will Storm/JStorm new a separate bolt instance for every task/thread?For example:builder.setBolt("xxx", new XXXBolt(), 10)Assume that there is only one worker, how many XXXBolt's java object instance will be there? 1 or 10?In fact, I did some test by add the following code in the execute method:LOGGER.debug("BoltID: {}, ThreadID: {} - {}", System.identityHashCode(this), Thread.currentThread().getName(), Thread.currentThread().getId());The result is:BoltID: 558207946, ThreadID: analyze_bolt:4-BoltExecutors - 107BoltID: 1052936813, ThreadID: analyze_bolt:5-BoltExecutors - 103Seems that there is instance for every thread, but can't be sure that will or not bolt instance will be reused among tasks/threads, and the document does not make this clear either.-- Dollyn