You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by 이승진 <sw...@navercorp.com> on 2014/10/13 03:39:01 UTC
regarding shared componenets of a single bolt
Dear all,
One simple question about parallelism of storm and shared variables.
If I set parallelism count to 4 for example, and there are some member variables in that bolt.
Is it shared across 4 bolts or not?
Seems like each is seperate object and they don't share nothing, which is the opposite way from what I expected.
For example, if I run this bolt in local mode, 4 counters is being incremented seperately,
public class PercolatorBolt extends BaseBasicBolt {
private int i = 0;
@Override
public void prepare(Map stormConf, TopologyContext context) {
}
public void execute(Tuple tuple, BasicOutputCollector collector) {
synchronized(this){
System.out.println(i++);
}
}
}
I want to here from you about this topic, thanks.
Sincerely,
Re: regarding shared componenets of a single bolt
Posted by Nathan Leung <nc...@gmail.com>.
Each bolt task has its own bolt object, which is a single instance of the
bolt class. There's no way for all bolts to share the same object,
especially if they are spread across workers. As Itai indicated, if you
want to share a counter you can use memcache or redis or something similar.
Alternatively you can use an aggregation bolt.
On Oct 13, 2014 8:03 AM, "Itai Frenkel" <It...@forter.com> wrote:
> Hi,
>
>
> Would you consider using memcached or redis for cross bolt counters, or
> do you have any requirement to perform the increments inside the Bolt?
>
>
> see also:
>
> http://storm.incubator.apache.org/documentation/Common-patterns.html
>
>
> Regards,
>
> Itai
>
>
> ------------------------------
> *From:* 이승진 <sw...@navercorp.com>
> *Sent:* Monday, October 13, 2014 4:39 AM
> *To:* user@storm.apache.org
> *Subject:* regarding shared componenets of a single bolt
>
>
> Dear all,
>
>
>
> One simple question about parallelism of storm and shared variables.
>
>
>
> If I set parallelism count to 4 for example, and there are some member
> variables in that bolt.
>
>
>
> Is it shared across 4 bolts or not?
>
>
>
> Seems like each is seperate object and they don't share nothing, which is
> the opposite way from what I expected.
>
>
>
> For example, if I run this bolt in local mode, 4 counters is being
> incremented seperately,
>
>
>
> public class PercolatorBolt extends BaseBasicBolt {
>
> private int i = 0;
>
> @Override
>
> public void prepare(Map stormConf, TopologyContext context) {
>
> }
>
> public void execute(Tuple tuple, BasicOutputCollector collector) {
>
> synchronized(this){
>
> System.out.println(i++);
>
> }
>
> }
>
> }
>
> I want to here from you about this topic, thanks.
>
>
>
> Sincerely,
>
>
Re: regarding shared componenets of a single bolt
Posted by Itai Frenkel <It...@forter.com>.
Hi,
Would you consider using memcached or redis for cross bolt counters, or do you have any requirement to perform the increments inside the Bolt?
see also:
http://storm.incubator.apache.org/documentation/Common-patterns.html?
Regards,
Itai
________________________________
From: 이승진 <sw...@navercorp.com>
Sent: Monday, October 13, 2014 4:39 AM
To: user@storm.apache.org
Subject: regarding shared componenets of a single bolt
Dear all,
One simple question about parallelism of storm and shared variables.
If I set parallelism count to 4 for example, and there are some member variables in that bolt.
Is it shared across 4 bolts or not?
Seems like each is seperate object and they don't share nothing, which is the opposite way from what I expected.
For example, if I run this bolt in local mode, 4 counters is being incremented seperately,
public class PercolatorBolt extends BaseBasicBolt {
private int i = 0;
@Override
public void prepare(Map stormConf, TopologyContext context) {
}
public void execute(Tuple tuple, BasicOutputCollector collector) {
synchronized(this){
System.out.println(i++);
}
}
}
I want to here from you about this topic, thanks.
Sincerely,
[http://ack.mail.navercorp.com/readReceipt/notify/?img=rfnmFqKrFxudKA2YaqumFxE%2FMxISKoiCK4KrMxJoMrtqM6E%2FKzUqa6F4pEIo%2BrkSKAu5W49vpSl51zlqDBFdp6d5MreRhoR9brkZtHFdWXiR74JmM6FcpLlTb4b%3D.gif]