You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by sam mohel <sa...@gmail.com> on 2017/01/23 09:39:39 UTC
simple question about grouping
i have text file contains data . size of this file is 3.5 MB . My topology
consists of one spout and one bolt so is that possible to make all
processing in one bolt and in this case what is the role of grouping here ?
Thanks in advance
Re: simple question about grouping
Posted by sam mohel <sa...@gmail.com>.
thanks for the link it's so helpful
On Tue, Jan 24, 2017 at 7:31 AM, Navin Ipe <na...@searchlighthealth.com>
wrote:
> More specifically, the bottom of this page explains what the numbers mean
> in terms of tasks and executors: http://nrecursions.blogspot.
> in/2016/05/more-concepts-of-apache-storm-you-need.html
>
> On Mon, Jan 23, 2017 at 5:28 PM, sam mohel <sa...@gmail.com> wrote:
>
>> really thanks for your help
>>
>> On Mon, Jan 23, 2017 at 1:59 PM, Arun Mahadevan <ar...@apache.org> wrote:
>>
>>> There is no magic number, it depends on the specific problem you are
>>> trying to solve. You start with some reasonable value for the parallelism
>>> and tune it based on your requirements. You could also start with a higher
>>> number of “tasks” than the parallelism and then you can rebalance your
>>> topology and adjust parallelism on the fly to scale up or down.
>>>
>>>
>>>
>>> See the slides from Taylor’s “Scaling Storm” presentation, you might
>>> find it useful - http://www.slideshare.net/ptgo
>>> etz/scaling-apache-storm-strata-hadoopworld-2014
>>>
>>>
>>>
>>>
>>>
>>> *From: *sam mohel <sa...@gmail.com>
>>> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
>>> *Date: *Monday, January 23, 2017 at 4:58 PM
>>>
>>> *To: *"user@storm.apache.org" <us...@storm.apache.org>
>>> *Subject: *Re: simple question about grouping
>>>
>>>
>>>
>>> Many thanks , but how and when can i decide that this number is perfect
>>> form me or not ?
>>>
>>>
>>>
>>> On Mon, Jan 23, 2017 at 1:27 PM, Arun Mahadevan <ar...@apache.org>
>>> wrote:
>>>
>>> > builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout");
>>> i found this example but couldn't know why he use number 4 ?
>>>
>>>
>>>
>>> This is the “parallelism hint” (the number of threads) for “MyBolt”. So
>>> in your example there will be 4 threads executing “MyBolt” across the
>>> workers in your cluster and the tuples from “MySpout” would be randomly
>>> distributed across all of the 4 instances of your bolt.
>>>
>>>
>>>
>>> Also see http://storm.apache.org/releases/1.0.1/Understanding-the-par
>>> allelism-of-a-Storm-topology.html
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *From: *sam mohel <sa...@gmail.com>
>>> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
>>> *Date: *Monday, January 23, 2017 at 4:47 PM
>>> *To: *"user@storm.apache.org" <us...@storm.apache.org>
>>> *Subject: *Re: simple question about grouping
>>>
>>>
>>>
>>> excuse me , if i have single spout and single bolt and the bolt doing 2
>>> process so can i do like this
>>> builder.setSpout("MySpout", new mySpout(), 1);
>>> builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i
>>> found this example but couldn't know why he use number 4 ?
>>>
>>>
>>>
>>> On Mon, Jan 23, 2017 at 1:13 PM, sam mohel <sa...@gmail.com> wrote:
>>>
>>> thanks for replying
>>>
>>>
>>>
>>> On Mon, Jan 23, 2017 at 1:14 PM, Arun Mahadevan <ar...@apache.org>
>>> wrote:
>>>
>>>
>>>
>>> Grouping makes sense only when you have more than one task for a bolt.
>>> If your bolt has more than one task, then the grouping will decide how the
>>> tuples from the spout are distributed to the individual tasks of the bolt.
>>> (shuffe = random, fields = keyed on some field and so on).
>>>
>>>
>>>
>>> See http://storm.apache.org/releases/current/Concepts.html
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Arun
>>>
>>>
>>>
>>>
>>>
>>> *From: *sam mohel <sa...@gmail.com>
>>> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
>>> *Date: *Monday, January 23, 2017 at 3:09 PM
>>> *To: *"user@storm.apache.org" <us...@storm.apache.org>, "
>>> dev@storm.apache.org" <de...@storm.apache.org>
>>> *Subject: *simple question about grouping
>>>
>>>
>>>
>>> i have text file contains data . size of this file is 3.5 MB . My
>>> topology consists of one spout and one bolt so is that possible to make all
>>> processing in one bolt and in this case what is the role of grouping here ?
>>>
>>> Thanks in advance
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>
>
> --
> Regards,
> Navin
>
Re: simple question about grouping
Posted by Navin Ipe <na...@searchlighthealth.com>.
More specifically, the bottom of this page explains what the numbers mean
in terms of tasks and executors:
http://nrecursions.blogspot.in/2016/05/more-concepts-of-apache-storm-you-need.html
On Mon, Jan 23, 2017 at 5:28 PM, sam mohel <sa...@gmail.com> wrote:
> really thanks for your help
>
> On Mon, Jan 23, 2017 at 1:59 PM, Arun Mahadevan <ar...@apache.org> wrote:
>
>> There is no magic number, it depends on the specific problem you are
>> trying to solve. You start with some reasonable value for the parallelism
>> and tune it based on your requirements. You could also start with a higher
>> number of “tasks” than the parallelism and then you can rebalance your
>> topology and adjust parallelism on the fly to scale up or down.
>>
>>
>>
>> See the slides from Taylor’s “Scaling Storm” presentation, you might find
>> it useful - http://www.slideshare.net/ptgoetz/scaling-apache-storm-strat
>> a-hadoopworld-2014
>>
>>
>>
>>
>>
>> *From: *sam mohel <sa...@gmail.com>
>> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
>> *Date: *Monday, January 23, 2017 at 4:58 PM
>>
>> *To: *"user@storm.apache.org" <us...@storm.apache.org>
>> *Subject: *Re: simple question about grouping
>>
>>
>>
>> Many thanks , but how and when can i decide that this number is perfect
>> form me or not ?
>>
>>
>>
>> On Mon, Jan 23, 2017 at 1:27 PM, Arun Mahadevan <ar...@apache.org> wrote:
>>
>> > builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout");
>> i found this example but couldn't know why he use number 4 ?
>>
>>
>>
>> This is the “parallelism hint” (the number of threads) for “MyBolt”. So
>> in your example there will be 4 threads executing “MyBolt” across the
>> workers in your cluster and the tuples from “MySpout” would be randomly
>> distributed across all of the 4 instances of your bolt.
>>
>>
>>
>> Also see http://storm.apache.org/releases/1.0.1/Understanding-the-
>> parallelism-of-a-Storm-topology.html
>>
>>
>>
>>
>>
>>
>>
>> *From: *sam mohel <sa...@gmail.com>
>> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
>> *Date: *Monday, January 23, 2017 at 4:47 PM
>> *To: *"user@storm.apache.org" <us...@storm.apache.org>
>> *Subject: *Re: simple question about grouping
>>
>>
>>
>> excuse me , if i have single spout and single bolt and the bolt doing 2
>> process so can i do like this
>> builder.setSpout("MySpout", new mySpout(), 1);
>> builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i
>> found this example but couldn't know why he use number 4 ?
>>
>>
>>
>> On Mon, Jan 23, 2017 at 1:13 PM, sam mohel <sa...@gmail.com> wrote:
>>
>> thanks for replying
>>
>>
>>
>> On Mon, Jan 23, 2017 at 1:14 PM, Arun Mahadevan <ar...@apache.org> wrote:
>>
>>
>>
>> Grouping makes sense only when you have more than one task for a bolt. If
>> your bolt has more than one task, then the grouping will decide how the
>> tuples from the spout are distributed to the individual tasks of the bolt.
>> (shuffe = random, fields = keyed on some field and so on).
>>
>>
>>
>> See http://storm.apache.org/releases/current/Concepts.html
>>
>>
>>
>> Thanks,
>>
>> Arun
>>
>>
>>
>>
>>
>> *From: *sam mohel <sa...@gmail.com>
>> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
>> *Date: *Monday, January 23, 2017 at 3:09 PM
>> *To: *"user@storm.apache.org" <us...@storm.apache.org>, "
>> dev@storm.apache.org" <de...@storm.apache.org>
>> *Subject: *simple question about grouping
>>
>>
>>
>> i have text file contains data . size of this file is 3.5 MB . My
>> topology consists of one spout and one bolt so is that possible to make all
>> processing in one bolt and in this case what is the role of grouping here ?
>>
>> Thanks in advance
>>
>>
>>
>>
>>
>>
>>
>
>
--
Regards,
Navin
Re: simple question about grouping
Posted by sam mohel <sa...@gmail.com>.
really thanks for your help
On Mon, Jan 23, 2017 at 1:59 PM, Arun Mahadevan <ar...@apache.org> wrote:
> There is no magic number, it depends on the specific problem you are
> trying to solve. You start with some reasonable value for the parallelism
> and tune it based on your requirements. You could also start with a higher
> number of “tasks” than the parallelism and then you can rebalance your
> topology and adjust parallelism on the fly to scale up or down.
>
>
>
> See the slides from Taylor’s “Scaling Storm” presentation, you might find
> it useful - http://www.slideshare.net/ptgoetz/scaling-apache-storm-
> strata-hadoopworld-2014
>
>
>
>
>
> *From: *sam mohel <sa...@gmail.com>
> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Date: *Monday, January 23, 2017 at 4:58 PM
>
> *To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Subject: *Re: simple question about grouping
>
>
>
> Many thanks , but how and when can i decide that this number is perfect
> form me or not ?
>
>
>
> On Mon, Jan 23, 2017 at 1:27 PM, Arun Mahadevan <ar...@apache.org> wrote:
>
> > builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i
> found this example but couldn't know why he use number 4 ?
>
>
>
> This is the “parallelism hint” (the number of threads) for “MyBolt”. So in
> your example there will be 4 threads executing “MyBolt” across the workers
> in your cluster and the tuples from “MySpout” would be randomly distributed
> across all of the 4 instances of your bolt.
>
>
>
> Also see http://storm.apache.org/releases/1.0.1/Understanding-
> the-parallelism-of-a-Storm-topology.html
>
>
>
>
>
>
>
> *From: *sam mohel <sa...@gmail.com>
> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Date: *Monday, January 23, 2017 at 4:47 PM
> *To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Subject: *Re: simple question about grouping
>
>
>
> excuse me , if i have single spout and single bolt and the bolt doing 2
> process so can i do like this
> builder.setSpout("MySpout", new mySpout(), 1);
> builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i
> found this example but couldn't know why he use number 4 ?
>
>
>
> On Mon, Jan 23, 2017 at 1:13 PM, sam mohel <sa...@gmail.com> wrote:
>
> thanks for replying
>
>
>
> On Mon, Jan 23, 2017 at 1:14 PM, Arun Mahadevan <ar...@apache.org> wrote:
>
>
>
> Grouping makes sense only when you have more than one task for a bolt. If
> your bolt has more than one task, then the grouping will decide how the
> tuples from the spout are distributed to the individual tasks of the bolt.
> (shuffe = random, fields = keyed on some field and so on).
>
>
>
> See http://storm.apache.org/releases/current/Concepts.html
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> *From: *sam mohel <sa...@gmail.com>
> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Date: *Monday, January 23, 2017 at 3:09 PM
> *To: *"user@storm.apache.org" <us...@storm.apache.org>, "
> dev@storm.apache.org" <de...@storm.apache.org>
> *Subject: *simple question about grouping
>
>
>
> i have text file contains data . size of this file is 3.5 MB . My topology
> consists of one spout and one bolt so is that possible to make all
> processing in one bolt and in this case what is the role of grouping here ?
>
> Thanks in advance
>
>
>
>
>
>
>
Re: simple question about grouping
Posted by Arun Mahadevan <ar...@apache.org>.
There is no magic number, it depends on the specific problem you are trying to solve. You start with some reasonable value for the parallelism and tune it based on your requirements. You could also start with a higher number of “tasks” than the parallelism and then you can rebalance your topology and adjust parallelism on the fly to scale up or down.
See the slides from Taylor’s “Scaling Storm” presentation, you might find it useful - http://www.slideshare.net/ptgoetz/scaling-apache-storm-strata-hadoopworld-2014
From: sam mohel <sa...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Monday, January 23, 2017 at 4:58 PM
To: "user@storm.apache.org" <us...@storm.apache.org>
Subject: Re: simple question about grouping
Many thanks , but how and when can i decide that this number is perfect form me or not ?
On Mon, Jan 23, 2017 at 1:27 PM, Arun Mahadevan <ar...@apache.org> wrote:
> builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i found this example but couldn't know why he use number 4 ?
This is the “parallelism hint” (the number of threads) for “MyBolt”. So in your example there will be 4 threads executing “MyBolt” across the workers in your cluster and the tuples from “MySpout” would be randomly distributed across all of the 4 instances of your bolt.
Also see http://storm.apache.org/releases/1.0.1/Understanding-the-parallelism-of-a-Storm-topology.html
From: sam mohel <sa...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Monday, January 23, 2017 at 4:47 PM
To: "user@storm.apache.org" <us...@storm.apache.org>
Subject: Re: simple question about grouping
excuse me , if i have single spout and single bolt and the bolt doing 2 process so can i do like this
builder.setSpout("MySpout", new mySpout(), 1);
builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i found this example but couldn't know why he use number 4 ?
On Mon, Jan 23, 2017 at 1:13 PM, sam mohel <sa...@gmail.com> wrote:
thanks for replying
On Mon, Jan 23, 2017 at 1:14 PM, Arun Mahadevan <ar...@apache.org> wrote:
Grouping makes sense only when you have more than one task for a bolt. If your bolt has more than one task, then the grouping will decide how the tuples from the spout are distributed to the individual tasks of the bolt. (shuffe = random, fields = keyed on some field and so on).
See http://storm.apache.org/releases/current/Concepts.html
Thanks,
Arun
From: sam mohel <sa...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Monday, January 23, 2017 at 3:09 PM
To: "user@storm.apache.org" <us...@storm.apache.org>, "dev@storm.apache.org" <de...@storm.apache.org>
Subject: simple question about grouping
i have text file contains data . size of this file is 3.5 MB . My topology consists of one spout and one bolt so is that possible to make all processing in one bolt and in this case what is the role of grouping here ?
Thanks in advance
Re: simple question about grouping
Posted by sam mohel <sa...@gmail.com>.
Many thanks , but how and when can i decide that this number is perfect
form me or not ?
On Mon, Jan 23, 2017 at 1:27 PM, Arun Mahadevan <ar...@apache.org> wrote:
> > builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i
> found this example but couldn't know why he use number 4 ?
>
>
>
> This is the “parallelism hint” (the number of threads) for “MyBolt”. So in
> your example there will be 4 threads executing “MyBolt” across the workers
> in your cluster and the tuples from “MySpout” would be randomly distributed
> across all of the 4 instances of your bolt.
>
>
>
> Also see http://storm.apache.org/releases/1.0.1/Understanding-
> the-parallelism-of-a-Storm-topology.html
>
>
>
>
>
>
>
> *From: *sam mohel <sa...@gmail.com>
> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Date: *Monday, January 23, 2017 at 4:47 PM
> *To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Subject: *Re: simple question about grouping
>
>
>
> excuse me , if i have single spout and single bolt and the bolt doing 2
> process so can i do like this
> builder.setSpout("MySpout", new mySpout(), 1);
> builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i
> found this example but couldn't know why he use number 4 ?
>
>
>
> On Mon, Jan 23, 2017 at 1:13 PM, sam mohel <sa...@gmail.com> wrote:
>
> thanks for replying
>
>
>
> On Mon, Jan 23, 2017 at 1:14 PM, Arun Mahadevan <ar...@apache.org> wrote:
>
>
>
> Grouping makes sense only when you have more than one task for a bolt. If
> your bolt has more than one task, then the grouping will decide how the
> tuples from the spout are distributed to the individual tasks of the bolt.
> (shuffe = random, fields = keyed on some field and so on).
>
>
>
> See http://storm.apache.org/releases/current/Concepts.html
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> *From: *sam mohel <sa...@gmail.com>
> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Date: *Monday, January 23, 2017 at 3:09 PM
> *To: *"user@storm.apache.org" <us...@storm.apache.org>, "
> dev@storm.apache.org" <de...@storm.apache.org>
> *Subject: *simple question about grouping
>
>
>
> i have text file contains data . size of this file is 3.5 MB . My topology
> consists of one spout and one bolt so is that possible to make all
> processing in one bolt and in this case what is the role of grouping here ?
>
> Thanks in advance
>
>
>
>
>
Re: simple question about grouping
Posted by Arun Mahadevan <ar...@apache.org>.
> builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i found this example but couldn't know why he use number 4 ?
This is the “parallelism hint” (the number of threads) for “MyBolt”. So in your example there will be 4 threads executing “MyBolt” across the workers in your cluster and the tuples from “MySpout” would be randomly distributed across all of the 4 instances of your bolt.
Also see http://storm.apache.org/releases/1.0.1/Understanding-the-parallelism-of-a-Storm-topology.html
From: sam mohel <sa...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Monday, January 23, 2017 at 4:47 PM
To: "user@storm.apache.org" <us...@storm.apache.org>
Subject: Re: simple question about grouping
excuse me , if i have single spout and single bolt and the bolt doing 2 process so can i do like this
builder.setSpout("MySpout", new mySpout(), 1);
builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i found this example but couldn't know why he use number 4 ?
On Mon, Jan 23, 2017 at 1:13 PM, sam mohel <sa...@gmail.com> wrote:
thanks for replying
On Mon, Jan 23, 2017 at 1:14 PM, Arun Mahadevan <ar...@apache.org> wrote:
Grouping makes sense only when you have more than one task for a bolt. If your bolt has more than one task, then the grouping will decide how the tuples from the spout are distributed to the individual tasks of the bolt. (shuffe = random, fields = keyed on some field and so on).
See http://storm.apache.org/releases/current/Concepts.html
Thanks,
Arun
From: sam mohel <sa...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Monday, January 23, 2017 at 3:09 PM
To: "user@storm.apache.org" <us...@storm.apache.org>, "dev@storm.apache.org" <de...@storm.apache.org>
Subject: simple question about grouping
i have text file contains data . size of this file is 3.5 MB . My topology consists of one spout and one bolt so is that possible to make all processing in one bolt and in this case what is the role of grouping here ?
Thanks in advance
Re: simple question about grouping
Posted by sam mohel <sa...@gmail.com>.
excuse me , if i have single spout and single bolt and the bolt doing 2
process so can i do like this
builder.setSpout("MySpout", new mySpout(), 1);
builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i
found this example but couldn't know why he use number 4 ?
On Mon, Jan 23, 2017 at 1:13 PM, sam mohel <sa...@gmail.com> wrote:
> thanks for replying
>
> On Mon, Jan 23, 2017 at 1:14 PM, Arun Mahadevan <ar...@apache.org> wrote:
>
>>
>>
>> Grouping makes sense only when you have more than one task for a bolt. If
>> your bolt has more than one task, then the grouping will decide how the
>> tuples from the spout are distributed to the individual tasks of the bolt.
>> (shuffe = random, fields = keyed on some field and so on).
>>
>>
>>
>> See http://storm.apache.org/releases/current/Concepts.html
>>
>>
>>
>> Thanks,
>>
>> Arun
>>
>>
>>
>>
>>
>> *From: *sam mohel <sa...@gmail.com>
>> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
>> *Date: *Monday, January 23, 2017 at 3:09 PM
>> *To: *"user@storm.apache.org" <us...@storm.apache.org>, "
>> dev@storm.apache.org" <de...@storm.apache.org>
>> *Subject: *simple question about grouping
>>
>>
>>
>> i have text file contains data . size of this file is 3.5 MB . My
>> topology consists of one spout and one bolt so is that possible to make all
>> processing in one bolt and in this case what is the role of grouping here ?
>>
>> Thanks in advance
>>
>
>
Re: simple question about grouping
Posted by sam mohel <sa...@gmail.com>.
thanks for replying
On Mon, Jan 23, 2017 at 1:14 PM, Arun Mahadevan <ar...@apache.org> wrote:
>
>
> Grouping makes sense only when you have more than one task for a bolt. If
> your bolt has more than one task, then the grouping will decide how the
> tuples from the spout are distributed to the individual tasks of the bolt.
> (shuffe = random, fields = keyed on some field and so on).
>
>
>
> See http://storm.apache.org/releases/current/Concepts.html
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> *From: *sam mohel <sa...@gmail.com>
> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Date: *Monday, January 23, 2017 at 3:09 PM
> *To: *"user@storm.apache.org" <us...@storm.apache.org>, "
> dev@storm.apache.org" <de...@storm.apache.org>
> *Subject: *simple question about grouping
>
>
>
> i have text file contains data . size of this file is 3.5 MB . My topology
> consists of one spout and one bolt so is that possible to make all
> processing in one bolt and in this case what is the role of grouping here ?
>
> Thanks in advance
>
Re: simple question about grouping
Posted by Arun Mahadevan <ar...@apache.org>.
Grouping makes sense only when you have more than one task for a bolt. If your bolt has more than one task, then the grouping will decide how the tuples from the spout are distributed to the individual tasks of the bolt. (shuffe = random, fields = keyed on some field and so on).
See http://storm.apache.org/releases/current/Concepts.html
Thanks,
Arun
From: sam mohel <sa...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Monday, January 23, 2017 at 3:09 PM
To: "user@storm.apache.org" <us...@storm.apache.org>, "dev@storm.apache.org" <de...@storm.apache.org>
Subject: simple question about grouping
i have text file contains data . size of this file is 3.5 MB . My topology consists of one spout and one bolt so is that possible to make all processing in one bolt and in this case what is the role of grouping here ?
Thanks in advance
Re: simple question about grouping
Posted by Arun Mahadevan <ar...@apache.org>.
Grouping makes sense only when you have more than one task for a bolt. If your bolt has more than one task, then the grouping will decide how the tuples from the spout are distributed to the individual tasks of the bolt. (shuffe = random, fields = keyed on some field and so on).
See http://storm.apache.org/releases/current/Concepts.html
Thanks,
Arun
From: sam mohel <sa...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Monday, January 23, 2017 at 3:09 PM
To: "user@storm.apache.org" <us...@storm.apache.org>, "dev@storm.apache.org" <de...@storm.apache.org>
Subject: simple question about grouping
i have text file contains data . size of this file is 3.5 MB . My topology consists of one spout and one bolt so is that possible to make all processing in one bolt and in this case what is the role of grouping here ?
Thanks in advance
Re: simple question about grouping
Posted by sam mohel <sa...@gmail.com>.
i hope someone can help . why the group is inactive ?
On Mon, Jan 23, 2017 at 11:39 AM, sam mohel <sa...@gmail.com> wrote:
> i have text file contains data . size of this file is 3.5 MB . My topology
> consists of one spout and one bolt so is that possible to make all
> processing in one bolt and in this case what is the role of grouping here ?
> Thanks in advance
>
Re: simple question about grouping
Posted by sam mohel <sa...@gmail.com>.
i hope someone can help . why the group is inactive ?
On Mon, Jan 23, 2017 at 11:39 AM, sam mohel <sa...@gmail.com> wrote:
> i have text file contains data . size of this file is 3.5 MB . My topology
> consists of one spout and one bolt so is that possible to make all
> processing in one bolt and in this case what is the role of grouping here ?
> Thanks in advance
>