You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Jins George <ge...@gmail.com> on 2016/07/18 22:34:59 UTC

Question on Storm 1.0 State Management

Hello Group,

I have a question on the implementation of State management in Storm(Storm
1.0 - IStatefulBolt) .  Is the sate is shared across instances of same Bolt
?

Say I have a  bolt with  parallelism as 5, Does each bolt instance have its
own copy of State or there is only one State which is shared across 5
instances of bolts


Thanks,
Jins George

Re: Question on Storm 1.0 State Management

Posted by Arun Mahadevan <ar...@apache.org>.
The state is mapped based on the task id, so yes in the case of fields grouping the ids might get remapped to a different task id. One option would be to re-hash the values in the state based on the new number of tasks. Right now there is no easy way (or scripts) for doing this re-hashing and It would work only if the grouping field is used as the state key. Another way could be to start with a clean state (e.g. truncate redis tables where state is stored) and re-emit the last saved values after you rebalance so that it gets to the right task and have the blots init from replayed state. I am not sure how well this would work out for your use case.

 

Thanks,

Arun

 

From: Jins George <ge...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Tuesday, July 19, 2016 at 10:12 AM
To: "user@storm.apache.org" <us...@storm.apache.org>
Subject: Re: Question on Storm 1.0 State Management

 

Thanks Arun for the clarification. A follow up question on the same when Increasing/Decreasing the Parallelism of a Bolt.

If I use field grouping on some ID field and I track state  in Bolt for that ID, after a parallelism increase, it is possible to have the state maintained in a different bolt instance right ?

Any suggestion on  handling such cases ?

 

Thanks,

Jins George

 

On Mon, Jul 18, 2016 at 9:17 PM, Arun Mahadevan <ar...@apache.org> wrote:

Each bolt instance (task) has its own state, so in your case each of the 5 instances would have its own state. All these state instances could be in a same underlying storage instance (e.g. same Redis cluster).

 

Thanks,

Arun

 

 

 

From: Jins George <ge...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Tuesday, July 19, 2016 at 4:04 AM
To: "user@storm.apache.org" <us...@storm.apache.org>
Subject: Question on Storm 1.0 State Management

 

Hello Group,

I have a question on the implementation of State management in Storm(Storm 1.0 - IStatefulBolt) .  Is the sate is shared across instances of same Bolt ? 

Say I have a  bolt with  parallelism as 5, Does each bolt instance have its own copy of State or there is only one State which is shared across 5 instances of bolts


Thanks,
Jins George




-- 

Thanks,
Jins George

Re: Question on Storm 1.0 State Management

Posted by Jins George <ge...@gmail.com>.
Thanks Arun for the clarification. A follow up question on the same when
Increasing/Decreasing the Parallelism of a Bolt.
If I use field grouping on some ID field and I track state  in Bolt for
that ID, after a parallelism increase, it is possible to have the state
maintained in a different bolt instance right ?

Any suggestion on  handling such cases ?

Thanks,
Jins George

On Mon, Jul 18, 2016 at 9:17 PM, Arun Mahadevan <ar...@apache.org> wrote:

> Each bolt instance (task) has its own state, so in your case each of the 5
> instances would have its own state. All these state instances could be in a
> same underlying storage instance (e.g. same Redis cluster).
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
>
>
> *From: *Jins George <ge...@gmail.com>
> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Date: *Tuesday, July 19, 2016 at 4:04 AM
> *To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Subject: *Question on Storm 1.0 State Management
>
>
>
> Hello Group,
>
> I have a question on the implementation of State management in Storm(Storm
> 1.0 - IStatefulBolt) .  Is the sate is shared across instances of same
> Bolt ?
>
> Say I have a  bolt with  parallelism as 5, Does each bolt instance have
> its own copy of State or there is only one State which is shared across 5
> instances of bolts
>
>
> Thanks,
> Jins George
>



-- 
Thanks,
Jins George

Re: Question on Storm 1.0 State Management

Posted by Arun Mahadevan <ar...@apache.org>.
Each bolt instance (task) has its own state, so in your case each of the 5 instances would have its own state. All these state instances could be in a same underlying storage instance (e.g. same Redis cluster).

 

Thanks,

Arun

 

 

 

From: Jins George <ge...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Tuesday, July 19, 2016 at 4:04 AM
To: "user@storm.apache.org" <us...@storm.apache.org>
Subject: Question on Storm 1.0 State Management

 

Hello Group,

I have a question on the implementation of State management in Storm(Storm 1.0 - IStatefulBolt) .  Is the sate is shared across instances of same Bolt ? 

Say I have a  bolt with  parallelism as 5, Does each bolt instance have its own copy of State or there is only one State which is shared across 5 instances of bolts


Thanks,
Jins George