Worker Task id during recovery

[Manikandan <ma...@gmail.com>]

Hi

When the worker dies and brought back by the supervisor, whats the task id
given to the new worker? Is it always same/different as/from the old one?

Thanks & Regards
Manikandan

Re: Worker Task id during recovery

[Noel Milton Vega <nm...@ComputingArchitects.Com>]

Yes... and I had to figure out the answer to that same question about 
two months ago myself. It was a question I
asked myself generally as:

      What run-time attributes of a task are remembered/persisted across 
task fail & restart sequences?.

Stated that way, the question involved not only task-id and task-index 
persistence, but also whether Storm might
redistribute any Field Group that the failed task may have been 
receiving, to surviving tasks... after all, the
in-memory state of the failed task was gone, so it's was possible to 
redistribute it's Fields Group somewhere
else (in the interest of keeping things moving in Storm).

I couldn't find documentation on this and an understanding was necessary 
because, for my purposes, it determined
whether-or-not (and how) I could recover the in-memory state of a failed 
task (without replay).

So I took to instrumenting tests using REDIS Sets which, because of item 
uniqueness in Sets, allowed me to easily
spot run-time attribute changes of a task before and after simulated 
failures (via UNIX 'kill -9').

I wrote up my setup and findings in following little PDF document. 
Basically what I found, at least anecdotally,
was that the task-ids, task-indices, as well as their Fields Groups (if 
any) were preserved. This, in turn,
meant I could create task-unique identifiers like so, 'fqid = 
componentID + . + taskIndex', as unique keys
into REDIS datastructures from which I could recover because, 
essentially, 'fqid' didn't change before and after
failure.

Maybe this will help you...

http://didata.us/assets.d/DiData-StormRedis-Portfolio.pdf


Regards,
Noel

On 03/14/2014 06:35 PM, Rick.C.Rankin@kp.org wrote:
> I believe once the topology is submitted you can count on the task 
> id's remaining the same. I use this to reload object data if one is 
> restarted. They could move to a different slot but should have the 
> same id.
>
>
>
> Rick Rankin
> Principal, Analytics & Assurance Services
> Technical Risk Office: Cyber Security
> Kaiser Permanente
> Phone: 510.847.5269
>
>
>
>
> *NOTICE TO RECIPIENT:* If you are not the intended recipient of this 
> e-mail, you are prohibited from sharing, copying, or otherwise using 
> or disclosing its contents.  If you have received this e-mail in 
> error, please notify the sender immediately by reply e-mail and 
> permanently delete this e-mail and any attachments without reading, 
> forwarding or saving them.  Thank you.
>
>
>
>
>
> From: Manikandan <ma...@gmail.com>
> To: user@storm.incubator.apache.org
> Date: 03/14/2014 03:21 PM
> Subject: Worker Task id during recovery
> ------------------------------------------------------------------------
>
>
>
> Hi
>
> When the worker dies and brought back by the supervisor, whats the 
> task id given to the new worker? Is it always same/different as/from 
> the old one?
>
> Thanks & Regards
> Manikandan

Re: Worker Task id during recovery

[Ri...@kp.org]

I believe once the topology is submitted you can count on the task id's 
remaining the same. I use this to reload object data if one is restarted. 
They could move to a different slot but should have the same id.
 
 

Rick Rankin 
Principal, Analytics & Assurance Services
Technical Risk Office: Cyber Security
Kaiser Permanente
Phone: 510.847.5269 




NOTICE TO RECIPIENT:  If you are not the intended recipient of this 
e-mail, you are prohibited from sharing, copying, or otherwise using or 
disclosing its contents.  If you have received this e-mail in error, 
please notify the sender immediately by reply e-mail and permanently 
delete this e-mail and any attachments without reading, forwarding or 
saving them.  Thank you.





From:   Manikandan <ma...@gmail.com>
To:     user@storm.incubator.apache.org
Date:   03/14/2014 03:21 PM
Subject:        Worker Task id during recovery



Hi

When the worker dies and brought back by the supervisor, whats the task id 
given to the new worker? Is it always same/different as/from the old one?

Thanks & Regards
Manikandan