Checkpointing Operator State and Failure

["McCullough, Alex" <Al...@capitalone.com>]

Hey All,

As I understand it, the only thing that should not be defined as transient in an operator is data that changes based on incoming tuples between windows.

If I have a static map I am loading in the constructor or set up method the operator, I am assume I can define it as transient and avoid the overhead of serializing/deserializing it. When an operator fails and a new one is spun up are the constructor and/or setup methods called? I’m assuming so or else I would need to checkpoint this map.

Thanks,
Alex
________________________________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.

Re: Checkpointing Operator State and Failure

[Munagala Ramanath <ra...@datatorrent.com>]

Yes, that should be OK -- the constructor and setup methods are called on
recovery from failure.

Ram

On Mon, Jun 27, 2016 at 6:27 AM, McCullough, Alex <
Alex.McCullough@capitalone.com> wrote:

> Hey All,
>
>
>
> As I understand it, the only thing that should not be defined as transient
> in an operator is data that changes based on incoming tuples between
> windows.
>
>
>
> If I have a static map I am loading in the constructor or set up method
> the operator, I am assume I can define it as transient and avoid the
> overhead of serializing/deserializing it. When an operator fails and a new
> one is spun up are the constructor and/or setup methods called? I’m
> assuming so or else I would need to checkpoint this map.
>
>
>
> Thanks,
>
> Alex
>
> ------------------------------
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>