Handling task process failure

[Dmitry Minkovsky <dm...@gmail.com>]

Suppose my task's process method fails for some reason. Say the problem is
not in the message but is attributable to some external condition: a
service is down, a network partition.

What is the best way to handle such failures? Currently I am just what
seems natural and writing the failures to a topic.  Is there a better way?


Thank you,
Dmitry

Re: Handling task process failure

[Yi Pan <ni...@gmail.com>]

Hi, Dmitry,

There isn't the best way for all scenarios, IMO. For example, if the
exception is critical and the application can not afford to ignore the
failure, throw the exception uncaught is proper, which would fail the
container and allows the application to restart from the previous
checkpointed offsets; on the other hand, if the error can be swallowed and
ignored and the application should proceed forward, capture the exception
and writing to an error topic then proceed w/ the application seems to be
the proper approach. It is mostly up to the application developer to
evaluate the severity of the failure and decide whether to stop the
application or continue.

-Yi

On Thu, Jul 16, 2015 at 11:33 AM, Dmitry Minkovsky <dm...@gmail.com>
wrote:

> Suppose my task's process method fails for some reason. Say the problem is
> not in the message but is attributable to some external condition: a
> service is down, a network partition.
>
> What is the best way to handle such failures? Currently I am just what
> seems natural and writing the failures to a topic.  Is there a better way?
>
>
> Thank you,
> Dmitry
>