[jira] [Commented] (KAFKA-2480) Handle non-CopycatExceptions from SinkTasks

["Liquan Pei (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908909#comment-14908909 ] 

Liquan Pei commented on KAFKA-2480:
-----------------------------------

Can you assign this to me?

> Handle non-CopycatExceptions from SinkTasks
> -------------------------------------------
>
>                 Key: KAFKA-2480
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2480
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: copycat
>            Reporter: Ewen Cheslack-Postava
>            Assignee: Ewen Cheslack-Postava
>             Fix For: 0.9.0.0
>
>
> Currently we catch Throwable in WorkerSinkTask, but we just log the exception. This can lead to data loss because it indicates the messages in the {{put(records)}} call probably were not handled properly. We need to decide what the policy for handling these types of exceptions should be -- try repeating the same records again, risking duplication? or skip them, risking loss? or kill the task immediately and require intervention since it's unclear what happened?
> SourceTasks don't have the same concern -- they can throw other exceptions and as long as we catch them, it is up to the connector to ensure that it does not lose data as a result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)