[jira] [Resolved] (KAFKA-6490) JSON SerializationException Stops Connect

["Ewen Cheslack-Postava (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-6490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ewen Cheslack-Postava resolved KAFKA-6490.
------------------------------------------
       Resolution: Fixed
    Fix Version/s: 2.0.0

Closing as this is effectively fixed by [https://cwiki.apache.org/confluence/display/KAFKA/KIP-298%3A+Error+Handling+in+Connect] which allows you to configure how errors are handled, and should apply to errors in Converters, Transformations, and Connectors.

> JSON SerializationException Stops Connect
> -----------------------------------------
>
>                 Key: KAFKA-6490
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6490
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 1.0.0
>            Reporter: William R. Speirs
>            Assignee: Prasanna Subburaj
>            Priority: Major
>             Fix For: 2.0.0
>
>         Attachments: KAFKA-6490_v1.patch
>
>
> If you configure KafkaConnect to parse JSON messages, and you send it a non-JSON message, the SerializationException message will bubble up to the top, and stop KafkaConnect. While I understand sending non-JSON to a JSON serializer is a bad idea, I think that a single malformed message stopping all of KafkaConnect is even worse.
> The data exception is thrown here: [https://github.com/apache/kafka/blob/trunk/connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java#L305]
>  
> From the call here: [https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L476]
> This bubbles all the way up to the top, and KafkaConnect simply stops with the message: {{ERROR WorkerSinkTask\{id=elasticsearch-sink-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:172)}}
> Thoughts on adding a {{try/catch}} around the {{for}} loop in WorkerSinkTask's {{convertMessages}} so messages that don't properly parse are logged, but simply ignored? This way KafkaConnect can keep working even when it encounters a message it cannot decode?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)