[jira] [Updated] (KAFKA-10273) Connect Converters should produce actionable error messages

["Randall Hauch (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Randall Hauch updated KAFKA-10273:
----------------------------------
    Labels: newbie  (was: )

> Connect Converters should produce actionable error messages
> -----------------------------------------------------------
>
>                 Key: KAFKA-10273
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10273
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>            Reporter: Greg Harris
>            Priority: Minor
>              Labels: newbie
>
> When runtime exceptions are thrown from converters, the current error messages are descriptive, but not actionable. For example:
> {noformat}
> Error encountered in task task-0. Executing stage 'VALUE_CONVERTER' with class 'org.apache.kafka.connect.json.JsonConverter', where consumed record is <snipped>
> Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka Connect data failed due to serialization error:
>  at org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:334)
> <snipped>
> Caused by: org.apache.kafka.common.errors.SerializationException: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('b' (code 98)): was expecting double-quote to start field name
>  at [Source: (byte[])"{brokenjson-:"bar 1"}"; line: 1, column: 3]{noformat}
> This describes the error accurately, explaining when it happened, and what caused the exception to be thrown. However, after receiving this error it's still unclear what remediation step to take next.
> Instead, in addition to the existing descriptive context and instigating error being displayed, these error messages should give users a lead to begin investigating and remediating the error.
> There are a few changes that users may use to remediate exceptions from converters:
>  * Reconfigure the converter (change schemas.enable on the JsonConverter to align with the topic contents)
>  * Use different data (advancing sink/source offsets manually to skip bad records, or changing topics)
>  * Remediate the converter's environment failure (resolving network faults or reconfiguring external systems)
> An error message which suggests manual inspection of the converter configuration would be user-actionable, and could be generically applied across converters without knowledge of specific configurations. This would point users towards the most common remediation step, reconfiguring the converter.
> For sink connectors we can also indicate that bad data in a topic may be skipped in a number of ways, such as changing offsets, ignoring errors, writing to a DLQ, etc. It's unclear which of these is worth suggesting, and it may change if an alternative strategy feature exists in the future. 
> Non-Goals/ Out-of-scope:
> The third type of remediation of the converter's environment is outside the scope of this issue, and requires converter-specific knowledge in order to make a suggestion.
> This change is not attempting to add converter-specific error messaging, and the framework messaging here needs to be converter-agnostic, and as such can only point out general classes of errors and remediation steps, not specific ones. We can programmatically generate lists of relevant configurations, but can't single any out as being the cause. We can say that the data is incompatible with the converter, but can't say why exactly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)