[jira] [Created] (KAFKA-10273) Connect Converters should produce actionable error messages

["Greg Harris (Jira)" <ji...@apache.org>]

Greg Harris created KAFKA-10273:
-----------------------------------

             Summary: Connect Converters should produce actionable error messages
                 Key: KAFKA-10273
                 URL: https://issues.apache.org/jira/browse/KAFKA-10273
             Project: Kafka
          Issue Type: Improvement
          Components: KafkaConnect
            Reporter: Greg Harris


When runtime exceptions are thrown from converters, the current error messages are descriptive, but not actionable. For example:
{noformat}
Error encountered in task task-0. Executing stage 'VALUE_CONVERTER' with class 'org.apache.kafka.connect.json.JsonConverter', where consumed record is <snipped>
Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka Connect data failed due to serialization error:
 at org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:334)
<snipped>
Caused by: org.apache.kafka.common.errors.SerializationException: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('b' (code 98)): was expecting double-quote to start field name
 at [Source: (byte[])"{brokenjson-:"bar 1"}"; line: 1, column: 3]{noformat}
This describes the error accurately, explaining when it happened, and what caused the exception to be thrown. However, after receiving this error it's still unclear what remediation step to take next.

Instead, in addition to the existing descriptive context and instigating error being displayed, these error messages should give users a lead to begin investigating and remediating the error.


There are a few changes that users may use to remediate exceptions from converters:
 * Reconfigure the converter (change schemas.enable on the JsonConverter to align with the topic contents)
 * Use different data (advancing sink/source offsets manually to skip bad records, or changing topics)
 * Remediate the converter's environment failure (resolving network faults or reconfiguring external systems)

An error message which suggests manual inspection of the converter configuration would be user-actionable, and could be generically applied across converters without knowledge of specific configurations. This would point users towards the most common remediation step, reconfiguring the converter.

We can also indicate that bad data in a topic may be skipped in a number of ways, such as changing offsets, ignoring errors, writing to a DLQ, etc. It's unclear which of these is worth suggesting, and it may change if an alternative strategy feature exists in the future. 

Non-Goals/ Out-of-scope:

The third type of remediation of the converter's environment is outside the scope of this issue, and requires converter-specific knowledge in order to make a suggestion.

This change is not attempting to add converter-specific error messaging, and the framework messaging here needs to be converter-agnostic, and as such can only point out general classes of errors and remediation steps, not specific ones. We can programmatically generate lists of relevant configurations, but can't single any out as being the cause. We can say that the data is incompatible with the converter, but can't say why exactly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)