You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Randall Hauch (JIRA)" <ji...@apache.org> on 2018/04/02 15:06:00 UTC
[jira] [Created] (KAFKA-6738) Kafka Connect handling of bad data
Randall Hauch created KAFKA-6738:
------------------------------------
Summary: Kafka Connect handling of bad data
Key: KAFKA-6738
URL: https://issues.apache.org/jira/browse/KAFKA-6738
Project: Kafka
Issue Type: Improvement
Components: KafkaConnect
Affects Versions: 1.1.0
Reporter: Randall Hauch
Assignee: Konstantine Karantasis
Kafka Connect connectors and tasks fail when they run into an unexpected situation or error, but the framework should provide more general "bad data handling" options, including (perhaps among others):
# fail fast, which is what we do today (assuming connector actually fails and doesn't eat errors)
# retry (possibly with configs to limit)
# drop data and move on
# dead letter queue
This needs to be addressed in a way that handles errors from:
# The connector itself (e.g. connectivity issues to the other system)
# Converters/serializers (bad data, unexpected format, etc)
# SMTs
# Ideally the framework as well, though we obviously want to fix known bugs anyway
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)