[jira] [Created] (KAFKA-9314) Connect put() and poll() retries not conforming to KIP-298

["Nigel Liang (Jira)" <ji...@apache.org>]

Nigel Liang created KAFKA-9314:
----------------------------------

             Summary: Connect put() and poll() retries not conforming to KIP-298
                 Key: KAFKA-9314
                 URL: https://issues.apache.org/jira/browse/KAFKA-9314
             Project: Kafka
          Issue Type: Bug
          Components: KafkaConnect
            Reporter: Nigel Liang
            Assignee: Nigel Liang


KIP-298 outlines the retry policy of Connect when errors are encountered. In particular, it proposes to retry on {{RetriableException}} on put() in SinkTask and poll() in SourceTask.

However, the code does not reflect this change. For instance, WorkerSourceTask handles {{RetriableException}} thrown from {{poll()}} by entering into a tight retry loop without backoff. This has led to connectors having to workaround by simply not retrying and failing the task always (https://github.com/confluentinc/kafka-connect-jms/pull/88). Users would need to manually restart the task to recover from even simple network glitches.

AFAICT from reading code, the same is true for {{WorkerSinkTask}} when calling {{put()}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)