You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Ksolves (Jira)" <ji...@apache.org> on 2024/06/19 09:58:00 UTC

[jira] [Updated] (KAFKA-16508) Infinite loop if output topic does not exisit

     [ https://issues.apache.org/jira/browse/KAFKA-16508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ksolves updated KAFKA-16508:
----------------------------
    Summary: Infinite loop if output topic does not exisit  (was: Infinte loop if output topic does not exisit)

> Infinite loop if output topic does not exisit
> ---------------------------------------------
>
>                 Key: KAFKA-16508
>                 URL: https://issues.apache.org/jira/browse/KAFKA-16508
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Matthias J. Sax
>            Assignee: Alieh Saeedi
>            Priority: Major
>
> Kafka Streams supports `ProductionExceptionHandler` to drop records on error when writing into an output topic.
> However, if the output topic does not exist, the corresponding error cannot be skipped over because the handler is not called.
> The issue is, that the producer internally retires to fetch the output topic metadata until it times out, an a `TimeoutException` (which is a `RetriableException`) is returned via the registered `Callback`. However, for `RetriableException` there is different code path and the `ProductionExceptionHandler` is not called.
> In general, Kafka Streams correctly tries to handle as many errors a possible internally, and a `RetriableError` falls into this category (and thus there is no need to call the handler). However, for this particular case, just retrying does not solve the issue – it's unclear if throwing a retryable `TimeoutException` is actually the right thing to do for the Producer? Also not sure what the right way to address this ticket would be (currently, we cannot really detect this case, except if we would do some nasty error message String comparison what sounds hacky...)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)