[jira] [Updated] (KAFKA-10297) Don't use deprecated producer config `retries`

["Matthias J. Sax (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-10297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias J. Sax updated KAFKA-10297:
------------------------------------
    Description: 
In 2.7.0 release, producer config `retries` gets deprecated via KIP-572.

Connect is still using this config what needs to be fixed (cf [https://github.com/apache/kafka/pull/8864/files#r439685920])
{quote}Btw: @hachikuji raise a concern about this issue, too: https://github.com/apache/kafka/pull/8864#pullrequestreview-443424531

> I just had one question about the proposal. Using retries=0 in the producer allows the user to have "at-most-once" delivery. This allows the application to implement its own retry logic for example. Do we still have a way to do this once this configuration is gone?

So maybe we need to do some follow up work in the `Producer` to make it work for Connect. But I would defer this to the follow up work.

My original though was, that setting `max.deliver.timeout.ms := request .timeout.ms` might prevent internal retries. But we need to verify this. It was also brought to my attention, that this might not work if the network disconnects -- only `retries=0` would prevent to re-open the connection but a low timeout would not prevent retries.

In KIP-572, we proposed for Kafka Streams itself, to treat `task.timeout.ms = 0` as "no retries" -- maybe we can do a similar thing for the producer?

There is also `max.block.ms` that we should consider. Unfortunately, I am not an expert on the `Producer`. But I would like to move forward with KIP-572 for now and are happy to help to resolve those questions.
{quote}

  was:
In 2.7.0 release, producer config `retries` gets deprecated via KIP-572.

Connect is still using this config what needs to be fixed.


> Don't use deprecated producer config `retries`
> ----------------------------------------------
>
>                 Key: KAFKA-10297
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10297
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>    Affects Versions: 2.7.0
>            Reporter: Matthias J. Sax
>            Priority: Blocker
>             Fix For: 2.7.0
>
>
> In 2.7.0 release, producer config `retries` gets deprecated via KIP-572.
> Connect is still using this config what needs to be fixed (cf [https://github.com/apache/kafka/pull/8864/files#r439685920])
> {quote}Btw: @hachikuji raise a concern about this issue, too: https://github.com/apache/kafka/pull/8864#pullrequestreview-443424531
> > I just had one question about the proposal. Using retries=0 in the producer allows the user to have "at-most-once" delivery. This allows the application to implement its own retry logic for example. Do we still have a way to do this once this configuration is gone?
> So maybe we need to do some follow up work in the `Producer` to make it work for Connect. But I would defer this to the follow up work.
> My original though was, that setting `max.deliver.timeout.ms := request .timeout.ms` might prevent internal retries. But we need to verify this. It was also brought to my attention, that this might not work if the network disconnects -- only `retries=0` would prevent to re-open the connection but a low timeout would not prevent retries.
> In KIP-572, we proposed for Kafka Streams itself, to treat `task.timeout.ms = 0` as "no retries" -- maybe we can do a similar thing for the producer?
> There is also `max.block.ms` that we should consider. Unfortunately, I am not an expert on the `Producer`. But I would like to move forward with KIP-572 for now and are happy to help to resolve those questions.
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)