You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by Jason Gustafson <ja...@confluent.io> on 2019/10/09 19:06:27 UTC

[DISCUSS] KIP-533: Add default api timeout to AdminClient

Hi All,

I wrote a short KIP to address a longstanding issue with timeout behavior
in the AdminClient:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-533%3A+Add+default+api+timeout+to+AdminClient
.

Take a look and let me know what you think.

Thanks,
Jason

Re: [DISCUSS] KIP-533: Add default api timeout to AdminClient

Posted by Colin McCabe <cm...@apache.org>.

Hi Jason,

Thanks for the KIP.  I think this is a long-overdue improvement.

It would be good to clarify that the "timeout" values for the options classes will now refer to the API timeouts, not to the request timeouts.  I think this is implied by the KIP, but it would be good to spell it out.

One interesting question is if we should let users configure the request timeout on a per-request basis as well.  I guess if we don't have a clear use-case, we could leave this as a possible future improvement.

I'm a bit concerned that the proposed changes will lead to faulty clients spamming the servers.  In particular, the default retry.backoff.ms is only 100 ms and you are now increasing the number of retries to be effectively unlimited.  I remember we had a discussion offline about this and I commented that we already had exponential backoff, but I was misremembering.  We actually only have it for reconnecting.

To address this, I think we should add exponential backoff here, like we did with reconnect backoff.  We could have a similar retry.backoff.max.ms to set the upper limit for backoff.

best,
Colin

On Wed, Oct 9, 2019, at 12:06, Jason Gustafson wrote:
> Hi All,
> 
> I wrote a short KIP to address a longstanding issue with timeout 
> behavior
> in the AdminClient:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-533%3A+Add+default+api+timeout+to+AdminClient
> .
> 
> Take a look and let me know what you think.
> 
> Thanks,
> Jason
>

Re: [DISCUSS] KIP-533: Add default api timeout to AdminClient

Posted by Jason Gustafson <ja...@confluent.io>.

This seems not very controversial, so I will go ahead and move to a vote in
the next couple days.

Thanks,
Jason

On Wed, Nov 13, 2019 at 10:23 AM Jason Gustafson <ja...@confluent.io> wrote:

> Hey Colin,
>
> Thanks for the suggestions. I've clarified the use of the API overrides
> and the request timeout.
>
> I think exponential backoff is a great idea. Can we do a separate KIP for
> this? I think we can add it to the consumer and producer at the same time.
>
> -Jason
>
> On Mon, Oct 21, 2019 at 2:01 PM Ismael Juma <is...@juma.me.uk> wrote:
>
>> Thanks for the KIP, this makes sense to me. It will be good to align the
>> AdminClient better with the consumer and producer when it comes to
>> timeouts.
>>
>> Ismael
>>
>> On Wed, Oct 9, 2019 at 12:06 PM Jason Gustafson <ja...@confluent.io>
>> wrote:
>>
>> > Hi All,
>> >
>> > I wrote a short KIP to address a longstanding issue with timeout
>> behavior
>> > in the AdminClient:
>> >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-533%3A+Add+default+api+timeout+to+AdminClient
>> > .
>> >
>> > Take a look and let me know what you think.
>> >
>> > Thanks,
>> > Jason
>> >
>>
>

Re: [DISCUSS] KIP-533: Add default api timeout to AdminClient

Posted by Jason Gustafson <ja...@confluent.io>.

Hey Colin,

Thanks for the suggestions. I've clarified the use of the API overrides and
the request timeout.

I think exponential backoff is a great idea. Can we do a separate KIP for
this? I think we can add it to the consumer and producer at the same time.

-Jason

On Mon, Oct 21, 2019 at 2:01 PM Ismael Juma <is...@juma.me.uk> wrote:

> Thanks for the KIP, this makes sense to me. It will be good to align the
> AdminClient better with the consumer and producer when it comes to
> timeouts.
>
> Ismael
>
> On Wed, Oct 9, 2019 at 12:06 PM Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > Hi All,
> >
> > I wrote a short KIP to address a longstanding issue with timeout behavior
> > in the AdminClient:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-533%3A+Add+default+api+timeout+to+AdminClient
> > .
> >
> > Take a look and let me know what you think.
> >
> > Thanks,
> > Jason
> >
>

Re: [DISCUSS] KIP-533: Add default api timeout to AdminClient

Posted by Ismael Juma <is...@juma.me.uk>.

Thanks for the KIP, this makes sense to me. It will be good to align the
AdminClient better with the consumer and producer when it comes to timeouts.

Ismael

On Wed, Oct 9, 2019 at 12:06 PM Jason Gustafson <ja...@confluent.io> wrote:

> Hi All,
>
> I wrote a short KIP to address a longstanding issue with timeout behavior
> in the AdminClient:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-533%3A+Add+default+api+timeout+to+AdminClient
> .
>
> Take a look and let me know what you think.
>
> Thanks,
> Jason
>