You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Pavel Moukhataev <m_...@mail.ru.INVALID> on 2018/09/24 10:48:11 UTC
[DISCUSS] Make org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable
I'd like to introduce new feature for kafka client:
Making org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable
Here is KPI
https://cwiki.apache.org/confluence/display/KAFKA/KIP-375%3A+Make+org.apache.kafka.clients.Metadata%23TOPIC_EXPIRY_MS+configurable
The problem is: if application sends records to some topic rarely then
topic metadata gets expired and sending thread is blocked to wait topic
metadata.
Easy fix is to make TOPIC_EXPIRY_MS configurable.
--
Pavel
+7-903-258-5544
skype://pavel.moukhataev
[DISCUSS] KIP-375 Kafka Clients - make Metadata#TOPIC_EXPIRY_MS configurable
Posted by Pavel Moukhataev <m_...@mail.ru.INVALID>.
Hello
https://cwiki.apache.org/confluence/display/KAFKA/KIP-375%3A+Kafka+Clients+-+make+Metadata%23TOPIC_EXPIRY_MS+configurable
I'd like to introduce new feature for kafka client:
Making org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable
Here is KPI
https://cwiki.apache.org/confluence/display/KAFKA/KIP-375%3A+Make+org.apache.kafka.clients.Metadata%23TOPIC_EXPIRY_MS+configurable
The problem is: if application sends records to some topic rarely then
topic metadata gets expired and sending thread is blocked to wait topic
metadata.
Easy fix is to make TOPIC_EXPIRY_MS configurable and make it higher than
topic send interval.
--
Pavel
+7-903-258-5544
skype://pavel.moukhataev
Re: [DISCUSS] Make org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable
Posted by Dongjin Lee <do...@apache.org>.
Hi Pavel,
Thanks for a great proposal. However, the code freeze of 2.1.0 is imminent,
so it seems like all the committers are on working frenzy. How about
discussing this KIP after the release of 2.1.0? I promise, I will reboot
this discussion as soon as 2.1.0 is released. I also experienced a similar
situation in the past.
Best,
Dongjin
On Mon, Sep 24, 2018 at 7:48 PM Pavel Moukhataev <m_...@mail.ru.invalid>
wrote:
> I'd like to introduce new feature for kafka client:
> Making org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable
> Here is KPI
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-375%3A+Make+org.apache.kafka.clients.Metadata%23TOPIC_EXPIRY_MS+configurable
>
> The problem is: if application sends records to some topic rarely then
> topic metadata gets expired and sending thread is blocked to wait topic
> metadata.
>
> Easy fix is to make TOPIC_EXPIRY_MS configurable.
>
> --
> Pavel
> +7-903-258-5544
> skype://pavel.moukhataev
>
--
*Dongjin Lee*
*A hitchhiker in the mathematical world.*
*github: <http://goog_969573159/>github.com/dongjinleekr
<http://github.com/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr
<http://kr.linkedin.com/in/dongjinleekr>slideshare:
www.slideshare.net/dongjinleekr
<http://www.slideshare.net/dongjinleekr>*
Fwd: [DISCUSS] Make org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable
Posted by Pavel Moukhataev <m_...@mail.ru.INVALID>.
Hello
I'd like to introduce new feature for kafka client:
Making org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable
Here is KPI
https://cwiki.apache.org/confluence/display/KAFKA/KIP-375%3A+Make+org.apache.kafka.clients.Metadata%23TOPIC_EXPIRY_MS+configurable
The problem is: if application sends records to some topic rarely then
topic metadata gets expired and sending thread is blocked to wait topic
metadata.
Easy fix is to make TOPIC_EXPIRY_MS configurable.
--
Pavel
+7-903-258-5544
skype://pavel.moukhataev
Re: [DISCUSS] KIP-375 Kafka Clients - make Metadata#TOPIC_EXPIRY_MS configurable
Posted by Viktor Somogyi-Vass <vi...@gmail.com>.
Hi Pavel,
I have changed my KIP's number to 377 so all sorted out.
Cheers,
Viktor
On Wed, Sep 26, 2018 at 4:53 PM Viktor Somogyi-Vass <vi...@gmail.com>
wrote:
> Hi Pavel,
>
> May I kindly ask you to increment your KIP number? :)
> As it turns out I have already created one with this number a little bit
> earlier. Apologies for the inconvenience.
>
> Viktor
>
> On Wed, Sep 26, 2018 at 4:48 PM Pavel Moukhataev <m_...@mail.ru.invalid>
> wrote:
>
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-375%3A+Kafka+Clients+-+make+Metadata%23TOPIC_EXPIRY_MS+configurable
>>
>> I'd like to introduce new feature for kafka client:
>> Making org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable
>>
>> The problem is: if application sends records to some topic rarely then
>> topic metadata gets expired and sending thread is blocked by waiting topic
>> metadata.
>>
>> Easy fix of this exact problem is to make Metadata#TOPIC_EXPIRY_MS
>> configurable and set it value higher than record sending period.
>>
>> --
>> Pavel
>> +7-903-258-5544
>> skype://pavel.moukhataev
>>
>
Re: [DISCUSS] KIP-375 Kafka Clients - make Metadata#TOPIC_EXPIRY_MS configurable
Posted by Viktor Somogyi-Vass <vi...@gmail.com>.
Hi Pavel,
May I kindly ask you to increment your KIP number? :)
As it turns out I have already created one with this number a little bit
earlier. Apologies for the inconvenience.
Viktor
On Wed, Sep 26, 2018 at 4:48 PM Pavel Moukhataev <m_...@mail.ru.invalid>
wrote:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-375%3A+Kafka+Clients+-+make+Metadata%23TOPIC_EXPIRY_MS+configurable
>
> I'd like to introduce new feature for kafka client:
> Making org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable
>
> The problem is: if application sends records to some topic rarely then
> topic metadata gets expired and sending thread is blocked by waiting topic
> metadata.
>
> Easy fix of this exact problem is to make Metadata#TOPIC_EXPIRY_MS
> configurable and set it value higher than record sending period.
>
> --
> Pavel
> +7-903-258-5544
> skype://pavel.moukhataev
>
[DISCUSS] KIP-375 Kafka Clients - make Metadata#TOPIC_EXPIRY_MS configurable
Posted by Pavel Moukhataev <m_...@mail.ru.INVALID>.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-375%3A+Kafka+Clients+-+make+Metadata%23TOPIC_EXPIRY_MS+configurable
I'd like to introduce new feature for kafka client:
Making org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable
The problem is: if application sends records to some topic rarely then
topic metadata gets expired and sending thread is blocked by waiting topic
metadata.
Easy fix of this exact problem is to make Metadata#TOPIC_EXPIRY_MS
configurable and set it value higher than record sending period.
--
Pavel
+7-903-258-5544
skype://pavel.moukhataev