You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Madhukar Bharti <bh...@gmail.com> on 2015/04/16 16:40:06 UTC
Why fetching meta-data for topic is done three times?
Hi All,
I came across a problem, If we use broker IP which is not reachable or out
of network. Then it takes more than configured time(request.timeout.ms).
After checking the log got to know that it is trying to fetch topic
meta-data three times by changing correlation id.
Due to this even though i keep (request.timeout.ms=1000) It takes 3 sec to
throw Exception. I am using Kafka0.8.1.1 with patch
https://issues.apache.org/jira/secure/attachment/12678547/kafka-1733-add-connectTimeoutMs.patch
I have attached the log. Please check this and clarify why it is behaving
like this. Whether it is by design or have to set some other property also.
Regards
Madhukar
Re: Why fetching meta-data for topic is done three times?
Posted by Madhukar Bharti <bh...@gmail.com>.
Hi Zakee,
Yes even if retries is set to 1 it takes 3 sec to throw an exception. I am
not sure in producer code whether they intentionally swallowing all
exception while doing send which internally does 3 tasks. If 1st itself get
failed whether to proceed for next is required?
Now for throwing exception in 1 sec we have set request.timeout.ms=300ms.
I am not sure whether putting less value for this will work always. If
there is some disturbance in n/w ping itself take about 600 ms.
What is recommended value should we use for this?
Thanks
Madhukar
On Fri, May 1, 2015 at 3:52 AM, Zakee <kz...@netzero.net> wrote:
> With reties 1 you still see the 3 secs delay? The idea is, you can change
> these property to reduce the time to throw exception to 1 secs or below.
> Does that help?
>
> Thanks
> Zakee
>
>
>
> > On Apr 28, 2015, at 10:29 PM, Madhukar Bharti <bh...@gmail.com>
> wrote:
> >
> > Hi Zakee,
> >
> >> message.send.max.retries is 1
> >
> > Regards,
> > Madhukar
> >
> > On Tue, Apr 28, 2015 at 6:17 PM, Madhukar Bharti <
> bhartimadhukar@gmail.com <ma...@gmail.com>>
> > wrote:
> >
> >> Hi Zakee,
> >>
> >> Thanks for your reply.
> >>
> >>> message.send.max.retries
> >> 3
> >>
> >>> retry.backoff.ms
> >> 100
> >>
> >>> topic.metadata.refresh.interval.ms
> >> 600*1000
> >>
> >> This is my properties.
> >>
> >> Regards,
> >> Madhukar
> >>
> >> On Tue, Apr 28, 2015 at 3:26 AM, Zakee <kz...@netzero.net> wrote:
> >>
> >>> What values do you have for below properties? Or are these set to
> >>> defaults?
> >>>
> >>> message.send.max.retries
> >>> retry.backoff.ms
> >>> topic.metadata.refresh.interval.ms
> >>>
> >>> Thanks
> >>> Zakee
> >>>
> >>>
> >>>
> >>>> On Apr 23, 2015, at 11:48 PM, Madhukar Bharti <
> bhartimadhukar@gmail.com>
> >>> wrote:
> >>>>
> >>>> Hi All,
> >>>>
> >>>> Once gone through code found that, While Producer starts it does three
> >>>> things:
> >>>>
> >>>> 1. Sends Meta-data request
> >>>> 2. Send message to broker(fetching broker list)
> >>>> 3. If number of message to be produce is grater than 0 then again
> tries
> >>> to
> >>>> refresh metadata for outstanding produce requests.
> >>>>
> >>>> Each of the request takes configured timeout and go to next and
> finally
> >>>> once all is done then it will throw Exception(if 3 also fails).
> >>>>
> >>>> Here the problem is, if we set timeout as 1 sec then to throw an
> >>> exception
> >>>> It takes 3 sec, so user request will be hanged up till 3 sec, that is
> >>>> comparatively high for response time and if all threads will be
> blocked
> >>> due
> >>>> to producer send then whole application will be blocked for 3 sec. So
> we
> >>>> want to reduce this time to 1 sec. in overall to throw Exception.
> >>>>
> >>>> What is the possible way to do this?
> >>>>
> >>>> Thanks
> >>>> Madhukar
> >>>>
> >>>> On Thu, Apr 16, 2015 at 8:10 PM, Madhukar Bharti <
> >>> bhartimadhukar@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Hi All,
> >>>>>
> >>>>> I came across a problem, If we use broker IP which is not reachable
> or
> >>> out
> >>>>> of network. Then it takes more than configured time(
> request.timeout.ms
> >>> ).
> >>>>> After checking the log got to know that it is trying to fetch topic
> >>>>> meta-data three times by changing correlation id.
> >>>>> Due to this even though i keep (request.timeout.ms=1000) It takes 3
> >>> sec
> >>>>> to throw Exception. I am using Kafka0.8.1.1 with patch
> >>>>>
> >>>
> https://issues.apache.org/jira/secure/attachment/12678547/kafka-1733-add-connectTimeoutMs.patch
> >>>>>
> >>>>>
> >>>>> I have attached the log. Please check this and clarify why it is
> >>> behaving
> >>>>> like this. Whether it is by design or have to set some other property
> >>> also.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Regards
> >>>>> Madhukar
> >>>>>
> >>>>>
> >>>>>
> >>>> ____________________________________________________________
> >>>> Want to place your ad here?
> >>>> Advertise on United Online
> >>>>
> >>>
> http://thirdpartyoffers.netzero.net/TGL3255/5539ed87d69846d871dafmp08duc
> >>>
> >>>
> >>
> >>
> > ____________________________________________________________
> > The WORST exercise for aging
> > Avoid this "healthy" exercise to look & feel 5-10 years YOUNGER
> > http://thirdpartyoffers.netzero.net/TGL3255/5540b94620e14394636c0mp13duc
> <http://thirdpartyoffers.netzero.net/TGL3255/5540b94620e14394636c0mp13duc>
Re: Why fetching meta-data for topic is done three times?
Posted by Zakee <kz...@netzero.net>.
With reties 1 you still see the 3 secs delay? The idea is, you can change these property to reduce the time to throw exception to 1 secs or below. Does that help?
Thanks
Zakee
> On Apr 28, 2015, at 10:29 PM, Madhukar Bharti <bh...@gmail.com> wrote:
>
> Hi Zakee,
>
>> message.send.max.retries is 1
>
> Regards,
> Madhukar
>
> On Tue, Apr 28, 2015 at 6:17 PM, Madhukar Bharti <bhartimadhukar@gmail.com <ma...@gmail.com>>
> wrote:
>
>> Hi Zakee,
>>
>> Thanks for your reply.
>>
>>> message.send.max.retries
>> 3
>>
>>> retry.backoff.ms
>> 100
>>
>>> topic.metadata.refresh.interval.ms
>> 600*1000
>>
>> This is my properties.
>>
>> Regards,
>> Madhukar
>>
>> On Tue, Apr 28, 2015 at 3:26 AM, Zakee <kz...@netzero.net> wrote:
>>
>>> What values do you have for below properties? Or are these set to
>>> defaults?
>>>
>>> message.send.max.retries
>>> retry.backoff.ms
>>> topic.metadata.refresh.interval.ms
>>>
>>> Thanks
>>> Zakee
>>>
>>>
>>>
>>>> On Apr 23, 2015, at 11:48 PM, Madhukar Bharti <bh...@gmail.com>
>>> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> Once gone through code found that, While Producer starts it does three
>>>> things:
>>>>
>>>> 1. Sends Meta-data request
>>>> 2. Send message to broker(fetching broker list)
>>>> 3. If number of message to be produce is grater than 0 then again tries
>>> to
>>>> refresh metadata for outstanding produce requests.
>>>>
>>>> Each of the request takes configured timeout and go to next and finally
>>>> once all is done then it will throw Exception(if 3 also fails).
>>>>
>>>> Here the problem is, if we set timeout as 1 sec then to throw an
>>> exception
>>>> It takes 3 sec, so user request will be hanged up till 3 sec, that is
>>>> comparatively high for response time and if all threads will be blocked
>>> due
>>>> to producer send then whole application will be blocked for 3 sec. So we
>>>> want to reduce this time to 1 sec. in overall to throw Exception.
>>>>
>>>> What is the possible way to do this?
>>>>
>>>> Thanks
>>>> Madhukar
>>>>
>>>> On Thu, Apr 16, 2015 at 8:10 PM, Madhukar Bharti <
>>> bhartimadhukar@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> I came across a problem, If we use broker IP which is not reachable or
>>> out
>>>>> of network. Then it takes more than configured time(request.timeout.ms
>>> ).
>>>>> After checking the log got to know that it is trying to fetch topic
>>>>> meta-data three times by changing correlation id.
>>>>> Due to this even though i keep (request.timeout.ms=1000) It takes 3
>>> sec
>>>>> to throw Exception. I am using Kafka0.8.1.1 with patch
>>>>>
>>> https://issues.apache.org/jira/secure/attachment/12678547/kafka-1733-add-connectTimeoutMs.patch
>>>>>
>>>>>
>>>>> I have attached the log. Please check this and clarify why it is
>>> behaving
>>>>> like this. Whether it is by design or have to set some other property
>>> also.
>>>>>
>>>>>
>>>>>
>>>>> Regards
>>>>> Madhukar
>>>>>
>>>>>
>>>>>
>>>> ____________________________________________________________
>>>> Want to place your ad here?
>>>> Advertise on United Online
>>>>
>>> http://thirdpartyoffers.netzero.net/TGL3255/5539ed87d69846d871dafmp08duc
>>>
>>>
>>
>>
> ____________________________________________________________
> The WORST exercise for aging
> Avoid this "healthy" exercise to look & feel 5-10 years YOUNGER
> http://thirdpartyoffers.netzero.net/TGL3255/5540b94620e14394636c0mp13duc <http://thirdpartyoffers.netzero.net/TGL3255/5540b94620e14394636c0mp13duc>
Re: Why fetching meta-data for topic is done three times?
Posted by Madhukar Bharti <bh...@gmail.com>.
Hi Zakee,
>message.send.max.retries is 1
Regards,
Madhukar
On Tue, Apr 28, 2015 at 6:17 PM, Madhukar Bharti <bh...@gmail.com>
wrote:
> Hi Zakee,
>
> Thanks for your reply.
>
> >message.send.max.retries
> 3
>
> >retry.backoff.ms
> 100
>
> >topic.metadata.refresh.interval.ms
> 600*1000
>
> This is my properties.
>
> Regards,
> Madhukar
>
> On Tue, Apr 28, 2015 at 3:26 AM, Zakee <kz...@netzero.net> wrote:
>
>> What values do you have for below properties? Or are these set to
>> defaults?
>>
>> message.send.max.retries
>> retry.backoff.ms
>> topic.metadata.refresh.interval.ms
>>
>> Thanks
>> Zakee
>>
>>
>>
>> > On Apr 23, 2015, at 11:48 PM, Madhukar Bharti <bh...@gmail.com>
>> wrote:
>> >
>> > Hi All,
>> >
>> > Once gone through code found that, While Producer starts it does three
>> > things:
>> >
>> > 1. Sends Meta-data request
>> > 2. Send message to broker(fetching broker list)
>> > 3. If number of message to be produce is grater than 0 then again tries
>> to
>> > refresh metadata for outstanding produce requests.
>> >
>> > Each of the request takes configured timeout and go to next and finally
>> > once all is done then it will throw Exception(if 3 also fails).
>> >
>> > Here the problem is, if we set timeout as 1 sec then to throw an
>> exception
>> > It takes 3 sec, so user request will be hanged up till 3 sec, that is
>> > comparatively high for response time and if all threads will be blocked
>> due
>> > to producer send then whole application will be blocked for 3 sec. So we
>> > want to reduce this time to 1 sec. in overall to throw Exception.
>> >
>> > What is the possible way to do this?
>> >
>> > Thanks
>> > Madhukar
>> >
>> > On Thu, Apr 16, 2015 at 8:10 PM, Madhukar Bharti <
>> bhartimadhukar@gmail.com>
>> > wrote:
>> >
>> >> Hi All,
>> >>
>> >> I came across a problem, If we use broker IP which is not reachable or
>> out
>> >> of network. Then it takes more than configured time(request.timeout.ms
>> ).
>> >> After checking the log got to know that it is trying to fetch topic
>> >> meta-data three times by changing correlation id.
>> >> Due to this even though i keep (request.timeout.ms=1000) It takes 3
>> sec
>> >> to throw Exception. I am using Kafka0.8.1.1 with patch
>> >>
>> https://issues.apache.org/jira/secure/attachment/12678547/kafka-1733-add-connectTimeoutMs.patch
>> >>
>> >>
>> >> I have attached the log. Please check this and clarify why it is
>> behaving
>> >> like this. Whether it is by design or have to set some other property
>> also.
>> >>
>> >>
>> >>
>> >> Regards
>> >> Madhukar
>> >>
>> >>
>> >>
>> > ____________________________________________________________
>> > Want to place your ad here?
>> > Advertise on United Online
>> >
>> http://thirdpartyoffers.netzero.net/TGL3255/5539ed87d69846d871dafmp08duc
>>
>>
>
>
Re: Why fetching meta-data for topic is done three times?
Posted by Madhukar Bharti <bh...@gmail.com>.
Hi Zakee,
Thanks for your reply.
>message.send.max.retries
3
>retry.backoff.ms
100
>topic.metadata.refresh.interval.ms
600*1000
This is my properties.
Regards,
Madhukar
On Tue, Apr 28, 2015 at 3:26 AM, Zakee <kz...@netzero.net> wrote:
> What values do you have for below properties? Or are these set to defaults?
>
> message.send.max.retries
> retry.backoff.ms
> topic.metadata.refresh.interval.ms
>
> Thanks
> Zakee
>
>
>
> > On Apr 23, 2015, at 11:48 PM, Madhukar Bharti <bh...@gmail.com>
> wrote:
> >
> > Hi All,
> >
> > Once gone through code found that, While Producer starts it does three
> > things:
> >
> > 1. Sends Meta-data request
> > 2. Send message to broker(fetching broker list)
> > 3. If number of message to be produce is grater than 0 then again tries
> to
> > refresh metadata for outstanding produce requests.
> >
> > Each of the request takes configured timeout and go to next and finally
> > once all is done then it will throw Exception(if 3 also fails).
> >
> > Here the problem is, if we set timeout as 1 sec then to throw an
> exception
> > It takes 3 sec, so user request will be hanged up till 3 sec, that is
> > comparatively high for response time and if all threads will be blocked
> due
> > to producer send then whole application will be blocked for 3 sec. So we
> > want to reduce this time to 1 sec. in overall to throw Exception.
> >
> > What is the possible way to do this?
> >
> > Thanks
> > Madhukar
> >
> > On Thu, Apr 16, 2015 at 8:10 PM, Madhukar Bharti <
> bhartimadhukar@gmail.com>
> > wrote:
> >
> >> Hi All,
> >>
> >> I came across a problem, If we use broker IP which is not reachable or
> out
> >> of network. Then it takes more than configured time(request.timeout.ms
> ).
> >> After checking the log got to know that it is trying to fetch topic
> >> meta-data three times by changing correlation id.
> >> Due to this even though i keep (request.timeout.ms=1000) It takes 3 sec
> >> to throw Exception. I am using Kafka0.8.1.1 with patch
> >>
> https://issues.apache.org/jira/secure/attachment/12678547/kafka-1733-add-connectTimeoutMs.patch
> >>
> >>
> >> I have attached the log. Please check this and clarify why it is
> behaving
> >> like this. Whether it is by design or have to set some other property
> also.
> >>
> >>
> >>
> >> Regards
> >> Madhukar
> >>
> >>
> >>
> > ____________________________________________________________
> > Want to place your ad here?
> > Advertise on United Online
> > http://thirdpartyoffers.netzero.net/TGL3255/5539ed87d69846d871dafmp08duc
>
>
Re: Why fetching meta-data for topic is done three times?
Posted by Zakee <kz...@netzero.net>.
What values do you have for below properties? Or are these set to defaults?
message.send.max.retries
retry.backoff.ms
topic.metadata.refresh.interval.ms
Thanks
Zakee
> On Apr 23, 2015, at 11:48 PM, Madhukar Bharti <bh...@gmail.com> wrote:
>
> Hi All,
>
> Once gone through code found that, While Producer starts it does three
> things:
>
> 1. Sends Meta-data request
> 2. Send message to broker(fetching broker list)
> 3. If number of message to be produce is grater than 0 then again tries to
> refresh metadata for outstanding produce requests.
>
> Each of the request takes configured timeout and go to next and finally
> once all is done then it will throw Exception(if 3 also fails).
>
> Here the problem is, if we set timeout as 1 sec then to throw an exception
> It takes 3 sec, so user request will be hanged up till 3 sec, that is
> comparatively high for response time and if all threads will be blocked due
> to producer send then whole application will be blocked for 3 sec. So we
> want to reduce this time to 1 sec. in overall to throw Exception.
>
> What is the possible way to do this?
>
> Thanks
> Madhukar
>
> On Thu, Apr 16, 2015 at 8:10 PM, Madhukar Bharti <bh...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> I came across a problem, If we use broker IP which is not reachable or out
>> of network. Then it takes more than configured time(request.timeout.ms).
>> After checking the log got to know that it is trying to fetch topic
>> meta-data three times by changing correlation id.
>> Due to this even though i keep (request.timeout.ms=1000) It takes 3 sec
>> to throw Exception. I am using Kafka0.8.1.1 with patch
>> https://issues.apache.org/jira/secure/attachment/12678547/kafka-1733-add-connectTimeoutMs.patch
>>
>>
>> I have attached the log. Please check this and clarify why it is behaving
>> like this. Whether it is by design or have to set some other property also.
>>
>>
>>
>> Regards
>> Madhukar
>>
>>
>>
> ____________________________________________________________
> Want to place your ad here?
> Advertise on United Online
> http://thirdpartyoffers.netzero.net/TGL3255/5539ed87d69846d871dafmp08duc
Re: Why fetching meta-data for topic is done three times?
Posted by Madhukar Bharti <bh...@gmail.com>.
Hi All,
Once gone through code found that, While Producer starts it does three
things:
1. Sends Meta-data request
2. Send message to broker(fetching broker list)
3. If number of message to be produce is grater than 0 then again tries to
refresh metadata for outstanding produce requests.
Each of the request takes configured timeout and go to next and finally
once all is done then it will throw Exception(if 3 also fails).
Here the problem is, if we set timeout as 1 sec then to throw an exception
It takes 3 sec, so user request will be hanged up till 3 sec, that is
comparatively high for response time and if all threads will be blocked due
to producer send then whole application will be blocked for 3 sec. So we
want to reduce this time to 1 sec. in overall to throw Exception.
What is the possible way to do this?
Thanks
Madhukar
On Thu, Apr 16, 2015 at 8:10 PM, Madhukar Bharti <bh...@gmail.com>
wrote:
> Hi All,
>
> I came across a problem, If we use broker IP which is not reachable or out
> of network. Then it takes more than configured time(request.timeout.ms).
> After checking the log got to know that it is trying to fetch topic
> meta-data three times by changing correlation id.
> Due to this even though i keep (request.timeout.ms=1000) It takes 3 sec
> to throw Exception. I am using Kafka0.8.1.1 with patch
> https://issues.apache.org/jira/secure/attachment/12678547/kafka-1733-add-connectTimeoutMs.patch
>
>
> I have attached the log. Please check this and clarify why it is behaving
> like this. Whether it is by design or have to set some other property also.
>
>
>
> Regards
> Madhukar
>
>
>