You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "Ghosh, Achintya (Contractor)" <Ac...@comcast.com> on 2016/08/26 13:32:03 UTC
Batch Expired
Hi there,
What is the recommended Producer setting for Producer as I see a lot of Batch Expired exception even though I put request.timeout=60000.
Producer settings:
acks=1
retries=3
batch.size=16384
linger.ms=5
buffer.memory=33554432
request.timeout.ms=60000
timeout.ms=60000
Thanks
Achintya
Re: Batch Expired
Posted by Mayuresh Gharat <gh...@gmail.com>.
Hi,
RequestTimeout is used for 2 cases :
1) Timing out the batches sitting in the accumulator.
2) Requests that are already sent over the wire and you have not yet heard
from the server.
In a case where there is a network partition, the client might not detect
it, till the actual TCP timeout that is I think around 30 minutes or more.
The requestTimeout setting kicks in before the TCP timeout and then the
request is re tried with fresh metadata or errored out depending upon the
retry setting.
Thanks,
Mayuresh
On Mon, Aug 29, 2016 at 9:32 AM, Ghosh, Achintya (Contractor) <
Achintya_Ghosh@comcast.com> wrote:
> Hi Krishna,
> Thank you for your response.
>
> Connections already made but if we increase the request timeout 5 times
> let's say request.timeout.ms= 5*60000 , then the number of 'Batch
> Expired ' exception is less, so what is the recommended value for '
> request.timeout.ms '.
> If we increase more, is there any impact?
>
> Thanks
> Achintya
>
> -----Original Message-----
> From: R Krishna [mailto:krishna81m@gmail.com]
> Sent: Friday, August 26, 2016 6:17 PM
> To: users@kafka.apache.org
> Cc: dev@kafka.apache.org
> Subject: Re: Batch Expired
>
> Are any requests at all making it? That is a pretty big timeout.
>
> However, I noticed if there is no connections made to broker, you can
> still get batch expiry.
>
>
> On Fri, Aug 26, 2016 at 6:32 AM, Ghosh, Achintya (Contractor) <
> Achintya_Ghosh@comcast.com> wrote:
>
> > Hi there,
> >
> > What is the recommended Producer setting for Producer as I see a lot
> > of Batch Expired exception even though I put request.timeout=60000.
> >
> > Producer settings:
> > acks=1
> > retries=3
> > batch.size=16384
> > linger.ms=5
> > buffer.memory=33554432
> > request.timeout.ms=60000
> > timeout.ms=60000
> >
> > Thanks
> > Achintya
> >
>
>
>
> --
> Radha Krishna, Proddaturi
> 253-234-5657
>
--
-Regards,
Mayuresh R. Gharat
(862) 250-7125
Re: Batch Expired
Posted by Mayuresh Gharat <gh...@gmail.com>.
Hi,
RequestTimeout is used for 2 cases :
1) Timing out the batches sitting in the accumulator.
2) Requests that are already sent over the wire and you have not yet heard
from the server.
In a case where there is a network partition, the client might not detect
it, till the actual TCP timeout that is I think around 30 minutes or more.
The requestTimeout setting kicks in before the TCP timeout and then the
request is re tried with fresh metadata or errored out depending upon the
retry setting.
Thanks,
Mayuresh
On Mon, Aug 29, 2016 at 9:32 AM, Ghosh, Achintya (Contractor) <
Achintya_Ghosh@comcast.com> wrote:
> Hi Krishna,
> Thank you for your response.
>
> Connections already made but if we increase the request timeout 5 times
> let's say request.timeout.ms= 5*60000 , then the number of 'Batch
> Expired ' exception is less, so what is the recommended value for '
> request.timeout.ms '.
> If we increase more, is there any impact?
>
> Thanks
> Achintya
>
> -----Original Message-----
> From: R Krishna [mailto:krishna81m@gmail.com]
> Sent: Friday, August 26, 2016 6:17 PM
> To: users@kafka.apache.org
> Cc: dev@kafka.apache.org
> Subject: Re: Batch Expired
>
> Are any requests at all making it? That is a pretty big timeout.
>
> However, I noticed if there is no connections made to broker, you can
> still get batch expiry.
>
>
> On Fri, Aug 26, 2016 at 6:32 AM, Ghosh, Achintya (Contractor) <
> Achintya_Ghosh@comcast.com> wrote:
>
> > Hi there,
> >
> > What is the recommended Producer setting for Producer as I see a lot
> > of Batch Expired exception even though I put request.timeout=60000.
> >
> > Producer settings:
> > acks=1
> > retries=3
> > batch.size=16384
> > linger.ms=5
> > buffer.memory=33554432
> > request.timeout.ms=60000
> > timeout.ms=60000
> >
> > Thanks
> > Achintya
> >
>
>
>
> --
> Radha Krishna, Proddaturi
> 253-234-5657
>
--
-Regards,
Mayuresh R. Gharat
(862) 250-7125
RE: Batch Expired
Posted by "Ghosh, Achintya (Contractor)" <Ac...@comcast.com>.
Hi Krishna,
Thank you for your response.
Connections already made but if we increase the request timeout 5 times let's say request.timeout.ms= 5*60000 , then the number of 'Batch Expired ' exception is less, so what is the recommended value for ' request.timeout.ms '.
If we increase more, is there any impact?
Thanks
Achintya
-----Original Message-----
From: R Krishna [mailto:krishna81m@gmail.com]
Sent: Friday, August 26, 2016 6:17 PM
To: users@kafka.apache.org
Cc: dev@kafka.apache.org
Subject: Re: Batch Expired
Are any requests at all making it? That is a pretty big timeout.
However, I noticed if there is no connections made to broker, you can still get batch expiry.
On Fri, Aug 26, 2016 at 6:32 AM, Ghosh, Achintya (Contractor) < Achintya_Ghosh@comcast.com> wrote:
> Hi there,
>
> What is the recommended Producer setting for Producer as I see a lot
> of Batch Expired exception even though I put request.timeout=60000.
>
> Producer settings:
> acks=1
> retries=3
> batch.size=16384
> linger.ms=5
> buffer.memory=33554432
> request.timeout.ms=60000
> timeout.ms=60000
>
> Thanks
> Achintya
>
--
Radha Krishna, Proddaturi
253-234-5657
RE: Batch Expired
Posted by "Ghosh, Achintya (Contractor)" <Ac...@comcast.com>.
Hi Krishna,
Thank you for your response.
Connections already made but if we increase the request timeout 5 times let's say request.timeout.ms= 5*60000 , then the number of 'Batch Expired ' exception is less, so what is the recommended value for ' request.timeout.ms '.
If we increase more, is there any impact?
Thanks
Achintya
-----Original Message-----
From: R Krishna [mailto:krishna81m@gmail.com]
Sent: Friday, August 26, 2016 6:17 PM
To: users@kafka.apache.org
Cc: dev@kafka.apache.org
Subject: Re: Batch Expired
Are any requests at all making it? That is a pretty big timeout.
However, I noticed if there is no connections made to broker, you can still get batch expiry.
On Fri, Aug 26, 2016 at 6:32 AM, Ghosh, Achintya (Contractor) < Achintya_Ghosh@comcast.com> wrote:
> Hi there,
>
> What is the recommended Producer setting for Producer as I see a lot
> of Batch Expired exception even though I put request.timeout=60000.
>
> Producer settings:
> acks=1
> retries=3
> batch.size=16384
> linger.ms=5
> buffer.memory=33554432
> request.timeout.ms=60000
> timeout.ms=60000
>
> Thanks
> Achintya
>
--
Radha Krishna, Proddaturi
253-234-5657
Re: Batch Expired
Posted by R Krishna <kr...@gmail.com>.
Are any requests at all making it? That is a pretty big timeout.
However, I noticed if there is no connections made to broker, you can still
get batch expiry.
On Fri, Aug 26, 2016 at 6:32 AM, Ghosh, Achintya (Contractor) <
Achintya_Ghosh@comcast.com> wrote:
> Hi there,
>
> What is the recommended Producer setting for Producer as I see a lot of
> Batch Expired exception even though I put request.timeout=60000.
>
> Producer settings:
> acks=1
> retries=3
> batch.size=16384
> linger.ms=5
> buffer.memory=33554432
> request.timeout.ms=60000
> timeout.ms=60000
>
> Thanks
> Achintya
>
--
Radha Krishna, Proddaturi
253-234-5657
Re: Batch Expired
Posted by R Krishna <kr...@gmail.com>.
Are any requests at all making it? That is a pretty big timeout.
However, I noticed if there is no connections made to broker, you can still
get batch expiry.
On Fri, Aug 26, 2016 at 6:32 AM, Ghosh, Achintya (Contractor) <
Achintya_Ghosh@comcast.com> wrote:
> Hi there,
>
> What is the recommended Producer setting for Producer as I see a lot of
> Batch Expired exception even though I put request.timeout=60000.
>
> Producer settings:
> acks=1
> retries=3
> batch.size=16384
> linger.ms=5
> buffer.memory=33554432
> request.timeout.ms=60000
> timeout.ms=60000
>
> Thanks
> Achintya
>
--
Radha Krishna, Proddaturi
253-234-5657