You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Saladi Naidu <na...@yahoo.com.INVALID> on 2017/08/10 18:57:36 UTC
Kafka Producer Errors
We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part of the upgrade our clients upgraded the client libraries as well. After the upgrade we are facing following issue
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for OM-LOCAL-DC3-REQUISITION-SUBMISSION-PRD-E2-14: 30027 ms has passed since last append
My understanding is that Producer batches the send requests going to topic/partition per leader and sends it once based on following parameters
ülinger.ms ß
is time based batchingübatch.size ß
is size based batching
We had batch.size as 16K and linger.ms 0 and we expected producer to send requests right away even if the batch size is not met because linger.ms is zero.
Our request.timeout.ms 30 seconds, looking at the error, it looks like requests are kept in the sender queue longer and when the Sender polling thread sees the requests are older, it is expiring them out. I have tried following multiple combinations and none seem to work
linger.ms 0 and batch.size - < 16 klinger.ms 0 and batch.size - 0 k
linger.ms 1000 and batch.size - < 0
Any clue on why linger.ms 0 or btach.size 0 is not taken into consideration ???
Thanks,Naidu Saladi
Re: Kafka Producer Errors
Posted by Saladi Naidu <na...@yahoo.com.INVALID>.
I thought the same and checked, GC pause was maximum of 8 seconds Naidu Saladi
On Monday, August 14, 2017 1:59 AM, Kamal C <ka...@gmail.com> wrote:
I think your application (where the producer resides) is facing GC issues.
The time taken for the GC might be higher than the `request.timeout.ms`.
Check your `jvm.log` and update the `request.timeout.ms`. The same property
is applicable to producer, consumer and broker. Increase the config only
for the KafkaProducer.
-- Kamal
On Sat, Aug 12, 2017 at 12:19 AM, Saladi Naidu <
naidusp2002@yahoo.com.invalid> wrote:
> Looks like some of the fonts/characters did not come through, am
> re-sending with correct details
> We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part
> of the upgrade our clients upgraded the client libraries as well. After the
> upgrade we are facing following issue
> org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for
> OM-LOCAL-DC3-REQUISITION-SUBMISSION-PRD-E2-14: 30027 ms has passed since
> last append
>
>
> My understanding is that Producer batches the send requests going to
> topic/partition per leader and sends it once based on following parameters
> linger.ms is time based batching
> batch.size is size based batching
>
> We had batch.size as 16K (default) and linger.ms 0 - We expected producer
> to send requests right away even if the batch size is not met because
> linger.ms is 0.
>
> Our request.timeout.ms 30 seconds, looking at the error, it looks like
> requests are kept in the sender queue longer and when the Sender polling
> thread sees the requests are older, it is expiring them out.
> I have tried following multiple combinations and none seem to work
>
> linger.ms 0 and batch.size - < 16 klinger.ms 0 and batch.size - 0 k
> linger.ms 1000 and batch.size - < 0
>
> Any clue on why linger.ms 0 or btach.size 0 is not taken into
> consideration ??? Naidu Saladi
>
>
> On Thursday, August 10, 2017 1:57 PM, Saladi Naidu
> <na...@yahoo.com.INVALID> wrote:
>
>
> We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part
> of the upgrade our clients upgraded the client libraries as well. After the
> upgrade we are facing following issue
> org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for
> OM-LOCAL-DC3-REQUISITION-SUBMISSION-PRD-E2-14: 30027 ms has passed since
> last append
>
>
> My understanding is that Producer batches the send requests going to
> topic/partition per leader and sends it once based on following parameters
> ülinger.ms <http://xn--linger-2ya.ms> ß
>
> is time based batchingübatch.size ß
>
> is size based batching
> We had batch.size as 16K and linger.ms 0 and we expected producer to send
> requests right away even if the batch size is not met because linger.ms
> is zero.
> Our request.timeout.ms 30 seconds, looking at the error, it looks like
> requests are kept in the sender queue longer and when the Sender polling
> thread sees the requests are older, it is expiring them out. I have tried
> following multiple combinations and none seem to work
>
> linger.ms 0 and batch.size - < 16 klinger.ms 0 and batch.size - 0 k
> linger.ms 1000 and batch.size - < 0
>
> Any clue on why linger.ms 0 or btach.size 0 is not taken into
> consideration ???
> Thanks,Naidu Saladi
>
>
>
Re: Kafka Producer Errors
Posted by Kamal C <ka...@gmail.com>.
I think your application (where the producer resides) is facing GC issues.
The time taken for the GC might be higher than the `request.timeout.ms`.
Check your `jvm.log` and update the `request.timeout.ms`. The same property
is applicable to producer, consumer and broker. Increase the config only
for the KafkaProducer.
-- Kamal
On Sat, Aug 12, 2017 at 12:19 AM, Saladi Naidu <
naidusp2002@yahoo.com.invalid> wrote:
> Looks like some of the fonts/characters did not come through, am
> re-sending with correct details
> We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part
> of the upgrade our clients upgraded the client libraries as well. After the
> upgrade we are facing following issue
> org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for
> OM-LOCAL-DC3-REQUISITION-SUBMISSION-PRD-E2-14: 30027 ms has passed since
> last append
>
>
> My understanding is that Producer batches the send requests going to
> topic/partition per leader and sends it once based on following parameters
> linger.ms is time based batching
> batch.size is size based batching
>
> We had batch.size as 16K (default) and linger.ms 0 - We expected producer
> to send requests right away even if the batch size is not met because
> linger.ms is 0.
>
> Our request.timeout.ms 30 seconds, looking at the error, it looks like
> requests are kept in the sender queue longer and when the Sender polling
> thread sees the requests are older, it is expiring them out.
> I have tried following multiple combinations and none seem to work
>
> linger.ms 0 and batch.size - < 16 klinger.ms 0 and batch.size - 0 k
> linger.ms 1000 and batch.size - < 0
>
> Any clue on why linger.ms 0 or btach.size 0 is not taken into
> consideration ??? Naidu Saladi
>
>
> On Thursday, August 10, 2017 1:57 PM, Saladi Naidu
> <na...@yahoo.com.INVALID> wrote:
>
>
> We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part
> of the upgrade our clients upgraded the client libraries as well. After the
> upgrade we are facing following issue
> org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for
> OM-LOCAL-DC3-REQUISITION-SUBMISSION-PRD-E2-14: 30027 ms has passed since
> last append
>
>
> My understanding is that Producer batches the send requests going to
> topic/partition per leader and sends it once based on following parameters
> ülinger.ms <http://xn--linger-2ya.ms> ß
>
> is time based batchingübatch.size ß
>
> is size based batching
> We had batch.size as 16K and linger.ms 0 and we expected producer to send
> requests right away even if the batch size is not met because linger.ms
> is zero.
> Our request.timeout.ms 30 seconds, looking at the error, it looks like
> requests are kept in the sender queue longer and when the Sender polling
> thread sees the requests are older, it is expiring them out. I have tried
> following multiple combinations and none seem to work
>
> linger.ms 0 and batch.size - < 16 klinger.ms 0 and batch.size - 0 k
> linger.ms 1000 and batch.size - < 0
>
> Any clue on why linger.ms 0 or btach.size 0 is not taken into
> consideration ???
> Thanks,Naidu Saladi
>
>
>
Re: Kafka Producer Errors
Posted by Saladi Naidu <na...@yahoo.com.INVALID>.
Looks like some of the fonts/characters did not come through, am re-sending with correct details
We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part of the upgrade our clients upgraded the client libraries as well. After the upgrade we are facing following issue
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for OM-LOCAL-DC3-REQUISITION-SUBMISSION-PRD-E2-14: 30027 ms has passed since last append
My understanding is that Producer batches the send requests going to topic/partition per leader and sends it once based on following parameters
linger.ms is time based batching
batch.size is size based batching
We had batch.size as 16K (default) and linger.ms 0 - We expected producer to send requests right away even if the batch size is not met because linger.ms is 0.
Our request.timeout.ms 30 seconds, looking at the error, it looks like requests are kept in the sender queue longer and when the Sender polling thread sees the requests are older, it is expiring them out.
I have tried following multiple combinations and none seem to work
linger.ms 0 and batch.size - < 16 klinger.ms 0 and batch.size - 0 k
linger.ms 1000 and batch.size - < 0
Any clue on why linger.ms 0 or btach.size 0 is not taken into consideration ??? Naidu Saladi
On Thursday, August 10, 2017 1:57 PM, Saladi Naidu <na...@yahoo.com.INVALID> wrote:
We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part of the upgrade our clients upgraded the client libraries as well. After the upgrade we are facing following issue
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for OM-LOCAL-DC3-REQUISITION-SUBMISSION-PRD-E2-14: 30027 ms has passed since last append
My understanding is that Producer batches the send requests going to topic/partition per leader and sends it once based on following parameters
ülinger.ms ß
is time based batchingübatch.size ß
is size based batching
We had batch.size as 16K and linger.ms 0 and we expected producer to send requests right away even if the batch size is not met because linger.ms is zero.
Our request.timeout.ms 30 seconds, looking at the error, it looks like requests are kept in the sender queue longer and when the Sender polling thread sees the requests are older, it is expiring them out. I have tried following multiple combinations and none seem to work
linger.ms 0 and batch.size - < 16 klinger.ms 0 and batch.size - 0 k
linger.ms 1000 and batch.size - < 0
Any clue on why linger.ms 0 or btach.size 0 is not taken into consideration ???
Thanks,Naidu Saladi