Streams - retry configuration publishing changelog updates?

[dan bress <da...@gmail.com>]

I'm seeing this timeout in my Kafka Streams application logs:

Caused by: org.apache.kafka.common.errors.TimeoutException: Expiring 7
record(s) for my-table-changelog-18: 30011 ms has passed since last append

What kind of retries to I get by default publishing to changelogs?  I have
these producer configs set:

2018-01-23 21:14:31,197 INF [main] ProducerConfig ProducerConfig values:
acks = all
retries = 30
retry.backoff.ms = 1000
request.timeout.ms = 30000

Does this mean I am getting 30 retries when trying to publish to
changelogs?  Or do I need to set another config(StreamsConfig) to get more
retries for changelog publishing?

Thanks!
Dan

Re: Streams - retry configuration publishing changelog updates?

[Guozhang Wang <wa...@gmail.com>]

Hello Dan,

It seems you are hitting a known issue that KIP-91 is trying to fix (it is
a general issue of producer itself):
https://cwiki.apache.org/confluence/display/KAFKA/KIP-91+Provide+Intuitive+User+Timeouts+in+The+Producer

It means that the records has never been sent out before expiring since the
metadata for finding the leader of "my-table-changelog-18" is never known,
it is likely due to your broker is not available and hence metadata refresh
never succeed. Hence, retries config does not help here since it is never
sent actually.

Before KIP-91 gets merged in, you will have to increase your
request.timeout.ms (btw it is already as high as 30 secs, do you have a
broker offline for about that long period?) to larger value so that the
producer can tolerate longer time that metadata is not available for
sending.

Guozhang


On Tue, Jan 23, 2018 at 5:27 PM, dan bress <da...@gmail.com> wrote:

> I'm seeing this timeout in my Kafka Streams application logs:
>
> Caused by: org.apache.kafka.common.errors.TimeoutException: Expiring 7
> record(s) for my-table-changelog-18: 30011 ms has passed since last append
>
> What kind of retries to I get by default publishing to changelogs?  I have
> these producer configs set:
>
> 2018-01-23 21:14:31,197 INF [main] ProducerConfig ProducerConfig values:
> acks = all
> retries = 30
> retry.backoff.ms = 1000
> request.timeout.ms = 30000
>
> Does this mean I am getting 30 retries when trying to publish to
> changelogs?  Or do I need to set another config(StreamsConfig) to get more
> retries for changelog publishing?
>
> Thanks!
> Dan
>



-- 
-- Guozhang