[jira] [Comment Edited] (KAFKA-8755) Stand-by Task of an Optimized Source Table Does Not Write Anything to its State Store

["Chris Pettitt (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16912642#comment-16912642 ] 

Chris Pettitt edited comment on KAFKA-8755 at 8/21/19 8:00 PM:
---------------------------------------------------------------

BUG 2: not a bug. It was updating at the commit interval, no faster. But I now have it only doing an update if the task is blocked on a newer consumer commit, which reduces ~100 `consumer.committed` calls down to 1 during failover. This improves both the optimized and the non-optimized cases.


was (Author: cpettitt-confluent):
BUG 2: not a bug. It was updating at the commit interval, no faster. But I now have it only doing an update if the task is blocked on a newer consumer commit, which reduces ~100 `consumer.committed` calls down to 1 during failover.

> Stand-by Task of an Optimized Source Table Does Not Write Anything to its State Store
> -------------------------------------------------------------------------------------
>
>                 Key: KAFKA-8755
>                 URL: https://issues.apache.org/jira/browse/KAFKA-8755
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 2.4.0
>            Reporter: Bruno Cadonna
>            Assignee: Chris Pettitt
>            Priority: Major
>              Labels: newbie
>         Attachments: StandbyTaskTest.java
>
>
> With the following topology:
> {code:java}
> builder.table(
>     INPUT_TOPIC, 
>     Consumed.with(Serdes.Integer(), Serdes.Integer()), 
>     Materialized.<Integer, Integer, KeyValueStore<Bytes, byte[]>>as(stateName)
> )
> {code}
> and with topology optimization turned on, Kafka Streams uses the input topic {{INPUT_TOPIC}} as the change log topic for state store {{stateName}}. A stand-by task for such a topology should read from {{INPUT_TOPIC}} and should write the records to its state store so that the streams client that runs the stand-by task can take over the execution of the topology in case of a failure with an up-to-date replica of the state.
> Currently, the stand-by task described above reads from the input topic but does not write the records to its state store. Thus, after a failure the stand-by task cannot provide any up-to-date state store and the streams client needs to construct the state from scratch before it can take over the execution.
> The described behaviour can be reproduced with the attached test.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)