[DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

[Navinder Brar <na...@yahoo.com.INVALID>]

Hi,
Starting a discussion on the KIP to Allow state stores to serve stale reads during rebalance(https://cwiki.apache.org/confluence/display/KAFKA/KIP-535%3A+Allow+state+stores+to+serve+stale+reads+during+rebalance).
Thanks & Regards,Navinder 
LinkedIn

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

[Vinoth Chandar <vc...@confluent.io>]

Hi Navinder,

Thanks for sharing the KIP! Few thoughts

- Can we link the JIRA, discussion thread also to the KIP
- Based on the discussion on KAFKA-6144, I was under the impression that
this KIP is also going to cover exposing of the standby information in
StreamsMetadata and thus subsume KAFKA-8994 . That would require a public
API change?
- KIP seems to be focussing on restoration when a new node is added.
KIP-441 is underway and has some major changes proposed for this. It would
be good to clarify dependencies if any. Without KIP-441, I am not very sure
if we should allow reads from nodes in RESTORING state, which could amount
to many minutes/few hours of stale reads?  This is different from allowing
querying standby replicas, which could be mostly caught up and the
staleness window could be much smaller/tolerable. (once again the focus on
KAFKA-8994)
- Finally, we may need to introduce a configuration to control this. Some
users may prefer errors to stale data. Can we also add it to the KIP?

Thanks
Vinoth




On Sun, Oct 13, 2019 at 3:31 PM Navinder Brar
<na...@yahoo.com.invalid> wrote:

> Hi,
> Starting a discussion on the KIP to Allow state stores to serve stale
> reads during rebalance(
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-535%3A+Allow+state+stores+to+serve+stale+reads+during+rebalance
> ).
> Thanks & Regards,Navinder
> LinkedIn
>