You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by Patrick Stuedi <ps...@confluent.io.INVALID> on 2021/11/03 20:43:54 UTC

[DISCUSS] KIP-791: Add Record Metadata to State Store Context

Hi everyone,

I would like to start the discussion for KIP-791: Add Record Metadata to
State Store Context.

The KIP can be found here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-791:+Add+Record+Metadata+to+State+Store+Context

Any feedback will be highly appreciated.

Many thanks,
 Patrick

Re: [DISCUSS] KIP-791: Add Record Metadata to State Store Context

Posted by John Roesler <vv...@apache.org>.

Thanks for the KIP, Patrick!

It looks like you addressed Guozhang's and Bruno's very good
feeback, and I like the result. The example especially helps
clarify how this property might be useful.

I'm in favor of this proposal.

Thanks,
-John

On Fri, 2021-11-05 at 12:03 +0100, Bruno Cadonna wrote:
> Hi Patrick,
> 
> Thank you for the KIP!
> 
> - Maybe some more details in the motivation would help to better 
> understand the background of the KIP. Currently, it is hard to judge 
> whether record metadata should be exposed or not. Can you maybe give an 
> example?
> 
> - Could you please replace RYW abbreviation with read-your-writes (at 
> least that is my guess about the meaning of RYW)?
> 
> Best,
> Bruno
> 
> 
> 
> On 03.11.21 22:43, Guozhang Wang wrote:
> > Thanks Patrick,
> > 
> > I looked at the KIP and it looks good to me overall. I think we need to
> > double check whether the record metadata reflect the "last processed
> > record" or the "currently processed record" where the latter may not have
> > been completely processed. In `ProcessorContext#recordMetadata` it returns
> > the latter, but that may not be the preferred case if you want to build the
> > consistency reasoning on top of.
> > 
> > Otherwise, LGTM.
> > 
> > 
> > Guozhang
> > 
> > On Wed, Nov 3, 2021 at 1:44 PM Patrick Stuedi <ps...@confluent.io.invalid>
> > wrote:
> > 
> > > Hi everyone,
> > > 
> > > I would like to start the discussion for KIP-791: Add Record Metadata to
> > > State Store Context.
> > > 
> > > The KIP can be found here:
> > > 
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-791:+Add+Record+Metadata+to+State+Store+Context
> > > 
> > > Any feedback will be highly appreciated.
> > > 
> > > Many thanks,
> > >   Patrick
> > > 
> > 
> >

Re: [DISCUSS] KIP-791: Add Record Metadata to State Store Context

Posted by Bruno Cadonna <ca...@apache.org>.

Hi Patrick,

Thank you for the KIP!

- Maybe some more details in the motivation would help to better 
understand the background of the KIP. Currently, it is hard to judge 
whether record metadata should be exposed or not. Can you maybe give an 
example?

- Could you please replace RYW abbreviation with read-your-writes (at 
least that is my guess about the meaning of RYW)?

Best,
Bruno



On 03.11.21 22:43, Guozhang Wang wrote:
> Thanks Patrick,
> 
> I looked at the KIP and it looks good to me overall. I think we need to
> double check whether the record metadata reflect the "last processed
> record" or the "currently processed record" where the latter may not have
> been completely processed. In `ProcessorContext#recordMetadata` it returns
> the latter, but that may not be the preferred case if you want to build the
> consistency reasoning on top of.
> 
> Otherwise, LGTM.
> 
> 
> Guozhang
> 
> On Wed, Nov 3, 2021 at 1:44 PM Patrick Stuedi <ps...@confluent.io.invalid>
> wrote:
> 
>> Hi everyone,
>>
>> I would like to start the discussion for KIP-791: Add Record Metadata to
>> State Store Context.
>>
>> The KIP can be found here:
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-791:+Add+Record+Metadata+to+State+Store+Context
>>
>> Any feedback will be highly appreciated.
>>
>> Many thanks,
>>   Patrick
>>
> 
>

Re: [DISCUSS] KIP-791: Add Record Metadata to State Store Context

Posted by Guozhang Wang <wa...@gmail.com>.

Thanks Patrick,

I looked at the KIP and it looks good to me overall. I think we need to
double check whether the record metadata reflect the "last processed
record" or the "currently processed record" where the latter may not have
been completely processed. In `ProcessorContext#recordMetadata` it returns
the latter, but that may not be the preferred case if you want to build the
consistency reasoning on top of.

Otherwise, LGTM.

Guozhang

On Wed, Nov 3, 2021 at 1:44 PM Patrick Stuedi <ps...@confluent.io.invalid>
wrote:

> Hi everyone,
>
> I would like to start the discussion for KIP-791: Add Record Metadata to
> State Store Context.
>
> The KIP can be found here:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-791:+Add+Record+Metadata+to+State+Store+Context
>
> Any feedback will be highly appreciated.
>
> Many thanks,
>  Patrick
>

-- 
-- Guozhang