You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by aishwarya kumar <as...@gmail.com> on 2019/10/01 14:48:42 UTC
Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL
Thank you all for the feedback, I will keep this thread open for discussion
for a couple of more days and then start with the voting process.
Best regards,
Aishwarya
On Fri, Sep 27, 2019, 12:37 PM John Roesler <jo...@confluent.io> wrote:
> Looks good to me! I have no further comments.
>
> Thanks again for the KIP, Aishwarya!
> -John
>
> On Fri, Sep 27, 2019 at 10:11 AM aishwarya kumar <as...@gmail.com>
> wrote:
> >
> > Hello John,
> >
> > Thank you for pointing this out to me, to maintain consistency across
> API's
> > it does make sense to allow users to define custom names for
> > their processors.
> >
> > I've made the change in the KIP:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL
> >
> > Best,
> > Aishwarya
> >
> > On Tue, Sep 24, 2019 at 11:54 AM John Roesler <jo...@confluent.io> wrote:
> >
> > > Hey Aishwarya,
> > >
> > > Thanks for the KIP! It looks good to me, although in a post-KIP-307
> > > world, we also need a "Named" parameter (to give the processor node a
> > > name, as opposed to the store itself).
> > >
> > > This would result in a total of four overloads:
> > > 1. no args
> > > 2. Named
> > > 3. Materialized
> > > 4. Materialized, Named
> > >
> > > I'd like to propose a re-design of the DSL in the future to clean this
> > > up, but for now, this is the pattern we have to follow.
> > >
> > > Thoughts?
> > >
> > > Thanks,
> > > -John
> > >
> > > On Tue, Sep 24, 2019 at 9:54 AM aishwarya kumar <as...@gmail.com>
> > > wrote:
> > > >
> > > > Thank you for the suggestion Matthais, i've made the necessary
> changes in
> > > > the KIP.
> > > >
> > > > Keeping this thread open for further input.
> > > > KIP link:
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL
> > > >
> > > > Best,
> > > > Aishwarya
> > > >
> > > > On Thu, Sep 19, 2019 at 10:50 AM aishwarya kumar <ash26389@gmail.com
> >
> > > wrote:
> > > >
> > > > > Thanks Matthias,
> > > > >
> > > > > That does make sense, let me update the KIP to reflect the
> > > Materialization
> > > > > scenario.
> > > > >
> > > > > Best,
> > > > > Aishwarya
> > > > >
> > > > > On Tue, Sep 17, 2019, 2:49 PM Matthias J. Sax <
> matthias@confluent.io>
> > > > > wrote:
> > > > >
> > > > >> Aishwarya,
> > > > >>
> > > > >> thanks for the KIP. Overall, I think it makes sense to allow
> > > converting
> > > > >> a KStream into a KTable.
> > > > >>
> > > > >> From the KIP:
> > > > >>
> > > > >> > materializing these KTables should only be allowed if the
> overloaded
> > > > >> function with Materialized is used (and if optimization is turned
> on
> > > it may
> > > > >> still be only logically materialized if the queryable name is not
> > > set).
> > > > >>
> > > > >> Can you elaborate? I think the behavior we want should align with
> the
> > > > >> behavior of `StreamsBuilder#table()`.
> > > > >>
> > > > >> From my understanding (correct me if I am wrong) it should be:
> > > > >>
> > > > >> (1) If optimization is turned off, the KTable will always be
> > > > >> materialized, independent which method is used. The KTable will
> not be
> > > > >> queryable though.
> > > > >>
> > > > >> (2) If optimization is turned on and if `toTable()` is used, the
> > > KTable
> > > > >> may or may not be materialized. For this case, even if the KTable
> is
> > > > >> materialized, the store would not be queryable.
> > > > >>
> > > > >> (3) If `toTable(Materialized)` is use and a `storeName` or
> > > > >> `StoreSupplier` is specified, the store will always be
> materialized
> > > and
> > > > >> also be queryable. Otherwise, case (1) or (2) applies.
> > > > >>
> > > > >>
> > > > >>
> > > > >> -Matthias
> > > > >>
> > > > >>
> > > > >> On 9/17/19 6:42 AM, aishwarya kumar wrote:
> > > > >> > Hi All,
> > > > >> >
> > > > >> > Keeping this thread alive!!
> > > > >> >
> > > > >> > The aim is to add two methods Kstream.toTable() &
> > > > >> > Kstream.toTable(Materialized<K,V>), so users can choose to
> convert
> > > their
> > > > >> > event stream into a changelog stream at any stage.
> > > > >> > wiki link :
> > > > >> >
> > > > >>
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
> > > > >> > jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
> > > > >> >
> > > > >> > Best,
> > > > >> > Aishwarya
> > > > >> >
> > > > >> > On Fri, Sep 13, 2019 at 10:49 AM aishwarya kumar <
> > > ash26389@gmail.com>
> > > > >> wrote:
> > > > >> >
> > > > >> >> Hello,
> > > > >> >>
> > > > >> >> Starting this thread to discuss KIP-532:
> > > > >> >> wiki link :
> > > > >> >>
> > > > >>
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
> > > > >> >> jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
> > > > >> >>
> > > > >> >> There has been some discussion around the use-case of this KIP
> in
> > > the
> > > > >> Jira
> > > > >> >> ticket.
> > > > >> >>
> > > > >> >> Regards,
> > > > >> >> Aishwarya
> > > > >> >>
> > > > >> >
> > > > >>
> > > > >>
> > >
>
Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL
Posted by aishwarya kumar <as...@gmail.com>.
Apologies for the delay.
I have made the changes in the KIP, i'll be starting the voting process
shortly.
Best regards,
Aishwarya Kumar
On Mon, Oct 7, 2019 at 6:06 PM Matthias J. Sax <ma...@confluent.io>
wrote:
> Aishwarya,
>
> Why is bullet point (2) formatted as "strike through"? If you intend to
> replace it with bullet point (3), just remove it completely. The KIP
> should reflect the actual proposal. Maybe move it to "Rejected
> Alternative" section?
>
> For (3c), it should also say, if `Materialize` specifies a queryable
> store name. If there is no store name provided, either (a) or (b) applies.
>
>
>
> Overall LGTM. Feel free to start a vote.
>
>
> -Matthias
>
>
>
> On 10/1/19 7:48 AM, aishwarya kumar wrote:
> > Thank you all for the feedback, I will keep this thread open for
> discussion
> > for a couple of more days and then start with the voting process.
> >
> > Best regards,
> > Aishwarya
> >
> > On Fri, Sep 27, 2019, 12:37 PM John Roesler <jo...@confluent.io> wrote:
> >
> >> Looks good to me! I have no further comments.
> >>
> >> Thanks again for the KIP, Aishwarya!
> >> -John
> >>
> >> On Fri, Sep 27, 2019 at 10:11 AM aishwarya kumar <as...@gmail.com>
> >> wrote:
> >>>
> >>> Hello John,
> >>>
> >>> Thank you for pointing this out to me, to maintain consistency across
> >> API's
> >>> it does make sense to allow users to define custom names for
> >>> their processors.
> >>>
> >>> I've made the change in the KIP:
> >>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL
> >>>
> >>> Best,
> >>> Aishwarya
> >>>
> >>> On Tue, Sep 24, 2019 at 11:54 AM John Roesler <jo...@confluent.io>
> wrote:
> >>>
> >>>> Hey Aishwarya,
> >>>>
> >>>> Thanks for the KIP! It looks good to me, although in a post-KIP-307
> >>>> world, we also need a "Named" parameter (to give the processor node a
> >>>> name, as opposed to the store itself).
> >>>>
> >>>> This would result in a total of four overloads:
> >>>> 1. no args
> >>>> 2. Named
> >>>> 3. Materialized
> >>>> 4. Materialized, Named
> >>>>
> >>>> I'd like to propose a re-design of the DSL in the future to clean this
> >>>> up, but for now, this is the pattern we have to follow.
> >>>>
> >>>> Thoughts?
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Tue, Sep 24, 2019 at 9:54 AM aishwarya kumar <as...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Thank you for the suggestion Matthais, i've made the necessary
> >> changes in
> >>>>> the KIP.
> >>>>>
> >>>>> Keeping this thread open for further input.
> >>>>> KIP link:
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL
> >>>>>
> >>>>> Best,
> >>>>> Aishwarya
> >>>>>
> >>>>> On Thu, Sep 19, 2019 at 10:50 AM aishwarya kumar <ash26389@gmail.com
> >>>
> >>>> wrote:
> >>>>>
> >>>>>> Thanks Matthias,
> >>>>>>
> >>>>>> That does make sense, let me update the KIP to reflect the
> >>>> Materialization
> >>>>>> scenario.
> >>>>>>
> >>>>>> Best,
> >>>>>> Aishwarya
> >>>>>>
> >>>>>> On Tue, Sep 17, 2019, 2:49 PM Matthias J. Sax <
> >> matthias@confluent.io>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Aishwarya,
> >>>>>>>
> >>>>>>> thanks for the KIP. Overall, I think it makes sense to allow
> >>>> converting
> >>>>>>> a KStream into a KTable.
> >>>>>>>
> >>>>>>> From the KIP:
> >>>>>>>
> >>>>>>>> materializing these KTables should only be allowed if the
> >> overloaded
> >>>>>>> function with Materialized is used (and if optimization is turned
> >> on
> >>>> it may
> >>>>>>> still be only logically materialized if the queryable name is not
> >>>> set).
> >>>>>>>
> >>>>>>> Can you elaborate? I think the behavior we want should align with
> >> the
> >>>>>>> behavior of `StreamsBuilder#table()`.
> >>>>>>>
> >>>>>>> From my understanding (correct me if I am wrong) it should be:
> >>>>>>>
> >>>>>>> (1) If optimization is turned off, the KTable will always be
> >>>>>>> materialized, independent which method is used. The KTable will
> >> not be
> >>>>>>> queryable though.
> >>>>>>>
> >>>>>>> (2) If optimization is turned on and if `toTable()` is used, the
> >>>> KTable
> >>>>>>> may or may not be materialized. For this case, even if the KTable
> >> is
> >>>>>>> materialized, the store would not be queryable.
> >>>>>>>
> >>>>>>> (3) If `toTable(Materialized)` is use and a `storeName` or
> >>>>>>> `StoreSupplier` is specified, the store will always be
> >> materialized
> >>>> and
> >>>>>>> also be queryable. Otherwise, case (1) or (2) applies.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> -Matthias
> >>>>>>>
> >>>>>>>
> >>>>>>> On 9/17/19 6:42 AM, aishwarya kumar wrote:
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> Keeping this thread alive!!
> >>>>>>>>
> >>>>>>>> The aim is to add two methods Kstream.toTable() &
> >>>>>>>> Kstream.toTable(Materialized<K,V>), so users can choose to
> >> convert
> >>>> their
> >>>>>>>> event stream into a changelog stream at any stage.
> >>>>>>>> wiki link :
> >>>>>>>>
> >>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
> >>>>>>>> jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Aishwarya
> >>>>>>>>
> >>>>>>>> On Fri, Sep 13, 2019 at 10:49 AM aishwarya kumar <
> >>>> ash26389@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hello,
> >>>>>>>>>
> >>>>>>>>> Starting this thread to discuss KIP-532:
> >>>>>>>>> wiki link :
> >>>>>>>>>
> >>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
> >>>>>>>>> jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
> >>>>>>>>>
> >>>>>>>>> There has been some discussion around the use-case of this KIP
> >> in
> >>>> the
> >>>>>>> Jira
> >>>>>>>>> ticket.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Aishwarya
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>
> >>
> >
>
>
Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL
Posted by "Matthias J. Sax" <ma...@confluent.io>.
Any update on this KIP?
On 10/7/19 3:06 PM, Matthias J. Sax wrote:
> Aishwarya,
>
> Why is bullet point (2) formatted as "strike through"? If you intend to
> replace it with bullet point (3), just remove it completely. The KIP
> should reflect the actual proposal. Maybe move it to "Rejected
> Alternative" section?
>
> For (3c), it should also say, if `Materialize` specifies a queryable
> store name. If there is no store name provided, either (a) or (b) applies.
>
>
>
> Overall LGTM. Feel free to start a vote.
>
>
> -Matthias
>
>
>
> On 10/1/19 7:48 AM, aishwarya kumar wrote:
>> Thank you all for the feedback, I will keep this thread open for discussion
>> for a couple of more days and then start with the voting process.
>>
>> Best regards,
>> Aishwarya
>>
>> On Fri, Sep 27, 2019, 12:37 PM John Roesler <jo...@confluent.io> wrote:
>>
>>> Looks good to me! I have no further comments.
>>>
>>> Thanks again for the KIP, Aishwarya!
>>> -John
>>>
>>> On Fri, Sep 27, 2019 at 10:11 AM aishwarya kumar <as...@gmail.com>
>>> wrote:
>>>>
>>>> Hello John,
>>>>
>>>> Thank you for pointing this out to me, to maintain consistency across
>>> API's
>>>> it does make sense to allow users to define custom names for
>>>> their processors.
>>>>
>>>> I've made the change in the KIP:
>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL
>>>>
>>>> Best,
>>>> Aishwarya
>>>>
>>>> On Tue, Sep 24, 2019 at 11:54 AM John Roesler <jo...@confluent.io> wrote:
>>>>
>>>>> Hey Aishwarya,
>>>>>
>>>>> Thanks for the KIP! It looks good to me, although in a post-KIP-307
>>>>> world, we also need a "Named" parameter (to give the processor node a
>>>>> name, as opposed to the store itself).
>>>>>
>>>>> This would result in a total of four overloads:
>>>>> 1. no args
>>>>> 2. Named
>>>>> 3. Materialized
>>>>> 4. Materialized, Named
>>>>>
>>>>> I'd like to propose a re-design of the DSL in the future to clean this
>>>>> up, but for now, this is the pattern we have to follow.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Thanks,
>>>>> -John
>>>>>
>>>>> On Tue, Sep 24, 2019 at 9:54 AM aishwarya kumar <as...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Thank you for the suggestion Matthais, i've made the necessary
>>> changes in
>>>>>> the KIP.
>>>>>>
>>>>>> Keeping this thread open for further input.
>>>>>> KIP link:
>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL
>>>>>>
>>>>>> Best,
>>>>>> Aishwarya
>>>>>>
>>>>>> On Thu, Sep 19, 2019 at 10:50 AM aishwarya kumar <ash26389@gmail.com
>>>>
>>>>> wrote:
>>>>>>
>>>>>>> Thanks Matthias,
>>>>>>>
>>>>>>> That does make sense, let me update the KIP to reflect the
>>>>> Materialization
>>>>>>> scenario.
>>>>>>>
>>>>>>> Best,
>>>>>>> Aishwarya
>>>>>>>
>>>>>>> On Tue, Sep 17, 2019, 2:49 PM Matthias J. Sax <
>>> matthias@confluent.io>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Aishwarya,
>>>>>>>>
>>>>>>>> thanks for the KIP. Overall, I think it makes sense to allow
>>>>> converting
>>>>>>>> a KStream into a KTable.
>>>>>>>>
>>>>>>>> From the KIP:
>>>>>>>>
>>>>>>>>> materializing these KTables should only be allowed if the
>>> overloaded
>>>>>>>> function with Materialized is used (and if optimization is turned
>>> on
>>>>> it may
>>>>>>>> still be only logically materialized if the queryable name is not
>>>>> set).
>>>>>>>>
>>>>>>>> Can you elaborate? I think the behavior we want should align with
>>> the
>>>>>>>> behavior of `StreamsBuilder#table()`.
>>>>>>>>
>>>>>>>> From my understanding (correct me if I am wrong) it should be:
>>>>>>>>
>>>>>>>> (1) If optimization is turned off, the KTable will always be
>>>>>>>> materialized, independent which method is used. The KTable will
>>> not be
>>>>>>>> queryable though.
>>>>>>>>
>>>>>>>> (2) If optimization is turned on and if `toTable()` is used, the
>>>>> KTable
>>>>>>>> may or may not be materialized. For this case, even if the KTable
>>> is
>>>>>>>> materialized, the store would not be queryable.
>>>>>>>>
>>>>>>>> (3) If `toTable(Materialized)` is use and a `storeName` or
>>>>>>>> `StoreSupplier` is specified, the store will always be
>>> materialized
>>>>> and
>>>>>>>> also be queryable. Otherwise, case (1) or (2) applies.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -Matthias
>>>>>>>>
>>>>>>>>
>>>>>>>> On 9/17/19 6:42 AM, aishwarya kumar wrote:
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> Keeping this thread alive!!
>>>>>>>>>
>>>>>>>>> The aim is to add two methods Kstream.toTable() &
>>>>>>>>> Kstream.toTable(Materialized<K,V>), so users can choose to
>>> convert
>>>>> their
>>>>>>>>> event stream into a changelog stream at any stage.
>>>>>>>>> wiki link :
>>>>>>>>>
>>>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
>>>>>>>>> jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Aishwarya
>>>>>>>>>
>>>>>>>>> On Fri, Sep 13, 2019 at 10:49 AM aishwarya kumar <
>>>>> ash26389@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> Starting this thread to discuss KIP-532:
>>>>>>>>>> wiki link :
>>>>>>>>>>
>>>>>>>>
>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
>>>>>>>>>> jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
>>>>>>>>>>
>>>>>>>>>> There has been some discussion around the use-case of this KIP
>>> in
>>>>> the
>>>>>>>> Jira
>>>>>>>>>> ticket.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Aishwarya
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>
>>>
>>
>
Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL
Posted by "Matthias J. Sax" <ma...@confluent.io>.
Aishwarya,
Why is bullet point (2) formatted as "strike through"? If you intend to
replace it with bullet point (3), just remove it completely. The KIP
should reflect the actual proposal. Maybe move it to "Rejected
Alternative" section?
For (3c), it should also say, if `Materialize` specifies a queryable
store name. If there is no store name provided, either (a) or (b) applies.
Overall LGTM. Feel free to start a vote.
-Matthias
On 10/1/19 7:48 AM, aishwarya kumar wrote:
> Thank you all for the feedback, I will keep this thread open for discussion
> for a couple of more days and then start with the voting process.
>
> Best regards,
> Aishwarya
>
> On Fri, Sep 27, 2019, 12:37 PM John Roesler <jo...@confluent.io> wrote:
>
>> Looks good to me! I have no further comments.
>>
>> Thanks again for the KIP, Aishwarya!
>> -John
>>
>> On Fri, Sep 27, 2019 at 10:11 AM aishwarya kumar <as...@gmail.com>
>> wrote:
>>>
>>> Hello John,
>>>
>>> Thank you for pointing this out to me, to maintain consistency across
>> API's
>>> it does make sense to allow users to define custom names for
>>> their processors.
>>>
>>> I've made the change in the KIP:
>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL
>>>
>>> Best,
>>> Aishwarya
>>>
>>> On Tue, Sep 24, 2019 at 11:54 AM John Roesler <jo...@confluent.io> wrote:
>>>
>>>> Hey Aishwarya,
>>>>
>>>> Thanks for the KIP! It looks good to me, although in a post-KIP-307
>>>> world, we also need a "Named" parameter (to give the processor node a
>>>> name, as opposed to the store itself).
>>>>
>>>> This would result in a total of four overloads:
>>>> 1. no args
>>>> 2. Named
>>>> 3. Materialized
>>>> 4. Materialized, Named
>>>>
>>>> I'd like to propose a re-design of the DSL in the future to clean this
>>>> up, but for now, this is the pattern we have to follow.
>>>>
>>>> Thoughts?
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Tue, Sep 24, 2019 at 9:54 AM aishwarya kumar <as...@gmail.com>
>>>> wrote:
>>>>>
>>>>> Thank you for the suggestion Matthais, i've made the necessary
>> changes in
>>>>> the KIP.
>>>>>
>>>>> Keeping this thread open for further input.
>>>>> KIP link:
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL
>>>>>
>>>>> Best,
>>>>> Aishwarya
>>>>>
>>>>> On Thu, Sep 19, 2019 at 10:50 AM aishwarya kumar <ash26389@gmail.com
>>>
>>>> wrote:
>>>>>
>>>>>> Thanks Matthias,
>>>>>>
>>>>>> That does make sense, let me update the KIP to reflect the
>>>> Materialization
>>>>>> scenario.
>>>>>>
>>>>>> Best,
>>>>>> Aishwarya
>>>>>>
>>>>>> On Tue, Sep 17, 2019, 2:49 PM Matthias J. Sax <
>> matthias@confluent.io>
>>>>>> wrote:
>>>>>>
>>>>>>> Aishwarya,
>>>>>>>
>>>>>>> thanks for the KIP. Overall, I think it makes sense to allow
>>>> converting
>>>>>>> a KStream into a KTable.
>>>>>>>
>>>>>>> From the KIP:
>>>>>>>
>>>>>>>> materializing these KTables should only be allowed if the
>> overloaded
>>>>>>> function with Materialized is used (and if optimization is turned
>> on
>>>> it may
>>>>>>> still be only logically materialized if the queryable name is not
>>>> set).
>>>>>>>
>>>>>>> Can you elaborate? I think the behavior we want should align with
>> the
>>>>>>> behavior of `StreamsBuilder#table()`.
>>>>>>>
>>>>>>> From my understanding (correct me if I am wrong) it should be:
>>>>>>>
>>>>>>> (1) If optimization is turned off, the KTable will always be
>>>>>>> materialized, independent which method is used. The KTable will
>> not be
>>>>>>> queryable though.
>>>>>>>
>>>>>>> (2) If optimization is turned on and if `toTable()` is used, the
>>>> KTable
>>>>>>> may or may not be materialized. For this case, even if the KTable
>> is
>>>>>>> materialized, the store would not be queryable.
>>>>>>>
>>>>>>> (3) If `toTable(Materialized)` is use and a `storeName` or
>>>>>>> `StoreSupplier` is specified, the store will always be
>> materialized
>>>> and
>>>>>>> also be queryable. Otherwise, case (1) or (2) applies.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -Matthias
>>>>>>>
>>>>>>>
>>>>>>> On 9/17/19 6:42 AM, aishwarya kumar wrote:
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> Keeping this thread alive!!
>>>>>>>>
>>>>>>>> The aim is to add two methods Kstream.toTable() &
>>>>>>>> Kstream.toTable(Materialized<K,V>), so users can choose to
>> convert
>>>> their
>>>>>>>> event stream into a changelog stream at any stage.
>>>>>>>> wiki link :
>>>>>>>>
>>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
>>>>>>>> jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Aishwarya
>>>>>>>>
>>>>>>>> On Fri, Sep 13, 2019 at 10:49 AM aishwarya kumar <
>>>> ash26389@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> Starting this thread to discuss KIP-532:
>>>>>>>>> wiki link :
>>>>>>>>>
>>>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
>>>>>>>>> jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
>>>>>>>>>
>>>>>>>>> There has been some discussion around the use-case of this KIP
>> in
>>>> the
>>>>>>> Jira
>>>>>>>>> ticket.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Aishwarya
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>
>