You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Panuwat Anawatmongkhon <pa...@gmail.com> on 2017/12/04 15:39:10 UTC
[DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Hi all,
I created a KIP.
https://cwiki.apache.org/confluence/display/KAFKA/KIP233%3A+Simplify+StreamsBuilder%23addGlobalStore
Cheers,
Benz
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by "Matthias J. Sax" <ma...@confluent.io>.
Benz,
> so when replace the old api usage
>> with the new one we don’t have any kind of auto renaming so the backup
>> topic name will be inconsistent.
Yes.
Note, that for global state stores, there won't be any internally
created changelog topics because the original input topic can be used
for fault-tolerance. With this regard, it's no issue to switch from old
API to new API.
However, internally Kafka Streams uses an counter to generate names, and
the new API will use this counter, too. Thus, generated name will change
affecting other operators.
Thus, users that upgrade, should stay on old API to avoid this issues.
We need to point this out in the upgrade and Java docs.
However, new applications, can use the new API.
-Matthias
On 3/19/18 1:47 AM, Panuwat Anawatmongkhon wrote:
> Hi Matthias,
> I am not sure weather I understand the impact correctly. With the old api
> user have to specify name for state store and the name is used for auto
> generated topics for fault tolerance, so when replace the old api usage
> with the new one we don’t have any kind of auto renaming so the backup
> topic name will be inconsistent.
> Cheers,
> Benz
>
> On Mon, 19 Mar 2561 at 10:59 Panuwat Anawatmongkhon <
> panuwat.anawatmongkhon@gmail.com> wrote:
>
>> I couldn’t find this thread in mailing list. this reply is just to trigger
>> it up so I can include it in KIP
>>
>> On Thu, 28 Dec 2560 at 03:07 Matthias J. Sax <ma...@confluent.io>
>> wrote:
>>
>>> @Matthias: just wanted to follow up on your question:
>>>
>>>>>>> I wanted to double check. If I understand the proposal, it would
>>> replace
>>>>>>> the explicit name with a name that is dynamically generated using the
>>>>>>> AtomicInteger index. Would this affect the naming of any internally
>>>>>>> generated topics?
>>>
>>> I think it would -- note, that the old API will not be removed but
>>> deprecated -- thus, you can still update without any issues staying with
>>> the old API -- only if you start to use the new API, it could impact an
>>> application.
>>>
>>> It should still be possible to upgrade to the new API if you invest the
>>> time to rename the corresponding topics correctly -- this will only work
>>> if you use a new application id or take your application offline though.
>>>
>>> -Matthias
>>>
>>>
>>> On 12/16/17 10:17 AM, Panuwat Anawatmongkhon wrote:
>>>> Hi all,
>>>> I would like to start the vote thread tomorrow, feel free to ask if
>>> there
>>>> is any concern.
>>>> Thank you
>>>>
>>>> On Thu, 7 Dec 2560 at 19:22 Panuwat Anawatmongkhon <
>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>>>>
>>>>>
>>>>> Yes, Matthias.
>>>>> The object will be used togerther with function table and function
>>> stream.
>>>>> I didn’t see how this will affect other part but if you do, please
>>> explain
>>>>> more on how this will affect generated topic name.
>>>>> Thank you
>>>>> Panuwat
>>>>>
>>>>>
>>>>> On Thu, 7 Dec 2560 at 00:01 Matthias Margush <
>>> matthias.margush@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi.
>>>>>>
>>>>>> I wanted to double check. If I understand the proposal, it would
>>> replace
>>>>>> the explicit name with a name that is dynamically generated using the
>>>>>> AtomicInteger index. Would this affect the naming of any internally
>>>>>> generated topics?
>>>>>>
>>>>>> On Wed, Dec 6, 2017 at 7:59 AM Panuwat Anawatmongkhon <
>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>>>>>>
>>>>>>> Thanks Bill.
>>>>>>>
>>>>>>> I can't think of reason to keep the old method too so if there is no
>>>>>>> further discussion by tomorrow, I would like to start the vote
>>> thread.
>>>>>>>
>>>>>>> On Tue, Dec 5, 2017 at 10:38 PM, Bill Bejeck <bb...@gmail.com>
>>> wrote:
>>>>>>>
>>>>>>>> Hi Panuwat,
>>>>>>>>
>>>>>>>> Thanks for the KIP, overall looks good to me.
>>>>>>>>
>>>>>>>> I want to play the devil's advocate for a second and ask do we want
>>> to
>>>>>>> keep
>>>>>>>> the older method with the extra parameters vs. deprecation?
>>>>>>>>
>>>>>>>> Although ATM I can't think of a good reason to keep the old method
>>>>>> with
>>>>>>> the
>>>>>>>> extra parameters.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Bill
>>>>>>>>
>>>>>>>> On Tue, Dec 5, 2017 at 5:48 AM, Ted Yu <yu...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Fine by me.
>>>>>>>>>
>>>>>>>>> On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
>>>>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Thank you, Matthias.
>>>>>>>>>>
>>>>>>>>>> Ted,
>>>>>>>>>> How about this.
>>>>>>>>>>
>>>>>>>>>> String globalTopicName = "testGlobalTopic";
>>>>>>>>>> String globalStoreName = "testAddGlobalStore";
>>>>>>>>>> final StreamsBuilder builder = new StreamsBuilder();
>>>>>>>>>> final KeyValueStoreBuilder globalStoreBuilder =
>>>>>>>>>> EasyMock.createNiceMock(KeyValueStoreBuilder.class);
>>>>>>>>>>
>>>>>>>
>>> EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
>>>>>>>>>> anyTimes();
>>>>>>>>>> EasyMock.replay(globalStoreBuilder);
>>>>>>>>>> builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
>>>>>>>>>> ConsumedInternal(),new MockProcessorSupplier());
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <
>>>>>>> matthias@confluent.io
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Panuwat,
>>>>>>>>>>>
>>>>>>>>>>> Thanks a lot for the KIP!
>>>>>>>>>>>
>>>>>>>>>>> Just one nit: `does not follow provide a good` -> spelling:
>>>>>> remove
>>>>>>>>>>> `follow` ?
>>>>>>>>>>>
>>>>>>>>>>> Otherwise, looks good to me.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -Matthias
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 12/4/17 10:49 AM, Ted Yu wrote:
>>>>>>>>>>>> Looks like you're implying logic similar to this:
>>>>>>>>>>>>
>>>>>>>>>>>> public synchronized <K, V> GlobalKTable<K, V>
>>>>>>> globalTable(final
>>>>>>>>>>> String
>>>>>>>>>>>> topic,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>> final
>>>>>>>>>>>> Consumed<K, V> consumed) {
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> StreamsBuilder is returned instead of GlobalKTable.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Can you add code snippet showing how the new API is used ?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
>>>>>>>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> What i am thinking right now is using the same approach as
>>>>>>>>>>>>> org.apache.kafka.streams.kstream.internals.
>>>>>>>> InternalStreamsBuilder#
>>>>>>>>>>>>> globalTable
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com>
>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can you describe how sourceName is inferred based on the new
>>>>>>> API
>>>>>>>> ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please fill out JIRA number.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> BTW here is the URL for the KIP:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>>>>>>>> 233%3A+Simplify+
>>>>>>>>>>>>> StreamsBuilder%23addGlobalStore
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
>>>>>>>>>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>> I created a KIP.
>>>>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/
>>>>>>>>>> KIP233%3A+Simplify+
>>>>>>>>>>>>>>> StreamsBuilder%23addGlobalStore
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>> Benz
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Panuwat Anawatmongkhon <pa...@gmail.com>.
Hi Matthias,
I am not sure weather I understand the impact correctly. With the old api
user have to specify name for state store and the name is used for auto
generated topics for fault tolerance, so when replace the old api usage
with the new one we don’t have any kind of auto renaming so the backup
topic name will be inconsistent.
Cheers,
Benz
On Mon, 19 Mar 2561 at 10:59 Panuwat Anawatmongkhon <
panuwat.anawatmongkhon@gmail.com> wrote:
> I couldn’t find this thread in mailing list. this reply is just to trigger
> it up so I can include it in KIP
>
> On Thu, 28 Dec 2560 at 03:07 Matthias J. Sax <ma...@confluent.io>
> wrote:
>
>> @Matthias: just wanted to follow up on your question:
>>
>> >>>> I wanted to double check. If I understand the proposal, it would
>> replace
>> >>>> the explicit name with a name that is dynamically generated using the
>> >>>> AtomicInteger index. Would this affect the naming of any internally
>> >>>> generated topics?
>>
>> I think it would -- note, that the old API will not be removed but
>> deprecated -- thus, you can still update without any issues staying with
>> the old API -- only if you start to use the new API, it could impact an
>> application.
>>
>> It should still be possible to upgrade to the new API if you invest the
>> time to rename the corresponding topics correctly -- this will only work
>> if you use a new application id or take your application offline though.
>>
>> -Matthias
>>
>>
>> On 12/16/17 10:17 AM, Panuwat Anawatmongkhon wrote:
>> > Hi all,
>> > I would like to start the vote thread tomorrow, feel free to ask if
>> there
>> > is any concern.
>> > Thank you
>> >
>> > On Thu, 7 Dec 2560 at 19:22 Panuwat Anawatmongkhon <
>> > panuwat.anawatmongkhon@gmail.com> wrote:
>> >
>> >>
>> >> Yes, Matthias.
>> >> The object will be used togerther with function table and function
>> stream.
>> >> I didn’t see how this will affect other part but if you do, please
>> explain
>> >> more on how this will affect generated topic name.
>> >> Thank you
>> >> Panuwat
>> >>
>> >>
>> >> On Thu, 7 Dec 2560 at 00:01 Matthias Margush <
>> matthias.margush@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi.
>> >>>
>> >>> I wanted to double check. If I understand the proposal, it would
>> replace
>> >>> the explicit name with a name that is dynamically generated using the
>> >>> AtomicInteger index. Would this affect the naming of any internally
>> >>> generated topics?
>> >>>
>> >>> On Wed, Dec 6, 2017 at 7:59 AM Panuwat Anawatmongkhon <
>> >>> panuwat.anawatmongkhon@gmail.com> wrote:
>> >>>
>> >>>> Thanks Bill.
>> >>>>
>> >>>> I can't think of reason to keep the old method too so if there is no
>> >>>> further discussion by tomorrow, I would like to start the vote
>> thread.
>> >>>>
>> >>>> On Tue, Dec 5, 2017 at 10:38 PM, Bill Bejeck <bb...@gmail.com>
>> wrote:
>> >>>>
>> >>>>> Hi Panuwat,
>> >>>>>
>> >>>>> Thanks for the KIP, overall looks good to me.
>> >>>>>
>> >>>>> I want to play the devil's advocate for a second and ask do we want
>> to
>> >>>> keep
>> >>>>> the older method with the extra parameters vs. deprecation?
>> >>>>>
>> >>>>> Although ATM I can't think of a good reason to keep the old method
>> >>> with
>> >>>> the
>> >>>>> extra parameters.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Bill
>> >>>>>
>> >>>>> On Tue, Dec 5, 2017 at 5:48 AM, Ted Yu <yu...@gmail.com> wrote:
>> >>>>>
>> >>>>>> Fine by me.
>> >>>>>>
>> >>>>>> On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
>> >>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>> >>>>>>
>> >>>>>>> Thank you, Matthias.
>> >>>>>>>
>> >>>>>>> Ted,
>> >>>>>>> How about this.
>> >>>>>>>
>> >>>>>>> String globalTopicName = "testGlobalTopic";
>> >>>>>>> String globalStoreName = "testAddGlobalStore";
>> >>>>>>> final StreamsBuilder builder = new StreamsBuilder();
>> >>>>>>> final KeyValueStoreBuilder globalStoreBuilder =
>> >>>>>>> EasyMock.createNiceMock(KeyValueStoreBuilder.class);
>> >>>>>>>
>> >>>>
>> EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
>> >>>>>>> anyTimes();
>> >>>>>>> EasyMock.replay(globalStoreBuilder);
>> >>>>>>> builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
>> >>>>>>> ConsumedInternal(),new MockProcessorSupplier());
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <
>> >>>> matthias@confluent.io
>> >>>>>>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Panuwat,
>> >>>>>>>>
>> >>>>>>>> Thanks a lot for the KIP!
>> >>>>>>>>
>> >>>>>>>> Just one nit: `does not follow provide a good` -> spelling:
>> >>> remove
>> >>>>>>>> `follow` ?
>> >>>>>>>>
>> >>>>>>>> Otherwise, looks good to me.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> -Matthias
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On 12/4/17 10:49 AM, Ted Yu wrote:
>> >>>>>>>>> Looks like you're implying logic similar to this:
>> >>>>>>>>>
>> >>>>>>>>> public synchronized <K, V> GlobalKTable<K, V>
>> >>>> globalTable(final
>> >>>>>>>> String
>> >>>>>>>>> topic,
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>> final
>> >>>>>>>>> Consumed<K, V> consumed) {
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> StreamsBuilder is returned instead of GlobalKTable.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Can you add code snippet showing how the new API is used ?
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
>> >>>>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> What i am thinking right now is using the same approach as
>> >>>>>>>>>> org.apache.kafka.streams.kstream.internals.
>> >>>>> InternalStreamsBuilder#
>> >>>>>>>>>> globalTable
>> >>>>>>>>>>
>> >>>>>>>>>> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com>
>> >>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Can you describe how sourceName is inferred based on the new
>> >>>> API
>> >>>>> ?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Please fill out JIRA number.
>> >>>>>>>>>>>
>> >>>>>>>>>>> BTW here is the URL for the KIP:
>> >>>>>>>>>>>
>> >>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> >>>>>>> 233%3A+Simplify+
>> >>>>>>>>>> StreamsBuilder%23addGlobalStore
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
>> >>>>>>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Hi all,
>> >>>>>>>>>>>> I created a KIP.
>> >>>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/
>> >>>>>>> KIP233%3A+Simplify+
>> >>>>>>>>>>>> StreamsBuilder%23addGlobalStore
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Cheers,
>> >>>>>>>>>>>> Benz
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>>
>>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Panuwat Anawatmongkhon <pa...@gmail.com>.
I couldn’t find this thread in mailing list. this reply is just to trigger
it up so I can include it in KIP
On Thu, 28 Dec 2560 at 03:07 Matthias J. Sax <ma...@confluent.io> wrote:
> @Matthias: just wanted to follow up on your question:
>
> >>>> I wanted to double check. If I understand the proposal, it would
> replace
> >>>> the explicit name with a name that is dynamically generated using the
> >>>> AtomicInteger index. Would this affect the naming of any internally
> >>>> generated topics?
>
> I think it would -- note, that the old API will not be removed but
> deprecated -- thus, you can still update without any issues staying with
> the old API -- only if you start to use the new API, it could impact an
> application.
>
> It should still be possible to upgrade to the new API if you invest the
> time to rename the corresponding topics correctly -- this will only work
> if you use a new application id or take your application offline though.
>
> -Matthias
>
>
> On 12/16/17 10:17 AM, Panuwat Anawatmongkhon wrote:
> > Hi all,
> > I would like to start the vote thread tomorrow, feel free to ask if there
> > is any concern.
> > Thank you
> >
> > On Thu, 7 Dec 2560 at 19:22 Panuwat Anawatmongkhon <
> > panuwat.anawatmongkhon@gmail.com> wrote:
> >
> >>
> >> Yes, Matthias.
> >> The object will be used togerther with function table and function
> stream.
> >> I didn’t see how this will affect other part but if you do, please
> explain
> >> more on how this will affect generated topic name.
> >> Thank you
> >> Panuwat
> >>
> >>
> >> On Thu, 7 Dec 2560 at 00:01 Matthias Margush <
> matthias.margush@gmail.com>
> >> wrote:
> >>
> >>> Hi.
> >>>
> >>> I wanted to double check. If I understand the proposal, it would
> replace
> >>> the explicit name with a name that is dynamically generated using the
> >>> AtomicInteger index. Would this affect the naming of any internally
> >>> generated topics?
> >>>
> >>> On Wed, Dec 6, 2017 at 7:59 AM Panuwat Anawatmongkhon <
> >>> panuwat.anawatmongkhon@gmail.com> wrote:
> >>>
> >>>> Thanks Bill.
> >>>>
> >>>> I can't think of reason to keep the old method too so if there is no
> >>>> further discussion by tomorrow, I would like to start the vote thread.
> >>>>
> >>>> On Tue, Dec 5, 2017 at 10:38 PM, Bill Bejeck <bb...@gmail.com>
> wrote:
> >>>>
> >>>>> Hi Panuwat,
> >>>>>
> >>>>> Thanks for the KIP, overall looks good to me.
> >>>>>
> >>>>> I want to play the devil's advocate for a second and ask do we want
> to
> >>>> keep
> >>>>> the older method with the extra parameters vs. deprecation?
> >>>>>
> >>>>> Although ATM I can't think of a good reason to keep the old method
> >>> with
> >>>> the
> >>>>> extra parameters.
> >>>>>
> >>>>> Thanks,
> >>>>> Bill
> >>>>>
> >>>>> On Tue, Dec 5, 2017 at 5:48 AM, Ted Yu <yu...@gmail.com> wrote:
> >>>>>
> >>>>>> Fine by me.
> >>>>>>
> >>>>>> On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
> >>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
> >>>>>>
> >>>>>>> Thank you, Matthias.
> >>>>>>>
> >>>>>>> Ted,
> >>>>>>> How about this.
> >>>>>>>
> >>>>>>> String globalTopicName = "testGlobalTopic";
> >>>>>>> String globalStoreName = "testAddGlobalStore";
> >>>>>>> final StreamsBuilder builder = new StreamsBuilder();
> >>>>>>> final KeyValueStoreBuilder globalStoreBuilder =
> >>>>>>> EasyMock.createNiceMock(KeyValueStoreBuilder.class);
> >>>>>>>
> >>>> EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
> >>>>>>> anyTimes();
> >>>>>>> EasyMock.replay(globalStoreBuilder);
> >>>>>>> builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
> >>>>>>> ConsumedInternal(),new MockProcessorSupplier());
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <
> >>>> matthias@confluent.io
> >>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Panuwat,
> >>>>>>>>
> >>>>>>>> Thanks a lot for the KIP!
> >>>>>>>>
> >>>>>>>> Just one nit: `does not follow provide a good` -> spelling:
> >>> remove
> >>>>>>>> `follow` ?
> >>>>>>>>
> >>>>>>>> Otherwise, looks good to me.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> -Matthias
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 12/4/17 10:49 AM, Ted Yu wrote:
> >>>>>>>>> Looks like you're implying logic similar to this:
> >>>>>>>>>
> >>>>>>>>> public synchronized <K, V> GlobalKTable<K, V>
> >>>> globalTable(final
> >>>>>>>> String
> >>>>>>>>> topic,
> >>>>>>>>>
> >>>>>>>>>
> >>>> final
> >>>>>>>>> Consumed<K, V> consumed) {
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> StreamsBuilder is returned instead of GlobalKTable.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Can you add code snippet showing how the new API is used ?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
> >>>>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> What i am thinking right now is using the same approach as
> >>>>>>>>>> org.apache.kafka.streams.kstream.internals.
> >>>>> InternalStreamsBuilder#
> >>>>>>>>>> globalTable
> >>>>>>>>>>
> >>>>>>>>>> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com>
> >>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Can you describe how sourceName is inferred based on the new
> >>>> API
> >>>>> ?
> >>>>>>>>>>>
> >>>>>>>>>>> Please fill out JIRA number.
> >>>>>>>>>>>
> >>>>>>>>>>> BTW here is the URL for the KIP:
> >>>>>>>>>>>
> >>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>>>>>> 233%3A+Simplify+
> >>>>>>>>>> StreamsBuilder%23addGlobalStore
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
> >>>>>>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi all,
> >>>>>>>>>>>> I created a KIP.
> >>>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/
> >>>>>>> KIP233%3A+Simplify+
> >>>>>>>>>>>> StreamsBuilder%23addGlobalStore
> >>>>>>>>>>>>
> >>>>>>>>>>>> Cheers,
> >>>>>>>>>>>> Benz
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by "Matthias J. Sax" <ma...@confluent.io>.
@Matthias: just wanted to follow up on your question:
>>>> I wanted to double check. If I understand the proposal, it would replace
>>>> the explicit name with a name that is dynamically generated using the
>>>> AtomicInteger index. Would this affect the naming of any internally
>>>> generated topics?
I think it would -- note, that the old API will not be removed but
deprecated -- thus, you can still update without any issues staying with
the old API -- only if you start to use the new API, it could impact an
application.
It should still be possible to upgrade to the new API if you invest the
time to rename the corresponding topics correctly -- this will only work
if you use a new application id or take your application offline though.
-Matthias
On 12/16/17 10:17 AM, Panuwat Anawatmongkhon wrote:
> Hi all,
> I would like to start the vote thread tomorrow, feel free to ask if there
> is any concern.
> Thank you
>
> On Thu, 7 Dec 2560 at 19:22 Panuwat Anawatmongkhon <
> panuwat.anawatmongkhon@gmail.com> wrote:
>
>>
>> Yes, Matthias.
>> The object will be used togerther with function table and function stream.
>> I didn’t see how this will affect other part but if you do, please explain
>> more on how this will affect generated topic name.
>> Thank you
>> Panuwat
>>
>>
>> On Thu, 7 Dec 2560 at 00:01 Matthias Margush <ma...@gmail.com>
>> wrote:
>>
>>> Hi.
>>>
>>> I wanted to double check. If I understand the proposal, it would replace
>>> the explicit name with a name that is dynamically generated using the
>>> AtomicInteger index. Would this affect the naming of any internally
>>> generated topics?
>>>
>>> On Wed, Dec 6, 2017 at 7:59 AM Panuwat Anawatmongkhon <
>>> panuwat.anawatmongkhon@gmail.com> wrote:
>>>
>>>> Thanks Bill.
>>>>
>>>> I can't think of reason to keep the old method too so if there is no
>>>> further discussion by tomorrow, I would like to start the vote thread.
>>>>
>>>> On Tue, Dec 5, 2017 at 10:38 PM, Bill Bejeck <bb...@gmail.com> wrote:
>>>>
>>>>> Hi Panuwat,
>>>>>
>>>>> Thanks for the KIP, overall looks good to me.
>>>>>
>>>>> I want to play the devil's advocate for a second and ask do we want to
>>>> keep
>>>>> the older method with the extra parameters vs. deprecation?
>>>>>
>>>>> Although ATM I can't think of a good reason to keep the old method
>>> with
>>>> the
>>>>> extra parameters.
>>>>>
>>>>> Thanks,
>>>>> Bill
>>>>>
>>>>> On Tue, Dec 5, 2017 at 5:48 AM, Ted Yu <yu...@gmail.com> wrote:
>>>>>
>>>>>> Fine by me.
>>>>>>
>>>>>> On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>>>>>>
>>>>>>> Thank you, Matthias.
>>>>>>>
>>>>>>> Ted,
>>>>>>> How about this.
>>>>>>>
>>>>>>> String globalTopicName = "testGlobalTopic";
>>>>>>> String globalStoreName = "testAddGlobalStore";
>>>>>>> final StreamsBuilder builder = new StreamsBuilder();
>>>>>>> final KeyValueStoreBuilder globalStoreBuilder =
>>>>>>> EasyMock.createNiceMock(KeyValueStoreBuilder.class);
>>>>>>>
>>>> EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
>>>>>>> anyTimes();
>>>>>>> EasyMock.replay(globalStoreBuilder);
>>>>>>> builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
>>>>>>> ConsumedInternal(),new MockProcessorSupplier());
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <
>>>> matthias@confluent.io
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Panuwat,
>>>>>>>>
>>>>>>>> Thanks a lot for the KIP!
>>>>>>>>
>>>>>>>> Just one nit: `does not follow provide a good` -> spelling:
>>> remove
>>>>>>>> `follow` ?
>>>>>>>>
>>>>>>>> Otherwise, looks good to me.
>>>>>>>>
>>>>>>>>
>>>>>>>> -Matthias
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 12/4/17 10:49 AM, Ted Yu wrote:
>>>>>>>>> Looks like you're implying logic similar to this:
>>>>>>>>>
>>>>>>>>> public synchronized <K, V> GlobalKTable<K, V>
>>>> globalTable(final
>>>>>>>> String
>>>>>>>>> topic,
>>>>>>>>>
>>>>>>>>>
>>>> final
>>>>>>>>> Consumed<K, V> consumed) {
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> StreamsBuilder is returned instead of GlobalKTable.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Can you add code snippet showing how the new API is used ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
>>>>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> What i am thinking right now is using the same approach as
>>>>>>>>>> org.apache.kafka.streams.kstream.internals.
>>>>> InternalStreamsBuilder#
>>>>>>>>>> globalTable
>>>>>>>>>>
>>>>>>>>>> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com>
>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Can you describe how sourceName is inferred based on the new
>>>> API
>>>>> ?
>>>>>>>>>>>
>>>>>>>>>>> Please fill out JIRA number.
>>>>>>>>>>>
>>>>>>>>>>> BTW here is the URL for the KIP:
>>>>>>>>>>>
>>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>>>>> 233%3A+Simplify+
>>>>>>>>>> StreamsBuilder%23addGlobalStore
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
>>>>>>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>> I created a KIP.
>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/
>>>>>>> KIP233%3A+Simplify+
>>>>>>>>>>>> StreamsBuilder%23addGlobalStore
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>> Benz
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Panuwat Anawatmongkhon <pa...@gmail.com>.
Hi all,
I would like to start the vote thread tomorrow, feel free to ask if there
is any concern.
Thank you
On Thu, 7 Dec 2560 at 19:22 Panuwat Anawatmongkhon <
panuwat.anawatmongkhon@gmail.com> wrote:
>
> Yes, Matthias.
> The object will be used togerther with function table and function stream.
> I didn’t see how this will affect other part but if you do, please explain
> more on how this will affect generated topic name.
> Thank you
> Panuwat
>
>
> On Thu, 7 Dec 2560 at 00:01 Matthias Margush <ma...@gmail.com>
> wrote:
>
>> Hi.
>>
>> I wanted to double check. If I understand the proposal, it would replace
>> the explicit name with a name that is dynamically generated using the
>> AtomicInteger index. Would this affect the naming of any internally
>> generated topics?
>>
>> On Wed, Dec 6, 2017 at 7:59 AM Panuwat Anawatmongkhon <
>> panuwat.anawatmongkhon@gmail.com> wrote:
>>
>> > Thanks Bill.
>> >
>> > I can't think of reason to keep the old method too so if there is no
>> > further discussion by tomorrow, I would like to start the vote thread.
>> >
>> > On Tue, Dec 5, 2017 at 10:38 PM, Bill Bejeck <bb...@gmail.com> wrote:
>> >
>> > > Hi Panuwat,
>> > >
>> > > Thanks for the KIP, overall looks good to me.
>> > >
>> > > I want to play the devil's advocate for a second and ask do we want to
>> > keep
>> > > the older method with the extra parameters vs. deprecation?
>> > >
>> > > Although ATM I can't think of a good reason to keep the old method
>> with
>> > the
>> > > extra parameters.
>> > >
>> > > Thanks,
>> > > Bill
>> > >
>> > > On Tue, Dec 5, 2017 at 5:48 AM, Ted Yu <yu...@gmail.com> wrote:
>> > >
>> > > > Fine by me.
>> > > >
>> > > > On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
>> > > > panuwat.anawatmongkhon@gmail.com> wrote:
>> > > >
>> > > > > Thank you, Matthias.
>> > > > >
>> > > > > Ted,
>> > > > > How about this.
>> > > > >
>> > > > > String globalTopicName = "testGlobalTopic";
>> > > > > String globalStoreName = "testAddGlobalStore";
>> > > > > final StreamsBuilder builder = new StreamsBuilder();
>> > > > > final KeyValueStoreBuilder globalStoreBuilder =
>> > > > > EasyMock.createNiceMock(KeyValueStoreBuilder.class);
>> > > > >
>> > EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
>> > > > > anyTimes();
>> > > > > EasyMock.replay(globalStoreBuilder);
>> > > > > builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
>> > > > > ConsumedInternal(),new MockProcessorSupplier());
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <
>> > matthias@confluent.io
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Panuwat,
>> > > > > >
>> > > > > > Thanks a lot for the KIP!
>> > > > > >
>> > > > > > Just one nit: `does not follow provide a good` -> spelling:
>> remove
>> > > > > > `follow` ?
>> > > > > >
>> > > > > > Otherwise, looks good to me.
>> > > > > >
>> > > > > >
>> > > > > > -Matthias
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On 12/4/17 10:49 AM, Ted Yu wrote:
>> > > > > > > Looks like you're implying logic similar to this:
>> > > > > > >
>> > > > > > > public synchronized <K, V> GlobalKTable<K, V>
>> > globalTable(final
>> > > > > > String
>> > > > > > > topic,
>> > > > > > >
>> > > > > > >
>> > final
>> > > > > > > Consumed<K, V> consumed) {
>> > > > > > >
>> > > > > > >
>> > > > > > > StreamsBuilder is returned instead of GlobalKTable.
>> > > > > > >
>> > > > > > >
>> > > > > > > Can you add code snippet showing how the new API is used ?
>> > > > > > >
>> > > > > > >
>> > > > > > > On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
>> > > > > > > panuwat.anawatmongkhon@gmail.com> wrote:
>> > > > > > >
>> > > > > > >> What i am thinking right now is using the same approach as
>> > > > > > >> org.apache.kafka.streams.kstream.internals.
>> > > InternalStreamsBuilder#
>> > > > > > >> globalTable
>> > > > > > >>
>> > > > > > >> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com>
>> wrote:
>> > > > > > >>
>> > > > > > >>> Can you describe how sourceName is inferred based on the new
>> > API
>> > > ?
>> > > > > > >>>
>> > > > > > >>> Please fill out JIRA number.
>> > > > > > >>>
>> > > > > > >>> BTW here is the URL for the KIP:
>> > > > > > >>>
>> > > > > > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > > > > 233%3A+Simplify+
>> > > > > > >> StreamsBuilder%23addGlobalStore
>> > > > > > >>>
>> > > > > > >>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
>> > > > > > >>> panuwat.anawatmongkhon@gmail.com> wrote:
>> > > > > > >>>
>> > > > > > >>>> Hi all,
>> > > > > > >>>> I created a KIP.
>> > > > > > >>>> https://cwiki.apache.org/confluence/display/KAFKA/
>> > > > > KIP233%3A+Simplify+
>> > > > > > >>>> StreamsBuilder%23addGlobalStore
>> > > > > > >>>>
>> > > > > > >>>> Cheers,
>> > > > > > >>>> Benz
>> > > > > > >>>>
>> > > > > > >>>
>> > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Panuwat Anawatmongkhon <pa...@gmail.com>.
Yes, Matthias.
The object will be used togerther with function table and function stream.
I didn’t see how this will affect other part but if you do, please explain
more on how this will affect generated topic name.
Thank you
Panuwat
On Thu, 7 Dec 2560 at 00:01 Matthias Margush <ma...@gmail.com>
wrote:
> Hi.
>
> I wanted to double check. If I understand the proposal, it would replace
> the explicit name with a name that is dynamically generated using the
> AtomicInteger index. Would this affect the naming of any internally
> generated topics?
>
> On Wed, Dec 6, 2017 at 7:59 AM Panuwat Anawatmongkhon <
> panuwat.anawatmongkhon@gmail.com> wrote:
>
> > Thanks Bill.
> >
> > I can't think of reason to keep the old method too so if there is no
> > further discussion by tomorrow, I would like to start the vote thread.
> >
> > On Tue, Dec 5, 2017 at 10:38 PM, Bill Bejeck <bb...@gmail.com> wrote:
> >
> > > Hi Panuwat,
> > >
> > > Thanks for the KIP, overall looks good to me.
> > >
> > > I want to play the devil's advocate for a second and ask do we want to
> > keep
> > > the older method with the extra parameters vs. deprecation?
> > >
> > > Although ATM I can't think of a good reason to keep the old method with
> > the
> > > extra parameters.
> > >
> > > Thanks,
> > > Bill
> > >
> > > On Tue, Dec 5, 2017 at 5:48 AM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > Fine by me.
> > > >
> > > > On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
> > > > panuwat.anawatmongkhon@gmail.com> wrote:
> > > >
> > > > > Thank you, Matthias.
> > > > >
> > > > > Ted,
> > > > > How about this.
> > > > >
> > > > > String globalTopicName = "testGlobalTopic";
> > > > > String globalStoreName = "testAddGlobalStore";
> > > > > final StreamsBuilder builder = new StreamsBuilder();
> > > > > final KeyValueStoreBuilder globalStoreBuilder =
> > > > > EasyMock.createNiceMock(KeyValueStoreBuilder.class);
> > > > >
> > EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
> > > > > anyTimes();
> > > > > EasyMock.replay(globalStoreBuilder);
> > > > > builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
> > > > > ConsumedInternal(),new MockProcessorSupplier());
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <
> > matthias@confluent.io
> > > >
> > > > > wrote:
> > > > >
> > > > > > Panuwat,
> > > > > >
> > > > > > Thanks a lot for the KIP!
> > > > > >
> > > > > > Just one nit: `does not follow provide a good` -> spelling:
> remove
> > > > > > `follow` ?
> > > > > >
> > > > > > Otherwise, looks good to me.
> > > > > >
> > > > > >
> > > > > > -Matthias
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 12/4/17 10:49 AM, Ted Yu wrote:
> > > > > > > Looks like you're implying logic similar to this:
> > > > > > >
> > > > > > > public synchronized <K, V> GlobalKTable<K, V>
> > globalTable(final
> > > > > > String
> > > > > > > topic,
> > > > > > >
> > > > > > >
> > final
> > > > > > > Consumed<K, V> consumed) {
> > > > > > >
> > > > > > >
> > > > > > > StreamsBuilder is returned instead of GlobalKTable.
> > > > > > >
> > > > > > >
> > > > > > > Can you add code snippet showing how the new API is used ?
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
> > > > > > > panuwat.anawatmongkhon@gmail.com> wrote:
> > > > > > >
> > > > > > >> What i am thinking right now is using the same approach as
> > > > > > >> org.apache.kafka.streams.kstream.internals.
> > > InternalStreamsBuilder#
> > > > > > >> globalTable
> > > > > > >>
> > > > > > >> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com>
> wrote:
> > > > > > >>
> > > > > > >>> Can you describe how sourceName is inferred based on the new
> > API
> > > ?
> > > > > > >>>
> > > > > > >>> Please fill out JIRA number.
> > > > > > >>>
> > > > > > >>> BTW here is the URL for the KIP:
> > > > > > >>>
> > > > > > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 233%3A+Simplify+
> > > > > > >> StreamsBuilder%23addGlobalStore
> > > > > > >>>
> > > > > > >>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
> > > > > > >>> panuwat.anawatmongkhon@gmail.com> wrote:
> > > > > > >>>
> > > > > > >>>> Hi all,
> > > > > > >>>> I created a KIP.
> > > > > > >>>> https://cwiki.apache.org/confluence/display/KAFKA/
> > > > > KIP233%3A+Simplify+
> > > > > > >>>> StreamsBuilder%23addGlobalStore
> > > > > > >>>>
> > > > > > >>>> Cheers,
> > > > > > >>>> Benz
> > > > > > >>>>
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Matthias Margush <ma...@gmail.com>.
Hi.
I wanted to double check. If I understand the proposal, it would replace
the explicit name with a name that is dynamically generated using the
AtomicInteger index. Would this affect the naming of any internally
generated topics?
On Wed, Dec 6, 2017 at 7:59 AM Panuwat Anawatmongkhon <
panuwat.anawatmongkhon@gmail.com> wrote:
> Thanks Bill.
>
> I can't think of reason to keep the old method too so if there is no
> further discussion by tomorrow, I would like to start the vote thread.
>
> On Tue, Dec 5, 2017 at 10:38 PM, Bill Bejeck <bb...@gmail.com> wrote:
>
> > Hi Panuwat,
> >
> > Thanks for the KIP, overall looks good to me.
> >
> > I want to play the devil's advocate for a second and ask do we want to
> keep
> > the older method with the extra parameters vs. deprecation?
> >
> > Although ATM I can't think of a good reason to keep the old method with
> the
> > extra parameters.
> >
> > Thanks,
> > Bill
> >
> > On Tue, Dec 5, 2017 at 5:48 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Fine by me.
> > >
> > > On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
> > > panuwat.anawatmongkhon@gmail.com> wrote:
> > >
> > > > Thank you, Matthias.
> > > >
> > > > Ted,
> > > > How about this.
> > > >
> > > > String globalTopicName = "testGlobalTopic";
> > > > String globalStoreName = "testAddGlobalStore";
> > > > final StreamsBuilder builder = new StreamsBuilder();
> > > > final KeyValueStoreBuilder globalStoreBuilder =
> > > > EasyMock.createNiceMock(KeyValueStoreBuilder.class);
> > > >
> EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
> > > > anyTimes();
> > > > EasyMock.replay(globalStoreBuilder);
> > > > builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
> > > > ConsumedInternal(),new MockProcessorSupplier());
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <
> matthias@confluent.io
> > >
> > > > wrote:
> > > >
> > > > > Panuwat,
> > > > >
> > > > > Thanks a lot for the KIP!
> > > > >
> > > > > Just one nit: `does not follow provide a good` -> spelling: remove
> > > > > `follow` ?
> > > > >
> > > > > Otherwise, looks good to me.
> > > > >
> > > > >
> > > > > -Matthias
> > > > >
> > > > >
> > > > >
> > > > > On 12/4/17 10:49 AM, Ted Yu wrote:
> > > > > > Looks like you're implying logic similar to this:
> > > > > >
> > > > > > public synchronized <K, V> GlobalKTable<K, V>
> globalTable(final
> > > > > String
> > > > > > topic,
> > > > > >
> > > > > >
> final
> > > > > > Consumed<K, V> consumed) {
> > > > > >
> > > > > >
> > > > > > StreamsBuilder is returned instead of GlobalKTable.
> > > > > >
> > > > > >
> > > > > > Can you add code snippet showing how the new API is used ?
> > > > > >
> > > > > >
> > > > > > On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
> > > > > > panuwat.anawatmongkhon@gmail.com> wrote:
> > > > > >
> > > > > >> What i am thinking right now is using the same approach as
> > > > > >> org.apache.kafka.streams.kstream.internals.
> > InternalStreamsBuilder#
> > > > > >> globalTable
> > > > > >>
> > > > > >> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com> wrote:
> > > > > >>
> > > > > >>> Can you describe how sourceName is inferred based on the new
> API
> > ?
> > > > > >>>
> > > > > >>> Please fill out JIRA number.
> > > > > >>>
> > > > > >>> BTW here is the URL for the KIP:
> > > > > >>>
> > > > > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 233%3A+Simplify+
> > > > > >> StreamsBuilder%23addGlobalStore
> > > > > >>>
> > > > > >>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
> > > > > >>> panuwat.anawatmongkhon@gmail.com> wrote:
> > > > > >>>
> > > > > >>>> Hi all,
> > > > > >>>> I created a KIP.
> > > > > >>>> https://cwiki.apache.org/confluence/display/KAFKA/
> > > > KIP233%3A+Simplify+
> > > > > >>>> StreamsBuilder%23addGlobalStore
> > > > > >>>>
> > > > > >>>> Cheers,
> > > > > >>>> Benz
> > > > > >>>>
> > > > > >>>
> > > > > >>
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Panuwat Anawatmongkhon <pa...@gmail.com>.
Thanks Bill.
I can't think of reason to keep the old method too so if there is no
further discussion by tomorrow, I would like to start the vote thread.
On Tue, Dec 5, 2017 at 10:38 PM, Bill Bejeck <bb...@gmail.com> wrote:
> Hi Panuwat,
>
> Thanks for the KIP, overall looks good to me.
>
> I want to play the devil's advocate for a second and ask do we want to keep
> the older method with the extra parameters vs. deprecation?
>
> Although ATM I can't think of a good reason to keep the old method with the
> extra parameters.
>
> Thanks,
> Bill
>
> On Tue, Dec 5, 2017 at 5:48 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > Fine by me.
> >
> > On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
> > panuwat.anawatmongkhon@gmail.com> wrote:
> >
> > > Thank you, Matthias.
> > >
> > > Ted,
> > > How about this.
> > >
> > > String globalTopicName = "testGlobalTopic";
> > > String globalStoreName = "testAddGlobalStore";
> > > final StreamsBuilder builder = new StreamsBuilder();
> > > final KeyValueStoreBuilder globalStoreBuilder =
> > > EasyMock.createNiceMock(KeyValueStoreBuilder.class);
> > > EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
> > > anyTimes();
> > > EasyMock.replay(globalStoreBuilder);
> > > builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
> > > ConsumedInternal(),new MockProcessorSupplier());
> > >
> > >
> > >
> > >
> > > On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <matthias@confluent.io
> >
> > > wrote:
> > >
> > > > Panuwat,
> > > >
> > > > Thanks a lot for the KIP!
> > > >
> > > > Just one nit: `does not follow provide a good` -> spelling: remove
> > > > `follow` ?
> > > >
> > > > Otherwise, looks good to me.
> > > >
> > > >
> > > > -Matthias
> > > >
> > > >
> > > >
> > > > On 12/4/17 10:49 AM, Ted Yu wrote:
> > > > > Looks like you're implying logic similar to this:
> > > > >
> > > > > public synchronized <K, V> GlobalKTable<K, V> globalTable(final
> > > > String
> > > > > topic,
> > > > >
> > > > > final
> > > > > Consumed<K, V> consumed) {
> > > > >
> > > > >
> > > > > StreamsBuilder is returned instead of GlobalKTable.
> > > > >
> > > > >
> > > > > Can you add code snippet showing how the new API is used ?
> > > > >
> > > > >
> > > > > On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
> > > > > panuwat.anawatmongkhon@gmail.com> wrote:
> > > > >
> > > > >> What i am thinking right now is using the same approach as
> > > > >> org.apache.kafka.streams.kstream.internals.
> InternalStreamsBuilder#
> > > > >> globalTable
> > > > >>
> > > > >> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com> wrote:
> > > > >>
> > > > >>> Can you describe how sourceName is inferred based on the new API
> ?
> > > > >>>
> > > > >>> Please fill out JIRA number.
> > > > >>>
> > > > >>> BTW here is the URL for the KIP:
> > > > >>>
> > > > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 233%3A+Simplify+
> > > > >> StreamsBuilder%23addGlobalStore
> > > > >>>
> > > > >>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
> > > > >>> panuwat.anawatmongkhon@gmail.com> wrote:
> > > > >>>
> > > > >>>> Hi all,
> > > > >>>> I created a KIP.
> > > > >>>> https://cwiki.apache.org/confluence/display/KAFKA/
> > > KIP233%3A+Simplify+
> > > > >>>> StreamsBuilder%23addGlobalStore
> > > > >>>>
> > > > >>>> Cheers,
> > > > >>>> Benz
> > > > >>>>
> > > > >>>
> > > > >>
> > > > >
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Bill Bejeck <bb...@gmail.com>.
Hi Panuwat,
Thanks for the KIP, overall looks good to me.
I want to play the devil's advocate for a second and ask do we want to keep
the older method with the extra parameters vs. deprecation?
Although ATM I can't think of a good reason to keep the old method with the
extra parameters.
Thanks,
Bill
On Tue, Dec 5, 2017 at 5:48 AM, Ted Yu <yu...@gmail.com> wrote:
> Fine by me.
>
> On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
> panuwat.anawatmongkhon@gmail.com> wrote:
>
> > Thank you, Matthias.
> >
> > Ted,
> > How about this.
> >
> > String globalTopicName = "testGlobalTopic";
> > String globalStoreName = "testAddGlobalStore";
> > final StreamsBuilder builder = new StreamsBuilder();
> > final KeyValueStoreBuilder globalStoreBuilder =
> > EasyMock.createNiceMock(KeyValueStoreBuilder.class);
> > EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
> > anyTimes();
> > EasyMock.replay(globalStoreBuilder);
> > builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
> > ConsumedInternal(),new MockProcessorSupplier());
> >
> >
> >
> >
> > On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <ma...@confluent.io>
> > wrote:
> >
> > > Panuwat,
> > >
> > > Thanks a lot for the KIP!
> > >
> > > Just one nit: `does not follow provide a good` -> spelling: remove
> > > `follow` ?
> > >
> > > Otherwise, looks good to me.
> > >
> > >
> > > -Matthias
> > >
> > >
> > >
> > > On 12/4/17 10:49 AM, Ted Yu wrote:
> > > > Looks like you're implying logic similar to this:
> > > >
> > > > public synchronized <K, V> GlobalKTable<K, V> globalTable(final
> > > String
> > > > topic,
> > > >
> > > > final
> > > > Consumed<K, V> consumed) {
> > > >
> > > >
> > > > StreamsBuilder is returned instead of GlobalKTable.
> > > >
> > > >
> > > > Can you add code snippet showing how the new API is used ?
> > > >
> > > >
> > > > On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
> > > > panuwat.anawatmongkhon@gmail.com> wrote:
> > > >
> > > >> What i am thinking right now is using the same approach as
> > > >> org.apache.kafka.streams.kstream.internals.InternalStreamsBuilder#
> > > >> globalTable
> > > >>
> > > >> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com> wrote:
> > > >>
> > > >>> Can you describe how sourceName is inferred based on the new API ?
> > > >>>
> > > >>> Please fill out JIRA number.
> > > >>>
> > > >>> BTW here is the URL for the KIP:
> > > >>>
> > > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 233%3A+Simplify+
> > > >> StreamsBuilder%23addGlobalStore
> > > >>>
> > > >>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
> > > >>> panuwat.anawatmongkhon@gmail.com> wrote:
> > > >>>
> > > >>>> Hi all,
> > > >>>> I created a KIP.
> > > >>>> https://cwiki.apache.org/confluence/display/KAFKA/
> > KIP233%3A+Simplify+
> > > >>>> StreamsBuilder%23addGlobalStore
> > > >>>>
> > > >>>> Cheers,
> > > >>>> Benz
> > > >>>>
> > > >>>
> > > >>
> > > >
> > >
> > >
> >
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Ted Yu <yu...@gmail.com>.
Fine by me.
On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
panuwat.anawatmongkhon@gmail.com> wrote:
> Thank you, Matthias.
>
> Ted,
> How about this.
>
> String globalTopicName = "testGlobalTopic";
> String globalStoreName = "testAddGlobalStore";
> final StreamsBuilder builder = new StreamsBuilder();
> final KeyValueStoreBuilder globalStoreBuilder =
> EasyMock.createNiceMock(KeyValueStoreBuilder.class);
> EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
> anyTimes();
> EasyMock.replay(globalStoreBuilder);
> builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
> ConsumedInternal(),new MockProcessorSupplier());
>
>
>
>
> On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <ma...@confluent.io>
> wrote:
>
> > Panuwat,
> >
> > Thanks a lot for the KIP!
> >
> > Just one nit: `does not follow provide a good` -> spelling: remove
> > `follow` ?
> >
> > Otherwise, looks good to me.
> >
> >
> > -Matthias
> >
> >
> >
> > On 12/4/17 10:49 AM, Ted Yu wrote:
> > > Looks like you're implying logic similar to this:
> > >
> > > public synchronized <K, V> GlobalKTable<K, V> globalTable(final
> > String
> > > topic,
> > >
> > > final
> > > Consumed<K, V> consumed) {
> > >
> > >
> > > StreamsBuilder is returned instead of GlobalKTable.
> > >
> > >
> > > Can you add code snippet showing how the new API is used ?
> > >
> > >
> > > On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
> > > panuwat.anawatmongkhon@gmail.com> wrote:
> > >
> > >> What i am thinking right now is using the same approach as
> > >> org.apache.kafka.streams.kstream.internals.InternalStreamsBuilder#
> > >> globalTable
> > >>
> > >> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com> wrote:
> > >>
> > >>> Can you describe how sourceName is inferred based on the new API ?
> > >>>
> > >>> Please fill out JIRA number.
> > >>>
> > >>> BTW here is the URL for the KIP:
> > >>>
> > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 233%3A+Simplify+
> > >> StreamsBuilder%23addGlobalStore
> > >>>
> > >>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
> > >>> panuwat.anawatmongkhon@gmail.com> wrote:
> > >>>
> > >>>> Hi all,
> > >>>> I created a KIP.
> > >>>> https://cwiki.apache.org/confluence/display/KAFKA/
> KIP233%3A+Simplify+
> > >>>> StreamsBuilder%23addGlobalStore
> > >>>>
> > >>>> Cheers,
> > >>>> Benz
> > >>>>
> > >>>
> > >>
> > >
> >
> >
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Panuwat Anawatmongkhon <pa...@gmail.com>.
Thank you, Matthias.
Ted,
How about this.
String globalTopicName = "testGlobalTopic";
String globalStoreName = "testAddGlobalStore";
final StreamsBuilder builder = new StreamsBuilder();
final KeyValueStoreBuilder globalStoreBuilder =
EasyMock.createNiceMock(KeyValueStoreBuilder.class);
EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).anyTimes();
EasyMock.replay(globalStoreBuilder);
builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
ConsumedInternal(),new MockProcessorSupplier());
On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <ma...@confluent.io>
wrote:
> Panuwat,
>
> Thanks a lot for the KIP!
>
> Just one nit: `does not follow provide a good` -> spelling: remove
> `follow` ?
>
> Otherwise, looks good to me.
>
>
> -Matthias
>
>
>
> On 12/4/17 10:49 AM, Ted Yu wrote:
> > Looks like you're implying logic similar to this:
> >
> > public synchronized <K, V> GlobalKTable<K, V> globalTable(final
> String
> > topic,
> >
> > final
> > Consumed<K, V> consumed) {
> >
> >
> > StreamsBuilder is returned instead of GlobalKTable.
> >
> >
> > Can you add code snippet showing how the new API is used ?
> >
> >
> > On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
> > panuwat.anawatmongkhon@gmail.com> wrote:
> >
> >> What i am thinking right now is using the same approach as
> >> org.apache.kafka.streams.kstream.internals.InternalStreamsBuilder#
> >> globalTable
> >>
> >> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com> wrote:
> >>
> >>> Can you describe how sourceName is inferred based on the new API ?
> >>>
> >>> Please fill out JIRA number.
> >>>
> >>> BTW here is the URL for the KIP:
> >>>
> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-233%3A+Simplify+
> >> StreamsBuilder%23addGlobalStore
> >>>
> >>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
> >>> panuwat.anawatmongkhon@gmail.com> wrote:
> >>>
> >>>> Hi all,
> >>>> I created a KIP.
> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP233%3A+Simplify+
> >>>> StreamsBuilder%23addGlobalStore
> >>>>
> >>>> Cheers,
> >>>> Benz
> >>>>
> >>>
> >>
> >
>
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by "Matthias J. Sax" <ma...@confluent.io>.
Panuwat,
Thanks a lot for the KIP!
Just one nit: `does not follow provide a good` -> spelling: remove
`follow` ?
Otherwise, looks good to me.
-Matthias
On 12/4/17 10:49 AM, Ted Yu wrote:
> Looks like you're implying logic similar to this:
>
> public synchronized <K, V> GlobalKTable<K, V> globalTable(final String
> topic,
>
> final
> Consumed<K, V> consumed) {
>
>
> StreamsBuilder is returned instead of GlobalKTable.
>
>
> Can you add code snippet showing how the new API is used ?
>
>
> On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
> panuwat.anawatmongkhon@gmail.com> wrote:
>
>> What i am thinking right now is using the same approach as
>> org.apache.kafka.streams.kstream.internals.InternalStreamsBuilder#
>> globalTable
>>
>> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com> wrote:
>>
>>> Can you describe how sourceName is inferred based on the new API ?
>>>
>>> Please fill out JIRA number.
>>>
>>> BTW here is the URL for the KIP:
>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-233%3A+Simplify+
>> StreamsBuilder%23addGlobalStore
>>>
>>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
>>> panuwat.anawatmongkhon@gmail.com> wrote:
>>>
>>>> Hi all,
>>>> I created a KIP.
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP233%3A+Simplify+
>>>> StreamsBuilder%23addGlobalStore
>>>>
>>>> Cheers,
>>>> Benz
>>>>
>>>
>>
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Ted Yu <yu...@gmail.com>.
Looks like you're implying logic similar to this:
public synchronized <K, V> GlobalKTable<K, V> globalTable(final String
topic,
final
Consumed<K, V> consumed) {
StreamsBuilder is returned instead of GlobalKTable.
Can you add code snippet showing how the new API is used ?
On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon <
panuwat.anawatmongkhon@gmail.com> wrote:
> What i am thinking right now is using the same approach as
> org.apache.kafka.streams.kstream.internals.InternalStreamsBuilder#
> globalTable
>
> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com> wrote:
>
> > Can you describe how sourceName is inferred based on the new API ?
> >
> > Please fill out JIRA number.
> >
> > BTW here is the URL for the KIP:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-233%3A+Simplify+
> StreamsBuilder%23addGlobalStore
> >
> > On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
> > panuwat.anawatmongkhon@gmail.com> wrote:
> >
> > > Hi all,
> > > I created a KIP.
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP233%3A+Simplify+
> > > StreamsBuilder%23addGlobalStore
> > >
> > > Cheers,
> > > Benz
> > >
> >
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Panuwat Anawatmongkhon <pa...@gmail.com>.
What i am thinking right now is using the same approach as
org.apache.kafka.streams.kstream.internals.InternalStreamsBuilder#globalTable
On Mon, 4 Dec 2560 at 23:10 Ted Yu <yu...@gmail.com> wrote:
> Can you describe how sourceName is inferred based on the new API ?
>
> Please fill out JIRA number.
>
> BTW here is the URL for the KIP:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-233%3A+Simplify+StreamsBuilder%23addGlobalStore
>
> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
> panuwat.anawatmongkhon@gmail.com> wrote:
>
> > Hi all,
> > I created a KIP.
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP233%3A+Simplify+
> > StreamsBuilder%23addGlobalStore
> >
> > Cheers,
> > Benz
> >
>
Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Posted by Ted Yu <yu...@gmail.com>.
Can you describe how sourceName is inferred based on the new API ?
Please fill out JIRA number.
BTW here is the URL for the KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-233%3A+Simplify+StreamsBuilder%23addGlobalStore
On Mon, Dec 4, 2017 at 7:39 AM, Panuwat Anawatmongkhon <
panuwat.anawatmongkhon@gmail.com> wrote:
> Hi all,
> I created a KIP.
> https://cwiki.apache.org/confluence/display/KAFKA/KIP233%3A+Simplify+
> StreamsBuilder%23addGlobalStore
>
> Cheers,
> Benz
>