You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Hao Li <hl...@confluent.io.INVALID> on 2022/03/24 05:25:07 UTC

[VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Hi all,

I'd like to start a vote on Kafka Streams KIP-825:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Posted by "Matthias J. Sax" <mj...@apache.org>.
Thanks for the update Hao.

Seems it's the best way forward. As we don't add it to 
`ReadOnlySessionStore` it's not exposed via IQ and thus the impact seems 
to be well contained.

No concerns from my side.

-Matthias

On 6/29/22 11:51 AM, Hao Li wrote:
> Hi Dev Team,
> 
> Just want to inform you here that we made an update to this KIP. To
> implement emit final in Session Window aggregation, we augment the
> `SessionStore` by adding an
> public API to range query from earliest session end time to latest session
> end time. API is like this:
> 
> ```
> public interface SessionStore<K, AGG> extends StateStore,
> ReadOnlySessionStore<K, AGG> {
> 
>      /**
>       * Return all the session window entries that ends between the
> specified range (both ends are inclusive).
>       * This function would be used to retrieve all closed and immutable
> windows.
>       *
>       * @param earliestSessionEndTime earliest session end time to search
> from, inclusive
>       * @param latestSessionEndTime latest session end time to search to,
> inclusive
>       */
>      default KeyValueIterator<Windowed<K>, AGG> findSessions(final long
> earliestSessionEndTime,
>                                                              final long
> latestSessionEndTime)
> {
>          throw new UnsupportedOperationException(
>                  "This API is not supported by this implementation of
> SessionStore.");
>      }
> 
>      // other existing functions
> }
> ```
> 
> You can take a look at the KIP for more information. Thanks Guozhang for
> adding it to the KIP.
> 
> Thanks,
> Hao
> 
> On Thu, Mar 24, 2022 at 4:17 PM Hao Li <hl...@confluent.io> wrote:
> 
>> The vote passed with 5 binding votes from John, Guozhang, Bruno, Matthias
>> and Bill.
>>
>> Thanks all for the feedback and vote.
>>
>> Hao
>>
>> On Thu, Mar 24, 2022 at 2:20 PM Bill Bejeck <bb...@gmail.com> wrote:
>>
>>> Thanks for KIP Hao!
>>>
>>> Glad to see we settled on option 1
>>>
>>> +1(binding)
>>>
>>> On Thu, Mar 24, 2022 at 5:13 PM Matthias J. Sax <mj...@apache.org> wrote:
>>>
>>>> +1 (binding)
>>>>
>>>>
>>>> On 3/24/22 1:52 PM, Hao Li wrote:
>>>>> I hit reply on my phone in the mail app and changed the title and text
>>>>> hoping it will start a new thread. Apparently it doesn't work...
>>>>>
>>>>> On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna <ca...@apache.org>
>>>> wrote:
>>>>>
>>>>>> Hi Hao,
>>>>>>
>>>>>> Actually, this is the VOTE thread. Do you use GMail? Sometimes it is
>>> a
>>>>>> bit weird how it shows e-mails in threads.
>>>>>>
>>>>>> Anyways, thank you for the KIP and your patience!
>>>>>>
>>>>>> +1 (binding)
>>>>>>
>>>>>> Best,
>>>>>> Bruno
>>>>>>
>>>>>> On 24.03.22 17:36, Hao Li wrote:
>>>>>>> Thanks John and Guozhang. Didn't realize I used this discussion
>>> thread
>>>>>> for
>>>>>>> voting. Let me start a new thread for voting. Will fix the KIP.
>>>>>>>
>>>>>>> On Thu, Mar 24, 2022 at 9:28 AM Guozhang Wang <wa...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 (binding).
>>>>>>>>
>>>>>>>> Thanks Hao!
>>>>>>>>
>>>>>>>>
>>>>>>>> Guozhang
>>>>>>>>
>>>>>>>> On Thu, Mar 24, 2022 at 9:20 AM John Roesler <vv...@apache.org>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks, Hao!
>>>>>>>>>
>>>>>>>>> I'm +1 (binding)
>>>>>>>>>
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I'd like to start a vote on Kafka Streams KIP-825:
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> -- Guozhang
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Thanks,
>> Hao
>>
> 
> 

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Posted by Hao Li <hl...@confluent.io.INVALID>.
Hi Dev Team,

Just want to inform you here that we made an update to this KIP. To
implement emit final in Session Window aggregation, we augment the
`SessionStore` by adding an
public API to range query from earliest session end time to latest session
end time. API is like this:

```
public interface SessionStore<K, AGG> extends StateStore,
ReadOnlySessionStore<K, AGG> {

    /**
     * Return all the session window entries that ends between the
specified range (both ends are inclusive).
     * This function would be used to retrieve all closed and immutable
windows.
     *
     * @param earliestSessionEndTime earliest session end time to search
from, inclusive
     * @param latestSessionEndTime latest session end time to search to,
inclusive
     */
    default KeyValueIterator<Windowed<K>, AGG> findSessions(final long
earliestSessionEndTime,
                                                            final long
latestSessionEndTime)
{
        throw new UnsupportedOperationException(
                "This API is not supported by this implementation of
SessionStore.");
    }

    // other existing functions
}
```

You can take a look at the KIP for more information. Thanks Guozhang for
adding it to the KIP.

Thanks,
Hao

On Thu, Mar 24, 2022 at 4:17 PM Hao Li <hl...@confluent.io> wrote:

> The vote passed with 5 binding votes from John, Guozhang, Bruno, Matthias
> and Bill.
>
> Thanks all for the feedback and vote.
>
> Hao
>
> On Thu, Mar 24, 2022 at 2:20 PM Bill Bejeck <bb...@gmail.com> wrote:
>
>> Thanks for KIP Hao!
>>
>> Glad to see we settled on option 1
>>
>> +1(binding)
>>
>> On Thu, Mar 24, 2022 at 5:13 PM Matthias J. Sax <mj...@apache.org> wrote:
>>
>> > +1 (binding)
>> >
>> >
>> > On 3/24/22 1:52 PM, Hao Li wrote:
>> > > I hit reply on my phone in the mail app and changed the title and text
>> > > hoping it will start a new thread. Apparently it doesn't work...
>> > >
>> > > On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna <ca...@apache.org>
>> > wrote:
>> > >
>> > >> Hi Hao,
>> > >>
>> > >> Actually, this is the VOTE thread. Do you use GMail? Sometimes it is
>> a
>> > >> bit weird how it shows e-mails in threads.
>> > >>
>> > >> Anyways, thank you for the KIP and your patience!
>> > >>
>> > >> +1 (binding)
>> > >>
>> > >> Best,
>> > >> Bruno
>> > >>
>> > >> On 24.03.22 17:36, Hao Li wrote:
>> > >>> Thanks John and Guozhang. Didn't realize I used this discussion
>> thread
>> > >> for
>> > >>> voting. Let me start a new thread for voting. Will fix the KIP.
>> > >>>
>> > >>> On Thu, Mar 24, 2022 at 9:28 AM Guozhang Wang <wa...@gmail.com>
>> > >> wrote:
>> > >>>
>> > >>>> +1 (binding).
>> > >>>>
>> > >>>> Thanks Hao!
>> > >>>>
>> > >>>>
>> > >>>> Guozhang
>> > >>>>
>> > >>>> On Thu, Mar 24, 2022 at 9:20 AM John Roesler <vv...@apache.org>
>> > >> wrote:
>> > >>>>
>> > >>>>> Thanks, Hao!
>> > >>>>>
>> > >>>>> I'm +1 (binding)
>> > >>>>>
>> > >>>>> -John
>> > >>>>>
>> > >>>>> On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
>> > >>>>>> Hi all,
>> > >>>>>>
>> > >>>>>> I'd like to start a vote on Kafka Streams KIP-825:
>> > >>>>>>
>> > >>>>>
>> > >>>>
>> > >>
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
>> > >>>>>
>> > >>>>>
>> > >>>>
>> > >>>> --
>> > >>>> -- Guozhang
>> > >>>>
>> > >>>
>> > >>>
>> > >>
>> > >
>> > >
>> >
>>
>
>
> --
> Thanks,
> Hao
>


-- 
Thanks,
Hao

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Posted by Hao Li <hl...@confluent.io.INVALID>.
The vote passed with 5 binding votes from John, Guozhang, Bruno, Matthias
and Bill.

Thanks all for the feedback and vote.

Hao

On Thu, Mar 24, 2022 at 2:20 PM Bill Bejeck <bb...@gmail.com> wrote:

> Thanks for KIP Hao!
>
> Glad to see we settled on option 1
>
> +1(binding)
>
> On Thu, Mar 24, 2022 at 5:13 PM Matthias J. Sax <mj...@apache.org> wrote:
>
> > +1 (binding)
> >
> >
> > On 3/24/22 1:52 PM, Hao Li wrote:
> > > I hit reply on my phone in the mail app and changed the title and text
> > > hoping it will start a new thread. Apparently it doesn't work...
> > >
> > > On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna <ca...@apache.org>
> > wrote:
> > >
> > >> Hi Hao,
> > >>
> > >> Actually, this is the VOTE thread. Do you use GMail? Sometimes it is a
> > >> bit weird how it shows e-mails in threads.
> > >>
> > >> Anyways, thank you for the KIP and your patience!
> > >>
> > >> +1 (binding)
> > >>
> > >> Best,
> > >> Bruno
> > >>
> > >> On 24.03.22 17:36, Hao Li wrote:
> > >>> Thanks John and Guozhang. Didn't realize I used this discussion
> thread
> > >> for
> > >>> voting. Let me start a new thread for voting. Will fix the KIP.
> > >>>
> > >>> On Thu, Mar 24, 2022 at 9:28 AM Guozhang Wang <wa...@gmail.com>
> > >> wrote:
> > >>>
> > >>>> +1 (binding).
> > >>>>
> > >>>> Thanks Hao!
> > >>>>
> > >>>>
> > >>>> Guozhang
> > >>>>
> > >>>> On Thu, Mar 24, 2022 at 9:20 AM John Roesler <vv...@apache.org>
> > >> wrote:
> > >>>>
> > >>>>> Thanks, Hao!
> > >>>>>
> > >>>>> I'm +1 (binding)
> > >>>>>
> > >>>>> -John
> > >>>>>
> > >>>>> On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
> > >>>>>> Hi all,
> > >>>>>>
> > >>>>>> I'd like to start a vote on Kafka Streams KIP-825:
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
> > >>>>>
> > >>>>>
> > >>>>
> > >>>> --
> > >>>> -- Guozhang
> > >>>>
> > >>>
> > >>>
> > >>
> > >
> > >
> >
>


-- 
Thanks,
Hao

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Posted by Bill Bejeck <bb...@gmail.com>.
Thanks for KIP Hao!

Glad to see we settled on option 1

+1(binding)

On Thu, Mar 24, 2022 at 5:13 PM Matthias J. Sax <mj...@apache.org> wrote:

> +1 (binding)
>
>
> On 3/24/22 1:52 PM, Hao Li wrote:
> > I hit reply on my phone in the mail app and changed the title and text
> > hoping it will start a new thread. Apparently it doesn't work...
> >
> > On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna <ca...@apache.org>
> wrote:
> >
> >> Hi Hao,
> >>
> >> Actually, this is the VOTE thread. Do you use GMail? Sometimes it is a
> >> bit weird how it shows e-mails in threads.
> >>
> >> Anyways, thank you for the KIP and your patience!
> >>
> >> +1 (binding)
> >>
> >> Best,
> >> Bruno
> >>
> >> On 24.03.22 17:36, Hao Li wrote:
> >>> Thanks John and Guozhang. Didn't realize I used this discussion thread
> >> for
> >>> voting. Let me start a new thread for voting. Will fix the KIP.
> >>>
> >>> On Thu, Mar 24, 2022 at 9:28 AM Guozhang Wang <wa...@gmail.com>
> >> wrote:
> >>>
> >>>> +1 (binding).
> >>>>
> >>>> Thanks Hao!
> >>>>
> >>>>
> >>>> Guozhang
> >>>>
> >>>> On Thu, Mar 24, 2022 at 9:20 AM John Roesler <vv...@apache.org>
> >> wrote:
> >>>>
> >>>>> Thanks, Hao!
> >>>>>
> >>>>> I'm +1 (binding)
> >>>>>
> >>>>> -John
> >>>>>
> >>>>> On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
> >>>>>> Hi all,
> >>>>>>
> >>>>>> I'd like to start a vote on Kafka Streams KIP-825:
> >>>>>>
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
> >>>>>
> >>>>>
> >>>>
> >>>> --
> >>>> -- Guozhang
> >>>>
> >>>
> >>>
> >>
> >
> >
>

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Posted by "Matthias J. Sax" <mj...@apache.org>.
+1 (binding)


On 3/24/22 1:52 PM, Hao Li wrote:
> I hit reply on my phone in the mail app and changed the title and text
> hoping it will start a new thread. Apparently it doesn't work...
> 
> On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna <ca...@apache.org> wrote:
> 
>> Hi Hao,
>>
>> Actually, this is the VOTE thread. Do you use GMail? Sometimes it is a
>> bit weird how it shows e-mails in threads.
>>
>> Anyways, thank you for the KIP and your patience!
>>
>> +1 (binding)
>>
>> Best,
>> Bruno
>>
>> On 24.03.22 17:36, Hao Li wrote:
>>> Thanks John and Guozhang. Didn't realize I used this discussion thread
>> for
>>> voting. Let me start a new thread for voting. Will fix the KIP.
>>>
>>> On Thu, Mar 24, 2022 at 9:28 AM Guozhang Wang <wa...@gmail.com>
>> wrote:
>>>
>>>> +1 (binding).
>>>>
>>>> Thanks Hao!
>>>>
>>>>
>>>> Guozhang
>>>>
>>>> On Thu, Mar 24, 2022 at 9:20 AM John Roesler <vv...@apache.org>
>> wrote:
>>>>
>>>>> Thanks, Hao!
>>>>>
>>>>> I'm +1 (binding)
>>>>>
>>>>> -John
>>>>>
>>>>> On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I'd like to start a vote on Kafka Streams KIP-825:
>>>>>>
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
>>>>>
>>>>>
>>>>
>>>> --
>>>> -- Guozhang
>>>>
>>>
>>>
>>
> 
> 

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Posted by Hao Li <hl...@confluent.io.INVALID>.
I hit reply on my phone in the mail app and changed the title and text
hoping it will start a new thread. Apparently it doesn't work...

On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna <ca...@apache.org> wrote:

> Hi Hao,
>
> Actually, this is the VOTE thread. Do you use GMail? Sometimes it is a
> bit weird how it shows e-mails in threads.
>
> Anyways, thank you for the KIP and your patience!
>
> +1 (binding)
>
> Best,
> Bruno
>
> On 24.03.22 17:36, Hao Li wrote:
> > Thanks John and Guozhang. Didn't realize I used this discussion thread
> for
> > voting. Let me start a new thread for voting. Will fix the KIP.
> >
> > On Thu, Mar 24, 2022 at 9:28 AM Guozhang Wang <wa...@gmail.com>
> wrote:
> >
> >> +1 (binding).
> >>
> >> Thanks Hao!
> >>
> >>
> >> Guozhang
> >>
> >> On Thu, Mar 24, 2022 at 9:20 AM John Roesler <vv...@apache.org>
> wrote:
> >>
> >>> Thanks, Hao!
> >>>
> >>> I'm +1 (binding)
> >>>
> >>> -John
> >>>
> >>> On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
> >>>> Hi all,
> >>>>
> >>>> I'd like to start a vote on Kafka Streams KIP-825:
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
> >>>
> >>>
> >>
> >> --
> >> -- Guozhang
> >>
> >
> >
>


-- 
Thanks,
Hao

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Posted by Bruno Cadonna <ca...@apache.org>.
Hi Hao,

Actually, this is the VOTE thread. Do you use GMail? Sometimes it is a 
bit weird how it shows e-mails in threads.

Anyways, thank you for the KIP and your patience!

+1 (binding)

Best,
Bruno

On 24.03.22 17:36, Hao Li wrote:
> Thanks John and Guozhang. Didn't realize I used this discussion thread for
> voting. Let me start a new thread for voting. Will fix the KIP.
> 
> On Thu, Mar 24, 2022 at 9:28 AM Guozhang Wang <wa...@gmail.com> wrote:
> 
>> +1 (binding).
>>
>> Thanks Hao!
>>
>>
>> Guozhang
>>
>> On Thu, Mar 24, 2022 at 9:20 AM John Roesler <vv...@apache.org> wrote:
>>
>>> Thanks, Hao!
>>>
>>> I'm +1 (binding)
>>>
>>> -John
>>>
>>> On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
>>>> Hi all,
>>>>
>>>> I'd like to start a vote on Kafka Streams KIP-825:
>>>>
>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
>>>
>>>
>>
>> --
>> -- Guozhang
>>
> 
> 

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Posted by Hao Li <hl...@confluent.io.INVALID>.
Thanks John and Guozhang. Didn't realize I used this discussion thread for
voting. Let me start a new thread for voting. Will fix the KIP.

On Thu, Mar 24, 2022 at 9:28 AM Guozhang Wang <wa...@gmail.com> wrote:

> +1 (binding).
>
> Thanks Hao!
>
>
> Guozhang
>
> On Thu, Mar 24, 2022 at 9:20 AM John Roesler <vv...@apache.org> wrote:
>
> > Thanks, Hao!
> >
> > I'm +1 (binding)
> >
> > -John
> >
> > On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
> > > Hi all,
> > >
> > > I'd like to start a vote on Kafka Streams KIP-825:
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
> >
> >
>
> --
> -- Guozhang
>


-- 
Thanks,
Hao

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Posted by Guozhang Wang <wa...@gmail.com>.
+1 (binding).

Thanks Hao!


Guozhang

On Thu, Mar 24, 2022 at 9:20 AM John Roesler <vv...@apache.org> wrote:

> Thanks, Hao!
>
> I'm +1 (binding)
>
> -John
>
> On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
> > Hi all,
> >
> > I'd like to start a vote on Kafka Streams KIP-825:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
>
>

-- 
-- Guozhang

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Posted by John Roesler <vv...@apache.org>.
Thanks, Hao!

I'm +1 (binding)

-John

On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
> Hi all,
> 
> I'd like to start a vote on Kafka Streams KIP-825:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced