You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Levani Kokhreidze <le...@gmail.com> on 2019/08/01 06:55:45 UTC

Re: [VOTE] KIP-221: Enhance KStream with Connecting Topic Creation and Repartition Hint

Thank you all!

The vote has been open for ~8 days. KIP has three binding votes (Bill, Guozhang, Matthias) and one non-binding (Sophie) so the KIP vote passes!
I’ll mark KIP as accepted and start working on it as soon as possible!

Regards,
Levani

> On Aug 1, 2019, at 2:37 AM, Matthias J. Sax <ma...@confluent.io> wrote:
> 
> +1 (binding)
> 
> On 7/31/19 8:36 AM, Guozhang Wang wrote:
>> Thanks for the update! +1 (binding).
>> 
>> On Tue, Jul 30, 2019 at 11:42 PM Levani Kokhreidze <le...@gmail.com>
>> wrote:
>> 
>>> Hello Guozhang,
>>> 
>>> Thanks for the feedback. That’s an interesting point. To be honest, I
>>> totally missed it. I wasn’t aware that there’s `groupBy` possibility on
>>> KTable.
>>> I don’t see any reasons why not to add same functionality to KTable
>>> interface.
>>> 
>>> I’ve updated the KIP:
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>> <
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>> 
>>> Please let me know if you have any other questions and/or concerns.
>>> 
>>> Regards,
>>> Levani
>>> 
>>>> On Jul 31, 2019, at 1:24 AM, Guozhang Wang <wa...@gmail.com> wrote:
>>>> 
>>>> Hello Levani,
>>>> 
>>>> Thanks for the KIP! Just got a quick question here about the scope: why
>>> do
>>>> we only want this for `KStream`, not `KTable#groupBy` for example?
>>>> 
>>>> 
>>>> Guozhang
>>>> 
>>>> 
>>>> On Tue, Jul 30, 2019 at 1:27 PM Bill Bejeck <bb...@gmail.com> wrote:
>>>> 
>>>>> Thanks for the KIP Levani.
>>>>> 
>>>>> +1 (binding)
>>>>> 
>>>>> -Bill
>>>>> 
>>>>> On Tue, Jul 30, 2019 at 3:37 PM Levani Kokhreidze <
>>> levani.codes@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> Hello,
>>>>>> 
>>>>>> Still waiting for feedback on this KIP.
>>>>>> Please let me know if you have any concerns and/or questions.
>>>>>> 
>>>>>> Regards,
>>>>>> Levani
>>>>>> 
>>>>>> 
>>>>>>> On Jul 24, 2019, at 8:20 PM, Sophie Blee-Goldman <sophie@confluent.io
>>>> 
>>>>>> wrote:
>>>>>>> 
>>>>>>> Looks good! Thanks Levani,
>>>>>>> 
>>>>>>> +1 (non-binding)
>>>>>>> 
>>>>>>> Sophie
>>>>>>> 
>>>>>>> On Tue, Jul 23, 2019 at 2:16 PM Levani Kokhreidze <
>>>>>> levani.codes@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hello,
>>>>>>>> 
>>>>>>>> I’d like to initialize voting on KIP-221:
>>>>>>>> 
>>>>>> 
>>>>> 
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>> <
>>>>>>>> 
>>>>>> 
>>>>> 
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>> 
>>>>>>>> If there’re any more concerns about the KIP, happy to discuss
>>> further.
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Levani
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> -- Guozhang
>>> 
>>> 
>> 
>

Re: [VOTE] KIP-221: Enhance KStream with Connecting Topic Creation and Repartition Hint

Posted by "Matthias J. Sax" <mj...@apache.org>.
While working on the follow up PR to deprecate `through()`, I realized
that we forgot to add `repartition()` to the Scala API.

I updated the KIP accordingly and my PR also include the update to the
Scala API (for adding `repartition()` and deprecating `through()`):
https://github.com/apache/kafka/pull/8679


Please let us know if there are any concerns.


-Matthias

On 5/15/20 10:27 AM, Matthias J. Sax wrote:
> Thanks Levani!
> 
> I took the liberty to update the KIP accordingly, because so far no
> concerns were raised about the change.
> 
> Also assigned the ticket to myself (let me know if you want to work on
> it instead). We should get the change into 2.6 release.
> 
> 
> -Matthias
> 
> 
> On 5/15/20 1:34 AM, Levani Kokhreidze wrote:
>> Hi Matthias,
>>
>> Thanks for your thoughts and sorry for taking so long to respond.
>> For me that makes total sense, so +1 from my side. 
>> I took the liberty of creating a ticket for it: https://issues.apache.org/jira/browse/KAFKA-10003 <https://issues.apache.org/jira/browse/KAFKA-10003>
>>
>> Regards,
>> Levani
>>
>>
>>> On May 12, 2020, at 8:30 PM, Guozhang Wang <wa...@gmail.com> wrote:
>>>
>>> Sounds fair to me; I think as a syntax sugar it is a good to have, but
>>> sometimes it was "enforced" to be used for repartitioning purposes.
>>>
>>> On Mon, May 11, 2020 at 7:08 PM Matthias J. Sax <mj...@apache.org> wrote:
>>>
>>>> As an afterthought to KIP-221, I am wondering if we should deprecate
>>>> `KStream#through()`?
>>>>
>>>> The reasoning is that I assume that most people don't want to manage
>>>> topics manually anyway and thus it might be good to guide users to use
>>>> repartition(). Furthermore, through() is really just syntactic sugar for
>>>> to() followed by builder.stream() (thus people don't really loose
>>>> functionality). So far, through() was very nice to have, especially with
>>>> PAPI integration in the DSL (users might need to do a manual
>>>> repartitioning before transform()) however this pattern should be
>>>> subsumed by repartition().
>>>>
>>>> Reducing the surface area of our API (instead of just enlarging it)
>>>> might be good.
>>>>
>>>> Thoughts?
>>>>
>>>>
>>>> -Matthias
>>>>
>>>> On 4/5/20 9:36 PM, John Roesler wrote:
>>>>> Thanks for the update, Levani!
>>>>> -John
>>>>>
>>>>> On Sat, Apr 4, 2020, at 04:36, Levani Kokhreidze wrote:
>>>>>> Hello,
>>>>>>
>>>>>> Small update regarding this KIP. As per John’s suggestion during the
>>>>>> code review
>>>>>> (https://github.com/apache/kafka/pull/7170#discussion_r392608571
>>>>>> <https://github.com/apache/kafka/pull/7170#discussion_r392608571>)
>>>>>> we’ve decided to remove KeyValueMapper overloads for the new
>>>>>> `repartition` operation for the first release of this feature.
>>>>>> Wiki page has been updated accordingly
>>>>>> (
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>> <
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>> )
>>>>>>
>>>>>> Regards,
>>>>>> Levani
>>>>>>
>>>>>>> On Aug 1, 2019, at 9:55 AM, Levani Kokhreidze <le...@gmail.com>
>>>> wrote:
>>>>>>>
>>>>>>> Thank you all!
>>>>>>>
>>>>>>> The vote has been open for ~8 days. KIP has three binding votes (Bill,
>>>> Guozhang, Matthias) and one non-binding (Sophie) so the KIP vote passes!
>>>>>>> I’ll mark KIP as accepted and start working on it as soon as possible!
>>>>>>>
>>>>>>> Regards,
>>>>>>> Levani
>>>>>>>
>>>>>>>> On Aug 1, 2019, at 2:37 AM, Matthias J. Sax <ma...@confluent.io>
>>>> wrote:
>>>>>>>>
>>>>>>>> +1 (binding)
>>>>>>>>
>>>>>>>> On 7/31/19 8:36 AM, Guozhang Wang wrote:
>>>>>>>>> Thanks for the update! +1 (binding).
>>>>>>>>>
>>>>>>>>> On Tue, Jul 30, 2019 at 11:42 PM Levani Kokhreidze <
>>>> levani.codes@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Guozhang,
>>>>>>>>>>
>>>>>>>>>> Thanks for the feedback. That’s an interesting point. To be honest,
>>>> I
>>>>>>>>>> totally missed it. I wasn’t aware that there’s `groupBy`
>>>> possibility on
>>>>>>>>>> KTable.
>>>>>>>>>> I don’t see any reasons why not to add same functionality to KTable
>>>>>>>>>> interface.
>>>>>>>>>>
>>>>>>>>>> I’ve updated the KIP:
>>>>>>>>>>
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>> <
>>>>>>>>>>
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>>>
>>>>>>>>>> Please let me know if you have any other questions and/or concerns.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Levani
>>>>>>>>>>
>>>>>>>>>>> On Jul 31, 2019, at 1:24 AM, Guozhang Wang <wa...@gmail.com>
>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hello Levani,
>>>>>>>>>>>
>>>>>>>>>>> Thanks for the KIP! Just got a quick question here about the
>>>> scope: why
>>>>>>>>>> do
>>>>>>>>>>> we only want this for `KStream`, not `KTable#groupBy` for example?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Guozhang
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jul 30, 2019 at 1:27 PM Bill Bejeck <bb...@gmail.com>
>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the KIP Levani.
>>>>>>>>>>>>
>>>>>>>>>>>> +1 (binding)
>>>>>>>>>>>>
>>>>>>>>>>>> -Bill
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jul 30, 2019 at 3:37 PM Levani Kokhreidze <
>>>>>>>>>> levani.codes@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Still waiting for feedback on this KIP.
>>>>>>>>>>>>> Please let me know if you have any concerns and/or questions.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Levani
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jul 24, 2019, at 8:20 PM, Sophie Blee-Goldman <
>>>> sophie@confluent.io
>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Looks good! Thanks Levani,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 (non-binding)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sophie
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jul 23, 2019 at 2:16 PM Levani Kokhreidze <
>>>>>>>>>>>>> levani.codes@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I’d like to initialize voting on KIP-221:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If there’re any more concerns about the KIP, happy to discuss
>>>>>>>>>> further.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Levani
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> -- Guozhang
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>> -- 
>>> -- Guozhang
>>
>>
>

Re: [VOTE] KIP-221: Enhance KStream with Connecting Topic Creation and Repartition Hint

Posted by "Matthias J. Sax" <mj...@apache.org>.
Thanks Levani!

I took the liberty to update the KIP accordingly, because so far no
concerns were raised about the change.

Also assigned the ticket to myself (let me know if you want to work on
it instead). We should get the change into 2.6 release.


-Matthias


On 5/15/20 1:34 AM, Levani Kokhreidze wrote:
> Hi Matthias,
> 
> Thanks for your thoughts and sorry for taking so long to respond.
> For me that makes total sense, so +1 from my side. 
> I took the liberty of creating a ticket for it: https://issues.apache.org/jira/browse/KAFKA-10003 <https://issues.apache.org/jira/browse/KAFKA-10003>
> 
> Regards,
> Levani
> 
> 
>> On May 12, 2020, at 8:30 PM, Guozhang Wang <wa...@gmail.com> wrote:
>>
>> Sounds fair to me; I think as a syntax sugar it is a good to have, but
>> sometimes it was "enforced" to be used for repartitioning purposes.
>>
>> On Mon, May 11, 2020 at 7:08 PM Matthias J. Sax <mj...@apache.org> wrote:
>>
>>> As an afterthought to KIP-221, I am wondering if we should deprecate
>>> `KStream#through()`?
>>>
>>> The reasoning is that I assume that most people don't want to manage
>>> topics manually anyway and thus it might be good to guide users to use
>>> repartition(). Furthermore, through() is really just syntactic sugar for
>>> to() followed by builder.stream() (thus people don't really loose
>>> functionality). So far, through() was very nice to have, especially with
>>> PAPI integration in the DSL (users might need to do a manual
>>> repartitioning before transform()) however this pattern should be
>>> subsumed by repartition().
>>>
>>> Reducing the surface area of our API (instead of just enlarging it)
>>> might be good.
>>>
>>> Thoughts?
>>>
>>>
>>> -Matthias
>>>
>>> On 4/5/20 9:36 PM, John Roesler wrote:
>>>> Thanks for the update, Levani!
>>>> -John
>>>>
>>>> On Sat, Apr 4, 2020, at 04:36, Levani Kokhreidze wrote:
>>>>> Hello,
>>>>>
>>>>> Small update regarding this KIP. As per John’s suggestion during the
>>>>> code review
>>>>> (https://github.com/apache/kafka/pull/7170#discussion_r392608571
>>>>> <https://github.com/apache/kafka/pull/7170#discussion_r392608571>)
>>>>> we’ve decided to remove KeyValueMapper overloads for the new
>>>>> `repartition` operation for the first release of this feature.
>>>>> Wiki page has been updated accordingly
>>>>> (
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>> <
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>> )
>>>>>
>>>>> Regards,
>>>>> Levani
>>>>>
>>>>>> On Aug 1, 2019, at 9:55 AM, Levani Kokhreidze <le...@gmail.com>
>>> wrote:
>>>>>>
>>>>>> Thank you all!
>>>>>>
>>>>>> The vote has been open for ~8 days. KIP has three binding votes (Bill,
>>> Guozhang, Matthias) and one non-binding (Sophie) so the KIP vote passes!
>>>>>> I’ll mark KIP as accepted and start working on it as soon as possible!
>>>>>>
>>>>>> Regards,
>>>>>> Levani
>>>>>>
>>>>>>> On Aug 1, 2019, at 2:37 AM, Matthias J. Sax <ma...@confluent.io>
>>> wrote:
>>>>>>>
>>>>>>> +1 (binding)
>>>>>>>
>>>>>>> On 7/31/19 8:36 AM, Guozhang Wang wrote:
>>>>>>>> Thanks for the update! +1 (binding).
>>>>>>>>
>>>>>>>> On Tue, Jul 30, 2019 at 11:42 PM Levani Kokhreidze <
>>> levani.codes@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hello Guozhang,
>>>>>>>>>
>>>>>>>>> Thanks for the feedback. That’s an interesting point. To be honest,
>>> I
>>>>>>>>> totally missed it. I wasn’t aware that there’s `groupBy`
>>> possibility on
>>>>>>>>> KTable.
>>>>>>>>> I don’t see any reasons why not to add same functionality to KTable
>>>>>>>>> interface.
>>>>>>>>>
>>>>>>>>> I’ve updated the KIP:
>>>>>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>> <
>>>>>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>>
>>>>>>>>> Please let me know if you have any other questions and/or concerns.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Levani
>>>>>>>>>
>>>>>>>>>> On Jul 31, 2019, at 1:24 AM, Guozhang Wang <wa...@gmail.com>
>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello Levani,
>>>>>>>>>>
>>>>>>>>>> Thanks for the KIP! Just got a quick question here about the
>>> scope: why
>>>>>>>>> do
>>>>>>>>>> we only want this for `KStream`, not `KTable#groupBy` for example?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Guozhang
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jul 30, 2019 at 1:27 PM Bill Bejeck <bb...@gmail.com>
>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks for the KIP Levani.
>>>>>>>>>>>
>>>>>>>>>>> +1 (binding)
>>>>>>>>>>>
>>>>>>>>>>> -Bill
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jul 30, 2019 at 3:37 PM Levani Kokhreidze <
>>>>>>>>> levani.codes@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello,
>>>>>>>>>>>>
>>>>>>>>>>>> Still waiting for feedback on this KIP.
>>>>>>>>>>>> Please let me know if you have any concerns and/or questions.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Levani
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On Jul 24, 2019, at 8:20 PM, Sophie Blee-Goldman <
>>> sophie@confluent.io
>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Looks good! Thanks Levani,
>>>>>>>>>>>>>
>>>>>>>>>>>>> +1 (non-binding)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sophie
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jul 23, 2019 at 2:16 PM Levani Kokhreidze <
>>>>>>>>>>>> levani.codes@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I’d like to initialize voting on KIP-221:
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If there’re any more concerns about the KIP, happy to discuss
>>>>>>>>> further.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Levani
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> -- Guozhang
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>>
>>
>> -- 
>> -- Guozhang
> 
>

Re: [VOTE] KIP-221: Enhance KStream with Connecting Topic Creation and Repartition Hint

Posted by Levani Kokhreidze <le...@gmail.com>.
Hi Matthias,

Thanks for your thoughts and sorry for taking so long to respond.
For me that makes total sense, so +1 from my side. 
I took the liberty of creating a ticket for it: https://issues.apache.org/jira/browse/KAFKA-10003 <https://issues.apache.org/jira/browse/KAFKA-10003>

Regards,
Levani


> On May 12, 2020, at 8:30 PM, Guozhang Wang <wa...@gmail.com> wrote:
> 
> Sounds fair to me; I think as a syntax sugar it is a good to have, but
> sometimes it was "enforced" to be used for repartitioning purposes.
> 
> On Mon, May 11, 2020 at 7:08 PM Matthias J. Sax <mj...@apache.org> wrote:
> 
>> As an afterthought to KIP-221, I am wondering if we should deprecate
>> `KStream#through()`?
>> 
>> The reasoning is that I assume that most people don't want to manage
>> topics manually anyway and thus it might be good to guide users to use
>> repartition(). Furthermore, through() is really just syntactic sugar for
>> to() followed by builder.stream() (thus people don't really loose
>> functionality). So far, through() was very nice to have, especially with
>> PAPI integration in the DSL (users might need to do a manual
>> repartitioning before transform()) however this pattern should be
>> subsumed by repartition().
>> 
>> Reducing the surface area of our API (instead of just enlarging it)
>> might be good.
>> 
>> Thoughts?
>> 
>> 
>> -Matthias
>> 
>> On 4/5/20 9:36 PM, John Roesler wrote:
>>> Thanks for the update, Levani!
>>> -John
>>> 
>>> On Sat, Apr 4, 2020, at 04:36, Levani Kokhreidze wrote:
>>>> Hello,
>>>> 
>>>> Small update regarding this KIP. As per John’s suggestion during the
>>>> code review
>>>> (https://github.com/apache/kafka/pull/7170#discussion_r392608571
>>>> <https://github.com/apache/kafka/pull/7170#discussion_r392608571>)
>>>> we’ve decided to remove KeyValueMapper overloads for the new
>>>> `repartition` operation for the first release of this feature.
>>>> Wiki page has been updated accordingly
>>>> (
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>> <
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>> )
>>>> 
>>>> Regards,
>>>> Levani
>>>> 
>>>>> On Aug 1, 2019, at 9:55 AM, Levani Kokhreidze <le...@gmail.com>
>> wrote:
>>>>> 
>>>>> Thank you all!
>>>>> 
>>>>> The vote has been open for ~8 days. KIP has three binding votes (Bill,
>> Guozhang, Matthias) and one non-binding (Sophie) so the KIP vote passes!
>>>>> I’ll mark KIP as accepted and start working on it as soon as possible!
>>>>> 
>>>>> Regards,
>>>>> Levani
>>>>> 
>>>>>> On Aug 1, 2019, at 2:37 AM, Matthias J. Sax <ma...@confluent.io>
>> wrote:
>>>>>> 
>>>>>> +1 (binding)
>>>>>> 
>>>>>> On 7/31/19 8:36 AM, Guozhang Wang wrote:
>>>>>>> Thanks for the update! +1 (binding).
>>>>>>> 
>>>>>>> On Tue, Jul 30, 2019 at 11:42 PM Levani Kokhreidze <
>> levani.codes@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hello Guozhang,
>>>>>>>> 
>>>>>>>> Thanks for the feedback. That’s an interesting point. To be honest,
>> I
>>>>>>>> totally missed it. I wasn’t aware that there’s `groupBy`
>> possibility on
>>>>>>>> KTable.
>>>>>>>> I don’t see any reasons why not to add same functionality to KTable
>>>>>>>> interface.
>>>>>>>> 
>>>>>>>> I’ve updated the KIP:
>>>>>>>> 
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>> <
>>>>>>>> 
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>> 
>>>>>>>> Please let me know if you have any other questions and/or concerns.
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Levani
>>>>>>>> 
>>>>>>>>> On Jul 31, 2019, at 1:24 AM, Guozhang Wang <wa...@gmail.com>
>> wrote:
>>>>>>>>> 
>>>>>>>>> Hello Levani,
>>>>>>>>> 
>>>>>>>>> Thanks for the KIP! Just got a quick question here about the
>> scope: why
>>>>>>>> do
>>>>>>>>> we only want this for `KStream`, not `KTable#groupBy` for example?
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Guozhang
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Tue, Jul 30, 2019 at 1:27 PM Bill Bejeck <bb...@gmail.com>
>> wrote:
>>>>>>>>> 
>>>>>>>>>> Thanks for the KIP Levani.
>>>>>>>>>> 
>>>>>>>>>> +1 (binding)
>>>>>>>>>> 
>>>>>>>>>> -Bill
>>>>>>>>>> 
>>>>>>>>>> On Tue, Jul 30, 2019 at 3:37 PM Levani Kokhreidze <
>>>>>>>> levani.codes@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hello,
>>>>>>>>>>> 
>>>>>>>>>>> Still waiting for feedback on this KIP.
>>>>>>>>>>> Please let me know if you have any concerns and/or questions.
>>>>>>>>>>> 
>>>>>>>>>>> Regards,
>>>>>>>>>>> Levani
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> On Jul 24, 2019, at 8:20 PM, Sophie Blee-Goldman <
>> sophie@confluent.io
>>>>>>>>> 
>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Looks good! Thanks Levani,
>>>>>>>>>>>> 
>>>>>>>>>>>> +1 (non-binding)
>>>>>>>>>>>> 
>>>>>>>>>>>> Sophie
>>>>>>>>>>>> 
>>>>>>>>>>>> On Tue, Jul 23, 2019 at 2:16 PM Levani Kokhreidze <
>>>>>>>>>>> levani.codes@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I’d like to initialize voting on KIP-221:
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>>>>> <
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> If there’re any more concerns about the KIP, happy to discuss
>>>>>>>> further.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Levani
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> -- Guozhang
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>> 
>> 
> 
> -- 
> -- Guozhang

Re: [VOTE] KIP-221: Enhance KStream with Connecting Topic Creation and Repartition Hint

Posted by Guozhang Wang <wa...@gmail.com>.
Sounds fair to me; I think as a syntax sugar it is a good to have, but
sometimes it was "enforced" to be used for repartitioning purposes.

On Mon, May 11, 2020 at 7:08 PM Matthias J. Sax <mj...@apache.org> wrote:

> As an afterthought to KIP-221, I am wondering if we should deprecate
> `KStream#through()`?
>
> The reasoning is that I assume that most people don't want to manage
> topics manually anyway and thus it might be good to guide users to use
> repartition(). Furthermore, through() is really just syntactic sugar for
> to() followed by builder.stream() (thus people don't really loose
> functionality). So far, through() was very nice to have, especially with
> PAPI integration in the DSL (users might need to do a manual
> repartitioning before transform()) however this pattern should be
> subsumed by repartition().
>
> Reducing the surface area of our API (instead of just enlarging it)
> might be good.
>
> Thoughts?
>
>
> -Matthias
>
> On 4/5/20 9:36 PM, John Roesler wrote:
> > Thanks for the update, Levani!
> > -John
> >
> > On Sat, Apr 4, 2020, at 04:36, Levani Kokhreidze wrote:
> >> Hello,
> >>
> >> Small update regarding this KIP. As per John’s suggestion during the
> >> code review
> >> (https://github.com/apache/kafka/pull/7170#discussion_r392608571
> >> <https://github.com/apache/kafka/pull/7170#discussion_r392608571>)
> >> we’ve decided to remove KeyValueMapper overloads for the new
> >> `repartition` operation for the first release of this feature.
> >> Wiki page has been updated accordingly
> >> (
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
> >)
> >>
> >> Regards,
> >> Levani
> >>
> >>> On Aug 1, 2019, at 9:55 AM, Levani Kokhreidze <le...@gmail.com>
> wrote:
> >>>
> >>> Thank you all!
> >>>
> >>> The vote has been open for ~8 days. KIP has three binding votes (Bill,
> Guozhang, Matthias) and one non-binding (Sophie) so the KIP vote passes!
> >>> I’ll mark KIP as accepted and start working on it as soon as possible!
> >>>
> >>> Regards,
> >>> Levani
> >>>
> >>>> On Aug 1, 2019, at 2:37 AM, Matthias J. Sax <ma...@confluent.io>
> wrote:
> >>>>
> >>>> +1 (binding)
> >>>>
> >>>> On 7/31/19 8:36 AM, Guozhang Wang wrote:
> >>>>> Thanks for the update! +1 (binding).
> >>>>>
> >>>>> On Tue, Jul 30, 2019 at 11:42 PM Levani Kokhreidze <
> levani.codes@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hello Guozhang,
> >>>>>>
> >>>>>> Thanks for the feedback. That’s an interesting point. To be honest,
> I
> >>>>>> totally missed it. I wasn’t aware that there’s `groupBy`
> possibility on
> >>>>>> KTable.
> >>>>>> I don’t see any reasons why not to add same functionality to KTable
> >>>>>> interface.
> >>>>>>
> >>>>>> I’ve updated the KIP:
> >>>>>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
> >>>>>> <
> >>>>>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
> >>>>>>>
> >>>>>> Please let me know if you have any other questions and/or concerns.
> >>>>>>
> >>>>>> Regards,
> >>>>>> Levani
> >>>>>>
> >>>>>>> On Jul 31, 2019, at 1:24 AM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >>>>>>>
> >>>>>>> Hello Levani,
> >>>>>>>
> >>>>>>> Thanks for the KIP! Just got a quick question here about the
> scope: why
> >>>>>> do
> >>>>>>> we only want this for `KStream`, not `KTable#groupBy` for example?
> >>>>>>>
> >>>>>>>
> >>>>>>> Guozhang
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Jul 30, 2019 at 1:27 PM Bill Bejeck <bb...@gmail.com>
> wrote:
> >>>>>>>
> >>>>>>>> Thanks for the KIP Levani.
> >>>>>>>>
> >>>>>>>> +1 (binding)
> >>>>>>>>
> >>>>>>>> -Bill
> >>>>>>>>
> >>>>>>>> On Tue, Jul 30, 2019 at 3:37 PM Levani Kokhreidze <
> >>>>>> levani.codes@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hello,
> >>>>>>>>>
> >>>>>>>>> Still waiting for feedback on this KIP.
> >>>>>>>>> Please let me know if you have any concerns and/or questions.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Levani
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> On Jul 24, 2019, at 8:20 PM, Sophie Blee-Goldman <
> sophie@confluent.io
> >>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Looks good! Thanks Levani,
> >>>>>>>>>>
> >>>>>>>>>> +1 (non-binding)
> >>>>>>>>>>
> >>>>>>>>>> Sophie
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jul 23, 2019 at 2:16 PM Levani Kokhreidze <
> >>>>>>>>> levani.codes@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hello,
> >>>>>>>>>>>
> >>>>>>>>>>> I’d like to initialize voting on KIP-221:
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
> >>>>>>>>>>> <
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
> >>>>>>>>>>>>
> >>>>>>>>>>> If there’re any more concerns about the KIP, happy to discuss
> >>>>>> further.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards,
> >>>>>>>>>>> Levani
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> -- Guozhang
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >>
>
>

-- 
-- Guozhang

Re: [VOTE] KIP-221: Enhance KStream with Connecting Topic Creation and Repartition Hint

Posted by "Matthias J. Sax" <mj...@apache.org>.
As an afterthought to KIP-221, I am wondering if we should deprecate
`KStream#through()`?

The reasoning is that I assume that most people don't want to manage
topics manually anyway and thus it might be good to guide users to use
repartition(). Furthermore, through() is really just syntactic sugar for
to() followed by builder.stream() (thus people don't really loose
functionality). So far, through() was very nice to have, especially with
PAPI integration in the DSL (users might need to do a manual
repartitioning before transform()) however this pattern should be
subsumed by repartition().

Reducing the surface area of our API (instead of just enlarging it)
might be good.

Thoughts?


-Matthias

On 4/5/20 9:36 PM, John Roesler wrote:
> Thanks for the update, Levani!
> -John
> 
> On Sat, Apr 4, 2020, at 04:36, Levani Kokhreidze wrote:
>> Hello,
>>
>> Small update regarding this KIP. As per John’s suggestion during the 
>> code review 
>> (https://github.com/apache/kafka/pull/7170#discussion_r392608571 
>> <https://github.com/apache/kafka/pull/7170#discussion_r392608571>) 
>> we’ve decided to remove KeyValueMapper overloads for the new 
>> `repartition` operation for the first release of this feature.
>> Wiki page has been updated accordingly 
>> (https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint <https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint>)
>>
>> Regards,
>> Levani
>>
>>> On Aug 1, 2019, at 9:55 AM, Levani Kokhreidze <le...@gmail.com> wrote:
>>>
>>> Thank you all!
>>>
>>> The vote has been open for ~8 days. KIP has three binding votes (Bill, Guozhang, Matthias) and one non-binding (Sophie) so the KIP vote passes!
>>> I’ll mark KIP as accepted and start working on it as soon as possible!
>>>
>>> Regards,
>>> Levani
>>>
>>>> On Aug 1, 2019, at 2:37 AM, Matthias J. Sax <ma...@confluent.io> wrote:
>>>>
>>>> +1 (binding)
>>>>
>>>> On 7/31/19 8:36 AM, Guozhang Wang wrote:
>>>>> Thanks for the update! +1 (binding).
>>>>>
>>>>> On Tue, Jul 30, 2019 at 11:42 PM Levani Kokhreidze <le...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Guozhang,
>>>>>>
>>>>>> Thanks for the feedback. That’s an interesting point. To be honest, I
>>>>>> totally missed it. I wasn’t aware that there’s `groupBy` possibility on
>>>>>> KTable.
>>>>>> I don’t see any reasons why not to add same functionality to KTable
>>>>>> interface.
>>>>>>
>>>>>> I’ve updated the KIP:
>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>> <
>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>
>>>>>> Please let me know if you have any other questions and/or concerns.
>>>>>>
>>>>>> Regards,
>>>>>> Levani
>>>>>>
>>>>>>> On Jul 31, 2019, at 1:24 AM, Guozhang Wang <wa...@gmail.com> wrote:
>>>>>>>
>>>>>>> Hello Levani,
>>>>>>>
>>>>>>> Thanks for the KIP! Just got a quick question here about the scope: why
>>>>>> do
>>>>>>> we only want this for `KStream`, not `KTable#groupBy` for example?
>>>>>>>
>>>>>>>
>>>>>>> Guozhang
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jul 30, 2019 at 1:27 PM Bill Bejeck <bb...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Thanks for the KIP Levani.
>>>>>>>>
>>>>>>>> +1 (binding)
>>>>>>>>
>>>>>>>> -Bill
>>>>>>>>
>>>>>>>> On Tue, Jul 30, 2019 at 3:37 PM Levani Kokhreidze <
>>>>>> levani.codes@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> Still waiting for feedback on this KIP.
>>>>>>>>> Please let me know if you have any concerns and/or questions.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Levani
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Jul 24, 2019, at 8:20 PM, Sophie Blee-Goldman <sophie@confluent.io
>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Looks good! Thanks Levani,
>>>>>>>>>>
>>>>>>>>>> +1 (non-binding)
>>>>>>>>>>
>>>>>>>>>> Sophie
>>>>>>>>>>
>>>>>>>>>> On Tue, Jul 23, 2019 at 2:16 PM Levani Kokhreidze <
>>>>>>>>> levani.codes@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello,
>>>>>>>>>>>
>>>>>>>>>>> I’d like to initialize voting on KIP-221:
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>>> <
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>>>>
>>>>>>>>>>> If there’re any more concerns about the KIP, happy to discuss
>>>>>> further.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Levani
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> -- Guozhang
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>

Re: [VOTE] KIP-221: Enhance KStream with Connecting Topic Creation and Repartition Hint

Posted by John Roesler <vv...@apache.org>.
Thanks for the update, Levani!
-John

On Sat, Apr 4, 2020, at 04:36, Levani Kokhreidze wrote:
> Hello,
> 
> Small update regarding this KIP. As per John’s suggestion during the 
> code review 
> (https://github.com/apache/kafka/pull/7170#discussion_r392608571 
> <https://github.com/apache/kafka/pull/7170#discussion_r392608571>) 
> we’ve decided to remove KeyValueMapper overloads for the new 
> `repartition` operation for the first release of this feature.
> Wiki page has been updated accordingly 
> (https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint <https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint>)
> 
> Regards,
> Levani
> 
> > On Aug 1, 2019, at 9:55 AM, Levani Kokhreidze <le...@gmail.com> wrote:
> > 
> > Thank you all!
> > 
> > The vote has been open for ~8 days. KIP has three binding votes (Bill, Guozhang, Matthias) and one non-binding (Sophie) so the KIP vote passes!
> > I’ll mark KIP as accepted and start working on it as soon as possible!
> > 
> > Regards,
> > Levani
> > 
> >> On Aug 1, 2019, at 2:37 AM, Matthias J. Sax <ma...@confluent.io> wrote:
> >> 
> >> +1 (binding)
> >> 
> >> On 7/31/19 8:36 AM, Guozhang Wang wrote:
> >>> Thanks for the update! +1 (binding).
> >>> 
> >>> On Tue, Jul 30, 2019 at 11:42 PM Levani Kokhreidze <le...@gmail.com>
> >>> wrote:
> >>> 
> >>>> Hello Guozhang,
> >>>> 
> >>>> Thanks for the feedback. That’s an interesting point. To be honest, I
> >>>> totally missed it. I wasn’t aware that there’s `groupBy` possibility on
> >>>> KTable.
> >>>> I don’t see any reasons why not to add same functionality to KTable
> >>>> interface.
> >>>> 
> >>>> I’ve updated the KIP:
> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
> >>>> <
> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
> >>>>> 
> >>>> Please let me know if you have any other questions and/or concerns.
> >>>> 
> >>>> Regards,
> >>>> Levani
> >>>> 
> >>>>> On Jul 31, 2019, at 1:24 AM, Guozhang Wang <wa...@gmail.com> wrote:
> >>>>> 
> >>>>> Hello Levani,
> >>>>> 
> >>>>> Thanks for the KIP! Just got a quick question here about the scope: why
> >>>> do
> >>>>> we only want this for `KStream`, not `KTable#groupBy` for example?
> >>>>> 
> >>>>> 
> >>>>> Guozhang
> >>>>> 
> >>>>> 
> >>>>> On Tue, Jul 30, 2019 at 1:27 PM Bill Bejeck <bb...@gmail.com> wrote:
> >>>>> 
> >>>>>> Thanks for the KIP Levani.
> >>>>>> 
> >>>>>> +1 (binding)
> >>>>>> 
> >>>>>> -Bill
> >>>>>> 
> >>>>>> On Tue, Jul 30, 2019 at 3:37 PM Levani Kokhreidze <
> >>>> levani.codes@gmail.com>
> >>>>>> wrote:
> >>>>>> 
> >>>>>>> Hello,
> >>>>>>> 
> >>>>>>> Still waiting for feedback on this KIP.
> >>>>>>> Please let me know if you have any concerns and/or questions.
> >>>>>>> 
> >>>>>>> Regards,
> >>>>>>> Levani
> >>>>>>> 
> >>>>>>> 
> >>>>>>>> On Jul 24, 2019, at 8:20 PM, Sophie Blee-Goldman <sophie@confluent.io
> >>>>> 
> >>>>>>> wrote:
> >>>>>>>> 
> >>>>>>>> Looks good! Thanks Levani,
> >>>>>>>> 
> >>>>>>>> +1 (non-binding)
> >>>>>>>> 
> >>>>>>>> Sophie
> >>>>>>>> 
> >>>>>>>> On Tue, Jul 23, 2019 at 2:16 PM Levani Kokhreidze <
> >>>>>>> levani.codes@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>> 
> >>>>>>>>> Hello,
> >>>>>>>>> 
> >>>>>>>>> I’d like to initialize voting on KIP-221:
> >>>>>>>>> 
> >>>>>>> 
> >>>>>> 
> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
> >>>>>>>>> <
> >>>>>>>>> 
> >>>>>>> 
> >>>>>> 
> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
> >>>>>>>>>> 
> >>>>>>>>> If there’re any more concerns about the KIP, happy to discuss
> >>>> further.
> >>>>>>>>> 
> >>>>>>>>> Regards,
> >>>>>>>>> Levani
> >>>>>>> 
> >>>>>>> 
> >>>>>> 
> >>>>> 
> >>>>> 
> >>>>> --
> >>>>> -- Guozhang
> >>>> 
> >>>> 
> >>> 
> >> 
> > 
> 
>

Re: [VOTE] KIP-221: Enhance KStream with Connecting Topic Creation and Repartition Hint

Posted by Levani Kokhreidze <le...@gmail.com>.
Hello,

Small update regarding this KIP. As per John’s suggestion during the code review (https://github.com/apache/kafka/pull/7170#discussion_r392608571 <https://github.com/apache/kafka/pull/7170#discussion_r392608571>) we’ve decided to remove KeyValueMapper overloads for the new `repartition` operation for the first release of this feature.
Wiki page has been updated accordingly (https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint <https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint>)

Regards,
Levani

> On Aug 1, 2019, at 9:55 AM, Levani Kokhreidze <le...@gmail.com> wrote:
> 
> Thank you all!
> 
> The vote has been open for ~8 days. KIP has three binding votes (Bill, Guozhang, Matthias) and one non-binding (Sophie) so the KIP vote passes!
> I’ll mark KIP as accepted and start working on it as soon as possible!
> 
> Regards,
> Levani
> 
>> On Aug 1, 2019, at 2:37 AM, Matthias J. Sax <ma...@confluent.io> wrote:
>> 
>> +1 (binding)
>> 
>> On 7/31/19 8:36 AM, Guozhang Wang wrote:
>>> Thanks for the update! +1 (binding).
>>> 
>>> On Tue, Jul 30, 2019 at 11:42 PM Levani Kokhreidze <le...@gmail.com>
>>> wrote:
>>> 
>>>> Hello Guozhang,
>>>> 
>>>> Thanks for the feedback. That’s an interesting point. To be honest, I
>>>> totally missed it. I wasn’t aware that there’s `groupBy` possibility on
>>>> KTable.
>>>> I don’t see any reasons why not to add same functionality to KTable
>>>> interface.
>>>> 
>>>> I’ve updated the KIP:
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>> <
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>> 
>>>> Please let me know if you have any other questions and/or concerns.
>>>> 
>>>> Regards,
>>>> Levani
>>>> 
>>>>> On Jul 31, 2019, at 1:24 AM, Guozhang Wang <wa...@gmail.com> wrote:
>>>>> 
>>>>> Hello Levani,
>>>>> 
>>>>> Thanks for the KIP! Just got a quick question here about the scope: why
>>>> do
>>>>> we only want this for `KStream`, not `KTable#groupBy` for example?
>>>>> 
>>>>> 
>>>>> Guozhang
>>>>> 
>>>>> 
>>>>> On Tue, Jul 30, 2019 at 1:27 PM Bill Bejeck <bb...@gmail.com> wrote:
>>>>> 
>>>>>> Thanks for the KIP Levani.
>>>>>> 
>>>>>> +1 (binding)
>>>>>> 
>>>>>> -Bill
>>>>>> 
>>>>>> On Tue, Jul 30, 2019 at 3:37 PM Levani Kokhreidze <
>>>> levani.codes@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Hello,
>>>>>>> 
>>>>>>> Still waiting for feedback on this KIP.
>>>>>>> Please let me know if you have any concerns and/or questions.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Levani
>>>>>>> 
>>>>>>> 
>>>>>>>> On Jul 24, 2019, at 8:20 PM, Sophie Blee-Goldman <sophie@confluent.io
>>>>> 
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Looks good! Thanks Levani,
>>>>>>>> 
>>>>>>>> +1 (non-binding)
>>>>>>>> 
>>>>>>>> Sophie
>>>>>>>> 
>>>>>>>> On Tue, Jul 23, 2019 at 2:16 PM Levani Kokhreidze <
>>>>>>> levani.codes@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hello,
>>>>>>>>> 
>>>>>>>>> I’d like to initialize voting on KIP-221:
>>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>> <
>>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-221:+Enhance+KStream+with+Connecting+Topic+Creation+and+Repartition+Hint
>>>>>>>>>> 
>>>>>>>>> If there’re any more concerns about the KIP, happy to discuss
>>>> further.
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> Levani
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> -- Guozhang
>>>> 
>>>> 
>>> 
>> 
>