You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by dguy <gi...@git.apache.org> on 2016/06/05 14:59:13 UTC

[GitHub] kafka pull request #1472: KAFKA-3561: Auto create through topic for KStream ...

GitHub user dguy opened a pull request:

    https://github.com/apache/kafka/pull/1472

    KAFKA-3561: Auto create through topic for KStream aggregation and join [WIP]

    @guozhangwang @enothereska @mjsax @miguno
    
    If you get a chance can you please take a look at this. I've done the repartitioning in the join, but it results in 2 internal topics for each join. This seems like overkill as sometimes we wouldn't need to repartition at all, others just 1 topic, and then sometimes both, but I'm not sure how we can know that. 
    
    I'd also need to implement something similar for leftJoin, but again, i'd like to see if i'm heading down the right path or if anyone has any other bright ideas.
    
    For reference - https://github.com/apache/kafka/pull/1453 - the previous PR
    
    Thanks for taking the time and looking forward to getting some welcome advice :-)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dguy/kafka KAFKA-3561

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/1472.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1472
    
----
commit 053c809b5aa322327988909d53027d4682df6825
Author: Damian Guy <da...@gmail.com>
Date:   2016-06-01T13:03:23Z

    repartition through internal topic on KStream.map()

commit ba1b50285f9eae028e531183500de90a2ae877c7
Author: Damian Guy <da...@continuum.local>
Date:   2016-06-04T15:17:51Z

    Merge remote-tracking branch 'upstream/trunk' into KAFKA-3561

commit a8c14f38914410bc3c7ff1e96c040bf1a1992cef
Author: Damian Guy <da...@continuum.local>
Date:   2016-06-05T14:41:12Z

    repartition on join

commit 1220c61464676881273e47d7e02ea8d502cd8fd4
Author: Damian Guy <da...@gmail.com>
Date:   2016-06-05T14:51:17Z

    repartition on join

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] kafka pull request #1472: KAFKA-3561: Auto create through topic for KStream ...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/kafka/pull/1472


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---