[jira] [Updated] (SAMZA-123) Move topic partition grouping to the AM and generalize

["Jakob Homan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jakob Homan updated SAMZA-123:
------------------------------

    Attachment: SAMZA-123-2.patch

Attaching patch rebased off master.  Still has GroupIntoNSets just to show how it would work, but all tests pass.  Still need to re-write the offsets tool after we get further into the review.  Not crazy about how TaskNameToSSPs came out; implementing a Map in Scala is insanely annoying.  May be better to just go with straight map (possibly type aliased) and helper methods.  

> Move topic partition grouping to the AM and generalize
> ------------------------------------------------------
>
>                 Key: SAMZA-123
>                 URL: https://issues.apache.org/jira/browse/SAMZA-123
>             Project: Samza
>          Issue Type: Sub-task
>          Components: container
>    Affects Versions: 0.6.0
>            Reporter: Jakob Homan
>            Assignee: Jakob Homan
>         Attachments: SAMZA-123-2.patch, SAMZA-123-design-doc.md, SAMZA-123-design-doc.pdf, SAMZA-123-draft.patch
>
>
> Currently the AM sends a set of all the topics and partitions to the container, which then groups them by partition and assigns each set to a task instance. By moving the grouping to the AM, we can assign arbitrary groups to task instances, which will allow more partitioning strategies, as discussed in SAMZA-71.



--
This message was sent by Atlassian JIRA
(v6.2#6252)