[jira] [Commented] (SAMZA-1555) Move creation of checkpoint and changelog streams to the Job Coordinators

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16335988#comment-16335988 ] 

ASF GitHub Bot commented on SAMZA-1555:
---------------------------------------

GitHub user dnishimura opened a pull request:

    https://github.com/apache/samza/pull/413

    SAMZA-1555: Move creation of checkpoint and changelog streams to the Job Coordinators

    **Overview**
    The purpose of this PR is to consolidate the creation of the changelog and checkpoint streams into the JobCoordinators. In the current state, the changelog stream is created from the JobModelManager and the checkpoint stream is created within the OffsetManager. The issue with creating the checkpoint in the OffsetManager is that the first call happens from the first SamzaContainer that runs and each subsequent SamzaContainer run will attempt to create the checkpoint stream.
    
    **Motivations**
    There are three driving forces for this refactoring. The first motivation is to assign the creation of the changelog and checkpoint streams to the JobCoordinators where it is most appropriate. This was discussed in more detail with @nickpan47  . The second motivation is to have any potential failure to stream creation happen no later than during job coordination. The third motivation is to accommodate future security work to provide a robust way to set ACLs on streams.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dnishimura/samza samza-1555-move-changelog-checkpoint-creation

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/413.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #413
    
----
commit 971fa91afb7f57fc8c099a7fb17e7bc96bc826c2
Author: Daniel Nishimura <dn...@...>
Date:   2018-01-16T21:20:07Z

    Move the responsibility of changelog and checkpoint stream creation to the job coordinators.

----


> Move creation of checkpoint and changelog streams to the Job Coordinators
> -------------------------------------------------------------------------
>
>                 Key: SAMZA-1555
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1555
>             Project: Samza
>          Issue Type: Improvement
>    Affects Versions: 0.15.0
>            Reporter: Daniel Nishimura
>            Assignee: Daniel Nishimura
>            Priority: Major
>
> This is an initial step ensuring components are responsible for only what they should be responsible for. For example, currently the JobModelManager creates the changelog stream. The JobModelManager should only create and manage the JobModel.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)