You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/09/06 23:57:00 UTC
[jira] [Commented] (SAMZA-1415) Add clearStream API in SystemAdmin

    [ https://issues.apache.org/jira/browse/SAMZA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156216#comment-16156216 ] 

ASF GitHub Bot commented on SAMZA-1415:
---------------------------------------

GitHub user xinyuiscool opened a pull request:

    https://github.com/apache/samza/pull/292

    SAMZA-1415: Add clearStream API in SystemAdmin and remove deprecated APIs

    The patch does the following:
    
    1) add clearStream() APi in SystemAdmin. Currently it's only supported in Kafka with broker configuring delete.topic.enable=true.
    
    2) remove the deprecated APIs including createChangeLogStream(), validateChangelogStream() and createCoordinatorStream().


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/xinyuiscool/samza SAMZA-1415

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/292.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #292
    
----
commit d2ae36f6074a1b224f43478e0b341837d41f061f
Author: Xinyu Liu <xi...@xiliu-ld1.linkedin.biz>
Date:   2017-09-06T17:11:23Z

    Initial impl for clearStream

commit 6b8298626c75b787058cc829de6fe012ed9f3875
Author: Xinyu Liu <xi...@xiliu-ld1.linkedin.biz>
Date:   2017-09-06T23:29:31Z

    Clean up the deprecated api for creating coordinator and changelog streams

----


> Add clearStream API in SystemAdmin
> ----------------------------------
>
>                 Key: SAMZA-1415
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1415
>             Project: Samza
>          Issue Type: Improvement
>    Affects Versions: 0.14.0
>            Reporter: Xinyu Liu
>            Assignee: Xinyu Liu
>
> For use cases like batch processing, samza need to re-process from beginning for each run. This also requires the ability to reset internal metadata streams so the processors will not accidentally read the meta data from previous runs, e.g. checkpoints. In order to support this use case, we need a new API in SystemAdmin to clear streams.
> For Kafka, clearStream will delete the topic in the brokers. This requires the following config for the brokers:
> *delete.topic.enable=true*



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)