You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Xinyu Liu (JIRA)" <ji...@apache.org> on 2019/01/08 21:02:00 UTC

[jira] [Commented] (SAMZA-1531) Support run.id in standalone for batch processing

    [ https://issues.apache.org/jira/browse/SAMZA-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737517#comment-16737517 ] 

Xinyu Liu commented on SAMZA-1531:
----------------------------------

As open source users are picking up beam on samza standalone, this is becoming the blocker for processing any batch data. [~spvenkat]: we need a fix for it soon. The run.id can just be the first model version that the leader agrees upon. Doens't need to be fancy. As along as it's unique across each run.

> Support run.id in standalone for batch processing
> -------------------------------------------------
>
>                 Key: SAMZA-1531
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1531
>             Project: Samza
>          Issue Type: Bug
>    Affects Versions: 0.14.0
>            Reporter: Xinyu Liu
>            Assignee: Shanthoosh Venkataraman
>            Priority: Major
>             Fix For: 0.15.0
>
>
> Similar to Yarn, we need to generate a unique run.id per job run in standalone. Right now the lock is acquired after the plan is generated, then the streams are created. Instead, we need to generate the id first, acquiring the lock based on this id, then generate the plan as well as the streams.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)