[jira] [Commented] (SAMZA-1073) Design top-level fluent APIs operators that are capable to be deployed in multi-stage jobs

["Yi Pan (Data Infrastructure) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859300#comment-15859300 ] 

Yi Pan (Data Infrastructure) commented on SAMZA-1073:
-----------------------------------------------------

Discussed w/ [~jmakes] and [~xinyu], since the fluent api is relevant to multiple projects (i.e. SAMZA-1041, SAMZA-1080), we will start a branch samza-fluent-api-v1 to share the code for development. The following are the tentative order of commits to this shared branch:

# Start open source branch samza-fluent-api-v1
# merge the top-level APIs and examples on top of Jagadish's window API commit
# move the classes from samza-operator to samza-core and remove the samza-operator module
# Jacob's stream spec patch (this can be parallel w/ 1 and 2)
# Xinyu's execution environment patch
# Boris and Navina's standalone branch


> Design top-level fluent APIs operators that are capable to be deployed in multi-stage jobs
> ------------------------------------------------------------------------------------------
>
>                 Key: SAMZA-1073
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1073
>             Project: Samza
>          Issue Type: New Feature
>            Reporter: Yi Pan (Data Infrastructure)
>            Assignee: Yi Pan (Data Infrastructure)
>         Attachments: SAMZA-1073operator-multi-stagejob-levelprogrammingAPI.pdf
>
>
> It would be nice to allow users to stay at logic level when using fluent API's operators, w/o concerning about physical partitions of the stream and potential grouping of operators into multiple / single Samza jobs (SAMZA-1041).
> Hence, the fluent API needs to be able to express the physical topics as boundaries between stages in the single logic DAG.
> Besides, users should be able to use fluent API to describe a logic expression at top level, not within a job or within a task.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)