[jira] [Assigned] (SAMZA-2116) Make sendTo and sink operators non-terminal

["Sanil Jain (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sanil Jain reassigned SAMZA-2116:
---------------------------------

    Assignee: Sanil Jain

> Make sendTo and sink operators non-terminal
> -------------------------------------------
>
>                 Key: SAMZA-2116
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2116
>             Project: Samza
>          Issue Type: Improvement
>            Reporter: Prateek Maheshwari
>            Assignee: Sanil Jain
>            Priority: Major
>
> sendTo(stream), sendTo(table) and sink are currently terminal operators in the High Level API. They don't return a MessageStream, and hence do not allow chaining of further operators.
> Consider the following use case:
> Consume a message -> send an event denoting processing start -> do some processing -> send an event denoting processing end.
> Currently there's no good way to do this, since:
>  # HL API does not allow chaining further operations on sendTo
>  # HL API does not guarantee order of execution across multiple operators chained on the same operator.
> The only way to do this today will be to use a sink function, which is not great.
> One consideration when making sendTo(stream) et al. non-terminal: unlike partitionBy, which returns data in the re-keyed MessageStream partition, sendTo will return the data in the original message stream partition. This will need to be clearly called out.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)