[jira] [Created] (SAMZA-2116) Make sendTo and sink operators non-terminal

["Prateek Maheshwari (JIRA)" <ji...@apache.org>]

Prateek Maheshwari created SAMZA-2116:
-----------------------------------------

             Summary: Make sendTo and sink operators non-terminal
                 Key: SAMZA-2116
                 URL: https://issues.apache.org/jira/browse/SAMZA-2116
             Project: Samza
          Issue Type: Improvement
            Reporter: Prateek Maheshwari


sendTo(stream), sendTo(table) and sink are currently terminal operators in the High Level API. They don't return a MessageStream, and hence do not allow chaining of further operators.

Consider the following use case:

Consume a message -> send an event denoting processing start -> do some processing -> send an event denoting processing end.

Currently there's no good way to do this, since:
 # HL API does not allow chaining further operations on sendTo
 # HL API does not guarantee order of execution across multiple operators chained on the same operator.

The only way to do this today will be to use a sink function, which is not great.



One consideration when making sendTo(stream) et al. non-terminal: unlike partitionBy, which returns data in the re-keyed MessageStream partition, sendTo will return the data in the original message stream partition. This will need to be clearly called out.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)