[jira] [Created] (ZEPPELIN-274) Add Support for Streaming (long-running) Tasks.

["Christian Tzolov (JIRA)" <ji...@apache.org>]

Christian Tzolov created ZEPPELIN-274:
-----------------------------------------

             Summary: Add Support for Streaming (long-running) Tasks.
                 Key: ZEPPELIN-274
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-274
             Project: Zeppelin
          Issue Type: New Feature
          Components: Core, Interpreters
    Affects Versions: 0.6.0
            Reporter: Christian Tzolov


The Zeppelin's Paragraph State Machines (SM) defines 3 active states (PENDING, FINISHED and FAILED). At high level the SM looks like this:

FINISHED - (On Run) -> PENDING - (On Success) -> FINISHED /or (On Failure) -> FAILED

The Stream (long-running) processes are continuous as opposed to typical Paragraph's batch-like jobs. Currently if you start a twitter search stream task in a paragraph (by clicking the Run button) the underlying streaming platform will open a stream and will return some information for the newly created stream. The Paragraph SM will consider the job done and will transfer from PENDING into  FINISHED state.  In reality the actual state should be STARTED (or alike) to reflect that the stream is running and should transfer into FINISHED only after the stream is closed.

To support long-running (aka streaming) jobs Zeppelin need to extend its SM with at least one additional state: 'STARTED'.

Such improvement would benefit streaming platforms like Spark Streaming, Flink, SpringXD and alike. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)