[jira] [Comment Edited] (CASSANDRA-5286) Streaming 2.0

["Yuki Morishita (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13685622#comment-13685622 ] 

Yuki Morishita edited comment on CASSANDRA-5286 at 6/17/13 2:40 PM:
--------------------------------------------------------------------

Pushed update to: https://github.com/yukim/cassandra/commits/5286-2
Doc is updated too: https://gist.github.com/yukim/5672508

Changes:

* API revised. Basically, you build your [stream plan|https://github.com/yukim/cassandra/blob/5286-2/src/java/org/apache/cassandra/streaming/StreamPlan.java], execute it, and get [future result of the streaming|https://github.com/yukim/cassandra/blob/5286-2/src/java/org/apache/cassandra/streaming/StreamResultFuture.java].
* Migrated all stream operation to new API including nodetool. Bulkloader is not working because of CASSANDRA-5542 though. Repair works with new API, but we should refactor after CASSANDRA-5426.
* Versioned streaming protocol

Still not implemented(but things we can do in separate tickets IMHO):

* JMX notification
* Stream different version of SSTable
* Progress updated by row on read path
                
      was (Author: yukim):
    Pushed update to: https://github.com/yukim/cassandra/commits/5286-2
Doc is updated too: https://gist.github.com/yukim/5672508

Changes:

* API revised. Basically, you build your [stream plan|https://github.com/yukim/cassandra/blob/5286-2/src/java/org/apache/cassandra/streaming/StreamPlan.java], execute it, and get [future result of the streaming|https://github.com/yukim/cassandra/blob/5286-2/src/java/org/apache/cassandra/streaming/StreamResultFuture.java].
* Migrated all stream operation to new API including nodetool. Bulkloader is not working because of CASSANDRA-5542 though. Repair works with new API, but we should refactor after CASSANDRA-5426.
* Versioned streaming protocol

Still not implemented(but things we can do in separate tickets IMHO):

* JMX notification
* Stream different version of SSTable
                  
> Streaming 2.0
> -------------
>
>                 Key: CASSANDRA-5286
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5286
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Yuki Morishita
>              Labels: streaming
>             Fix For: 2.0
>
>
> 2.0 is the good time to redesign streaming API including protocol to make streaming more performant and reliable.
> Design goals that come up in my mind:
> *Better performance*
>   - Protocol optimization
>   - Stream multiple files in parallel (CASSANDRA-4663)
>   - Persistent connection (CASSANDRA-4660)
> *Better control*
>   - Cleaner API for error handling
>   - Integrate both IN/OUT streams into one session, so the components(bootstrap, move, bulkload, repair...) that use streaming can manage them easily.
> *Better reporting*
>   - Better logging/tracing
>   - More metrics
>   - Progress reporting API for external client

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira