You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Stefania (JIRA)" <ji...@apache.org> on 2016/05/12 08:37:12 UTC

[jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests

    [ https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281320#comment-15281320 ] 

Stefania commented on CASSANDRA-11521:
--------------------------------------

Just a quick update. 

I have a tentative implementation [here|https://github.com/apache/cassandra/compare/trunk...stef1927:11521#diff-468c74b80d7a1d5b21948217659af747R49] based on a queue where a worker thread puts at most 3 pages and, if the queue stays full for a specified number of seconds, then the worker thread releases the resources and gives up. If the client keeps up, the worker continues to put pages on the queue without releasing resources (well the pagers still create a new partition iterator for each page but we can fix that later if it shows up in JFR).

Unfortunately, the initial results from the [unit tests|https://github.com/apache/cassandra/compare/trunk...stef1927:11521#diff-04e8835163e2a326515d61f448a8ebbcR75] show that pure streaming is still about 3 times faster.

If I've read the profiles correctly, this is due to the fact that the encoding of a {{ResultSet}} into a Netty {{ByteBuffer}} is still done synchronously when the client requests a page. Therefore, I plan to encode the results directly into a Netty BB ant put this, not a {{ResultSet}}, in the queue. Once this is done, the difference between the two approaches should just be the client request messages and the user of the pagers.

> Implement streaming for bulk read requests
> ------------------------------------------
>
>                 Key: CASSANDRA-11521
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11521
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Local Write-Read Paths
>            Reporter: Stefania
>            Assignee: Stefania
>             Fix For: 3.x
>
>
> Allow clients to stream data from a C* host, bypassing the coordination layer and eliminating the need to query individual pages one by one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)