You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Corentin Chary <co...@gmail.com> on 2016/12/07 15:10:32 UTC
streaming_connections_per_host - speeding up CPU bound bootstrap
Currently the StreamPlan created for bootstrap (and rebuild) will only
create one connection per host. If you have less nodes than cores,
this is likely to be CPU bound (a CPU seems to be able to process
~5MB/s).
Is there any reason why something naive like
https://github.com/iksaif/cassandra/commit/8352c21284811ca15d63183ceae0b11586623f31
would not work ?
I believe this is what is about
https://issues.apache.org/jira/browse/CASSANDRA-4663
See also: https://issues.apache.org/jira/browse/CASSANDRA-12229, but I
don't believe non-blocking I/O would change anything here.
--
Corentin Chary
http://xf.iksaif.net
Re: streaming_connections_per_host - speeding up CPU bound bootstrap
Posted by Corentin Chary <co...@gmail.com>.
Hard to say because this comment doesn't show the code that was tried.
My proposed change
(https://issues.apache.org/jira/secure/attachment/12842717/0001-streaming-add-a-way-to-configure-the-number-of-conne.patch)
should open multiple connections per hosts: this fixes blocking writes
on a single connection or CPU-bound (de)-serialization.
On Mon, Dec 12, 2016 at 2:04 AM, Nate McCall <zz...@gmail.com> wrote:
> I have not dug too deeply yet, but how would you compare/reconcile
> your proposed changes with this comment:
> https://issues.apache.org/jira/browse/CASSANDRA-4663?focusedCommentId=15342248&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15342248
>
> On Thu, Dec 8, 2016 at 4:10 AM, Corentin Chary <co...@gmail.com> wrote:
>> Currently the StreamPlan created for bootstrap (and rebuild) will only
>> create one connection per host. If you have less nodes than cores,
>> this is likely to be CPU bound (a CPU seems to be able to process
>> ~5MB/s).
>>
>> Is there any reason why something naive like
>> https://github.com/iksaif/cassandra/commit/8352c21284811ca15d63183ceae0b11586623f31
>> would not work ?
>>
>> I believe this is what is about
>> https://issues.apache.org/jira/browse/CASSANDRA-4663
>> See also: https://issues.apache.org/jira/browse/CASSANDRA-12229, but I
>> don't believe non-blocking I/O would change anything here.
>>
>> --
>> Corentin Chary
>> http://xf.iksaif.net
--
Corentin Chary
http://xf.iksaif.net
Re: streaming_connections_per_host - speeding up CPU bound bootstrap
Posted by Nate McCall <zz...@gmail.com>.
I have not dug too deeply yet, but how would you compare/reconcile
your proposed changes with this comment:
https://issues.apache.org/jira/browse/CASSANDRA-4663?focusedCommentId=15342248&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15342248
On Thu, Dec 8, 2016 at 4:10 AM, Corentin Chary <co...@gmail.com> wrote:
> Currently the StreamPlan created for bootstrap (and rebuild) will only
> create one connection per host. If you have less nodes than cores,
> this is likely to be CPU bound (a CPU seems to be able to process
> ~5MB/s).
>
> Is there any reason why something naive like
> https://github.com/iksaif/cassandra/commit/8352c21284811ca15d63183ceae0b11586623f31
> would not work ?
>
> I believe this is what is about
> https://issues.apache.org/jira/browse/CASSANDRA-4663
> See also: https://issues.apache.org/jira/browse/CASSANDRA-12229, but I
> don't believe non-blocking I/O would change anything here.
>
> --
> Corentin Chary
> http://xf.iksaif.net