[jira] [Commented] (DRILL-4313) C++ client should manage load balance of queries

["Parth Chandra (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-4313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122609#comment-15122609 ] 

Parth Chandra commented on DRILL-4313:
--------------------------------------

Looks like the C++ client does not do a good job of picking a random drillbit. The implementation (incorrectly) resets the random seed so the random number generator repeats the random number if many connections are made at the same time. Also, even with that fixed, it appears that with a small number of driillbits in the cluster, one can get a non-uniform distribution.
Experimenting with other methods, it looks like using a random_shuffle (similar to the Java DrillClient) gives a much better distribution.
Submitting a patch with this fixed.

> C++ client should manage load balance of queries
> ------------------------------------------------
>
>                 Key: DRILL-4313
>                 URL: https://issues.apache.org/jira/browse/DRILL-4313
>             Project: Apache Drill
>          Issue Type: Improvement
>            Reporter: Parth Chandra
>
> The current C++ client handles multiple parallel queries over the same connection, but that creates a bottleneck as the queries get sent to the same drillbit.
> The client can manage this more effectively by choosing from a configurable pool of connections and round robin queries to them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)