[jira] [Commented] (PHOENIX-1091) Implement hard ceiling on per-query concurrency

["James Taylor (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-1091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086910#comment-14086910 ] 

James Taylor commented on PHOENIX-1091:
---------------------------------------

Phoenix has a lot of knobs and dials to impact concurrency and fairness. Phoenix already round robins between queued work and requires that the all scans that cover the query be submitted together (as merge sorts are done among them). First, we should build a concurrency test-bed harness so we can understand better how to use the existing knobs and dials. For example, experiment with bump up the thread pool queue size (as [~lhofhansl] suggested), and potentially increase the max & target concurrency to chunk up a query into smaller pieces to improve on fairness and prevent starvation.

Also related is PHOENIX-180 actively being worked on by [~ramkrishna]. Once this is in place, work can be chunked up into more or less even sizes.

> Implement hard ceiling on per-query concurrency
> -----------------------------------------------
>
>                 Key: PHOENIX-1091
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1091
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 4.0.0
>            Reporter: Eli Levine
>
> Phoenix has targetConcurrency and maxConcurrency configuration settings. However, it seems that they only come into effect when they are higher than the number of regions. In clients operating over tables with a large number of regions this could result in the consumption of a large number of client-side threads per query, potentially leading to starvation of other queries in Phoenix clients that support concurrent queries (such as highly multi-tenant environments).
> Implementation of a hard per-query concurrency limit is suggested to avoid potential starvation due to each query taking too many client threads.



--
This message was sent by Atlassian JIRA
(v6.2#6252)