Impala Admission Control concurrent query runtime

["Roberto Cerioni, Paulo" <pr...@rbbn.com>]

Hello,


I've done some tests with concurrent queries running in the same queue and it seems the runtime always receive a penalty compared to a single query running, even if we have plenty of resources available to run multiple queries concurrently in that queue. In addition, I tried that by running the same query concurrently in different queues and the runtime received a penalty as well. For instance, if a query runtime running alone in the cluster is, let's say, 10 seconds, it seems that I can only achieve that performance when no other query is running in the cluster, otherwise there is a penalty of around 50% in runtime.


Other behavior I noticed was that the result of queries is being released in "batches" according with the concurrency I have set (that might even be the reason for the aforementioned issue). For example, if I run a single query, I get the answer in 1 second, but if I run 2 of that same query concurrently the results of both queries will come after 1.5s. If I run 5 of the same query concurrently, then the result of all 5 queries is going to come back at the same time after 3 seconds, and so on. The penalty in runtime always increases with the number of queries I run concurrently, because the first issued query apparently has to wait the whole "batch" to execute.


Could you please explain what is happening under the hood that might be driving this behavior? Is this supposed to work this way in the latest version of Impala we have or is there something in the configuration I can use to prevent this? Is that planned for Impala's roadmap?


Thanks,

Paulo.

Re: Impala Admission Control concurrent query runtime

[Alexander Behm <al...@cloudera.com>]

Hi Paulo,

for us to help you with performance issues it's very helpful to have the
query profiles of the queries you ran. Based on those profiles, we can
hopefully identify where the time is being spent and pinpoint the issue.

Sending us the profile of single query run and the 3 profiles of the 3
concurrent queries would be a good start. You can also poke around in the
profiles yourself and maybe you'll find something.

Alex

On Wed, Feb 28, 2018 at 11:37 AM, Roberto Cerioni, Paulo <
probertocerioni@rbbn.com> wrote:

> Hello,
>
>
> I've done some tests with concurrent queries running in the same queue and
> it seems the runtime always receive a penalty compared to a single query
> running, even if we have plenty of resources available to run multiple
> queries concurrently in that queue. In addition, I tried that by running
> the same query concurrently in different queues and the runtime received a
> penalty as well. For instance, if a query runtime running alone in the
> cluster is, let's say, 10 seconds, it seems that I can only achieve that
> performance when no other query is running in the cluster, otherwise there
> is a penalty of around 50% in runtime.
>
>
> Other behavior I noticed was that the result of queries is being released
> in "batches" according with the concurrency I have set (that might even be
> the reason for the aforementioned issue). For example, if I run a single
> query, I get the answer in 1 second, but if I run 2 of that same query
> concurrently the results of* both* queries will come after 1.5s. If I run
> 5 of the same query concurrently, then the result of all 5 queries is going
> to come back at the same time after 3 seconds, and so on. The penalty in
> runtime always increases with the number of queries I run concurrently,
> because the first issued query apparently has to wait the whole "batch" to
> execute.
>
>
> Could you please explain what is happening under the hood that might be
> driving this behavior? Is this supposed to work this way in the latest
> version of Impala we have or is there something in the configuration I can
> use to prevent this? Is that planned for Impala's roadmap?
>
>
> Thanks,
>
> Paulo.
>
>
>
>