[jira] [Resolved] (IMPALA-3807) Introduce support for dedicated Impalad coordinator(s)

["Dimitris Tsirogiannis (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dimitris Tsirogiannis resolved IMPALA-3807.
-------------------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.9.0

> Introduce support for dedicated Impalad coordinator(s)
> ------------------------------------------------------
>
>                 Key: IMPALA-3807
>                 URL: https://issues.apache.org/jira/browse/IMPALA-3807
>             Project: IMPALA
>          Issue Type: New Feature
>          Components: Backend
>    Affects Versions: Impala 2.8.0
>            Reporter: Mostafa Mokhtar
>              Labels: performance, scalability
>             Fix For: Impala 2.9.0
>
>
> After running large scale concurrent tests it was obvious that a dedicated coordinator.
> During the concurrency tests resource utilization in terms of CPU, Memory and Network were significantly higher than other worker nodes.
> The increase in resource utilization was mostly due to 
> * Sending exec plan fragment to remote fragments, ExecPlanFragment can become very large for complex queries with lots of files & partitions
> * Runtime filters consume lots of untracked memory when the bloom filters are aggregated
> * Runtime filters consume lots of Network throughput on the coordinator as it receives, aggregates and sends the filters
> * Aggregating RunProfile counters 
> * When CPU utilization on the coordinator is high remote fragments occasionally fail with "Couldn't get a client for impala-8-2.vpc.cloudera.com:22000 Reason: Couldn't open transport for impala-compete-8-2.vpc.cloudera.com:22000 (connect() failed: Connection timed out)"
> * Slow start of remote query fragments specially when the coordinator node is under heavy load
> * Admission control counters can become out of sync when using multiple coordinators 
> Metadata should naturally benefit from running a dedicated coordinator as metadata is less likely to become out of sync and one day we can get rid of expensive catalog updates. 
> The dedicated coordinator should only run the +Coordinator Fragment+ and not participate in any scans, aggs etc.. 
> This work will require CM support.
> Running a dedicated coordinator  significantly improved performance of complex TPC-DS queries
> |Query||Baseline (seconds)||Dedicated coord (seconds)||Speedup %|
> |74|73|43|70%|
> |75|152|91|67%|
> |4|127|78|63%|
> |25|53|34|56%|
> |95|35|23|52%|
> |11|82|54|52%|
> |85|37|25|48%|
> |17|71|48|48%|
> |64|147|101|46%|
> |29|65|47|38%|
> |72|408|312|31%|
> |18|36|28|29%|
> |44|36|28|29%|
> |60|6|5|20%|
> |77|12|10|20%|
> |50|43|36|19%|
> |5|60|51|18%|
> |81|14|12|17%|
> |23|198|170|16%|
> |80|29|25|16%|
> |35|22|19|16%|
> |68|15|13|15%|
> |78|551|486|13%|
> |31|36|32|13%|



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)