You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/07/15 04:00:00 UTC
[jira] [Created] (IMPALA-9956) Inlining functions in
Sorter::Partition() gives a significant speedup.
Tim Armstrong created IMPALA-9956:
-------------------------------------
Summary: Inlining functions in Sorter::Partition() gives a significant speedup.
Key: IMPALA-9956
URL: https://issues.apache.org/jira/browse/IMPALA-9956
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Tim Armstrong
Assignee: Tim Armstrong
In TPC-H q67, the sort gets ~9% faster just from this small change.
{noformat}
AFTER
Fetched 100 row(s) in 3.27s
[localhost:21000] tpcds_parquet> summary;
+-----------------+--------+-------+----------+----------+---------+------------+-----------+---------------+---------------------------+
| Operator | #Hosts | #Inst | Avg Time | Max Time | #Rows | Est. #Rows | Peak Mem | Est. Peak Mem | Detail |
+-----------------+--------+-------+----------+----------+---------+------------+-----------+---------------+---------------------------+
| F00:ROOT | 1 | 1 | 402.93us | 402.93us | | | 0 B | 0 B | |
| 11:TOP-N | 1 | 1 | 9.85ms | 9.85ms | 100 | 100 | 7.04 MB | 9.38 KB | |
| 10:ANALYTIC | 1 | 1 | 102.50ms | 102.50ms | 514.59K | 15.09M | 11.02 MB | 4.00 MB | |
| 09:SORT | 1 | 1 | 304.39ms | 304.39ms | 514.59K | 15.09M | 80.02 MB | 38.00 MB | |
| 08:AGGREGATE | 1 | 1 | 178.87ms | 178.87ms | 514.59K | 15.09M | 114.16 MB | 1.42 GB | FINALIZE |
| 07:AGGREGATE | 1 | 1 | 794.68ms | 794.68ms | 514.59K | 15.09M | 257.67 MB | 16.98 GB | FINALIZE |
| 06:HASH JOIN | 1 | 1 | 33.95ms | 33.95ms | 535.97K | 2.88M | 3.33 MB | 2.88 MB | INNER JOIN |
| |--03:SCAN HDFS | 1 | 1 | 2.77ms | 2.77ms | 18.00K | 18.00K | 2.69 MB | 80.00 MB | tpcds_parquet.item |
| 05:HASH JOIN | 1 | 1 | 8.02ms | 8.02ms | 535.97K | 2.88M | 2.05 MB | 1.94 MB | INNER JOIN |
| |--02:SCAN HDFS | 1 | 1 | 1.18ms | 1.18ms | 12 | 12 | 69.00 KB | 32.00 MB | tpcds_parquet.store |
| 04:HASH JOIN | 1 | 1 | 10.22ms | 10.22ms | 535.97K | 2.88M | 2.01 MB | 1.94 MB | INNER JOIN |
| |--01:SCAN HDFS | 1 | 1 | 2.62ms | 2.62ms | 365 | 7.30K | 1.33 MB | 80.00 MB | tpcds_parquet.date_dim |
| 00:SCAN HDFS | 1 | 1 | 9.06ms | 9.06ms | 535.97K | 2.88M | 14.37 MB | 64.00 MB | tpcds_parquet.store_sales |
+-----------------+--------+-------+----------+----------+---------+------------+-----------+---------------+---------------------------+
BEFORE
Fetched 100 row(s) in 3.33s
[localhost:21000] tpcds_parquet> summary;
+-----------------+--------+-------+----------+----------+---su------+------------+-----------+---------------+---------------------------+
| Operator | #Hosts | #Inst | Avg Time | Max Time | #Rows | Est. #Rows | Peak Mem | Est. Peak Mem | Detail |
+-----------------+--------+-------+----------+----------+---------+------------+-----------+---------------+---------------------------+
| F00:ROOT | 1 | 1 | 414.22us | 414.22us | | | 0 B | 0 B | |
| 11:TOP-N | 1 | 1 | 10.03ms | 10.03ms | 100 | 100 | 7.04 MB | 9.38 KB | |
| 10:ANALYTIC | 1 | 1 | 101.26ms | 101.26ms | 514.59K | 15.09M | 11.02 MB | 4.00 MB | |
| 09:SORT | 1 | 1 | 334.83ms | 334.83ms | 514.59K | 15.09M | 80.02 MB | 38.00 MB | |
| 08:AGGREGATE | 1 | 1 | 221.69ms | 221.69ms | 514.59K | 15.09M | 114.16 MB | 1.42 GB | FINALIZE |
| 07:AGGREGATE | 1 | 1 | 811.38ms | 811.38ms | 514.59K | 15.09M | 257.68 MB | 16.98 GB | FINALIZE |
| 06:HASH JOIN | 1 | 1 | 30.35ms | 30.35ms | 535.97K | 2.88M | 3.33 MB | 2.88 MB | INNER JOIN |
| |--03:SCAN HDFS | 1 | 1 | 2.71ms | 2.71ms | 18.00K | 18.00K | 2.63 MB | 80.00 MB | tpcds_parquet.item |
| 05:HASH JOIN | 1 | 1 | 7.48ms | 7.48ms | 535.97K | 2.88M | 2.07 MB | 1.94 MB | INNER JOIN |
| |--02:SCAN HDFS | 1 | 1 | 869.04us | 869.04us | 12 | 12 | 69.00 KB | 32.00 MB | tpcds_parquet.store |
| 04:HASH JOIN | 1 | 1 | 9.77ms | 9.77ms | 535.97K | 2.88M | 2.01 MB | 1.94 MB | INNER JOIN |
| |--01:SCAN HDFS | 1 | 1 | 2.83ms | 2.83ms | 365 | 7.30K | 1.33 MB | 80.00 MB | tpcds_parquet.date_dim |
| 00:SCAN HDFS | 1 | 1 | 8.31ms | 8.31ms | 535.97K | 2.88M | 14.35 MB | 64.00 MB | tpcds_parquet.store_sales |
+-----------------+--------+-------+----------+----------+---------+------------+-----------+---------------+---------------------------+
{noformat]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)