You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2022/02/21 01:34:00 UTC
[jira] [Commented] (IMPALA-11037) Bump ORC to 1.7-p4 to contain the improvement of ORC-1020
[ https://issues.apache.org/jira/browse/IMPALA-11037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17495273#comment-17495273 ]
Quanlong Huang commented on IMPALA-11037:
-----------------------------------------
Ran perf A/B test on TPCH dataset: [https://jenkins.impala.io/job/perf-AB-test/292]
{code:java}
Report Generated on 2022-02-20
Run Description: "4077bc849ae14bb92a463aeeb6c8f5c1fca658c9 vs d467a2f96d0ca03c77aae1b30b2bcacfff20a8e1"
Cluster Name: UNKNOWN
Lab Run Info: UNKNOWN
Impala Version: impalad version 4.1.0-SNAPSHOT RELEASE (2021-12-01)
Baseline Impala Version: impalad version 4.1.0-SNAPSHOT RELEASE (2021-12-01)
+----------+-------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+-------------------+---------+------------+------------+----------------+
| TPCH(42) | orc / def / block | 6.53 | -3.75% | 4.95 | -5.15% |
+----------+-------------------+---------+------------+------------+----------------+
+----------+----------+---------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+---------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval | Tval |
+----------+----------+---------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+---------+
| TPCH(42) | TPCH-Q1 | orc / block / block | 6.79 | 6.71 | +1.06% | 7.27% | 0.71% | 50 | -0.10% | -1.53 | 1.01 |
| TPCH(42) | TPCH-Q13 | orc / block / block | 7.48 | 7.46 | +0.20% | 1.10% | 0.93% | 50 | +0.02% | 0.99 | 1.00 |
| TPCH(42) | TPCH-Q21 | orc / block / block | 31.42 | 31.60 | -0.59% | 6.16% | 6.44% | 50 | +0.12% | 0.03 | -0.47 |
| TPCH(42) | TPCH-Q18 | orc / block / block | 12.20 | 12.26 | -0.53% | 1.55% | 1.92% | 50 | -0.41% | -2.01 | -1.53 |
| TPCH(42) | TPCH-Q9 | orc / block / block | 14.76 | 15.00 | -1.60% | 0.56% | 0.51% | 50 | -1.59% | -8.32 | -15.10 |
| TPCH(42) | TPCH-Q22 | orc / block / block | 2.36 | 2.39 | -1.29% | 2.19% | 2.04% | 50 | -2.04% | -3.18 | -3.06 |
| TPCH(42) | TPCH-Q10 | orc / block / block | 5.63 | 5.75 | -2.07% | 0.81% | 1.10% | 50 | -1.86% | -7.53 | -10.81 |
| TPCH(42) | TPCH-Q2 | orc / block / block | 1.60 | 1.64 | -2.73% | 3.55% | 2.72% | 50 | -3.13% | -3.66 | -4.39 |
| TPCH(42) | TPCH-Q3 | orc / block / block | 6.45 | 6.66 | -3.25% | 0.83% | 0.72% | 50 | -3.23% | -8.48 | -21.22 |
| TPCH(42) | TPCH-Q16 | orc / block / block | 2.35 | 2.44 | -3.57% | 1.75% | 1.76% | 50 | -4.22% | -7.00 | -10.36 |
| TPCH(42) | TPCH-Q19 | orc / block / block | 4.80 | 5.03 | -4.49% | 0.60% | 0.46% | 50 | -4.60% | -8.53 | -43.03 |
| TPCH(42) | TPCH-Q5 | orc / block / block | 5.00 | 5.30 | I -5.56% | 0.95% | 1.06% | 50 | I -5.89% | -8.53 | -28.37 |
| TPCH(42) | TPCH-Q14 | orc / block / block | 4.47 | 4.73 | I -5.68% | 1.22% | 1.05% | 50 | I -6.25% | -8.52 | -25.80 |
| TPCH(42) | TPCH-Q6 | orc / block / block | 2.46 | 2.61 | I -5.91% | 0.97% | 0.87% | 50 | I -6.13% | -8.53 | -33.10 |
| TPCH(42) | TPCH-Q7 | orc / block / block | 5.66 | 6.07 | I -6.80% | 1.24% | 0.77% | 50 | I -7.18% | -8.53 | -34.51 |
| TPCH(42) | TPCH-Q11 | orc / block / block | 1.20 | 1.29 | I -7.03% | 2.22% | 1.98% | 50 | I -8.31% | -8.50 | -17.35 |
| TPCH(42) | TPCH-Q8 | orc / block / block | 6.07 | 6.55 | I -7.42% | 1.16% | 1.02% | 50 | I -8.26% | -8.53 | -35.49 |
| TPCH(42) | TPCH-Q4 | orc / block / block | 4.03 | 4.37 | I -7.91% | 0.93% | 0.90% | 50 | I -8.77% | -8.53 | -44.83 |
| TPCH(42) | TPCH-Q17 | orc / block / block | 5.60 | 6.16 | I -9.17% | 4.54% | 5.88% | 50 | I -9.95% | -7.16 | -9.02 |
| TPCH(42) | TPCH-Q15 | orc / block / block | 5.63 | 6.33 | I -11.04% | 0.51% | 0.43% | 50 | I -12.54% | -8.53 | -124.63 |
| TPCH(42) | TPCH-Q12 | orc / block / block | 3.86 | 4.43 | I -12.89% | 1.39% | 1.24% | 50 | I -14.48% | -8.53 | -52.56 |
| TPCH(42) | TPCH-Q20 | orc / block / block | 3.85 | 4.43 | I -13.02% | 0.79% | 0.86% | 50 | I -14.58% | -8.53 | -83.66 |
+----------+----------+---------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+---------+
{code}
Attached the report details ( [^perf-report-ORC-1020-tpch-42.txt] ). CC [~drorke], [~rizaon]
> Bump ORC to 1.7-p4 to contain the improvement of ORC-1020
> ---------------------------------------------------------
>
> Key: IMPALA-11037
> URL: https://issues.apache.org/jira/browse/IMPALA-11037
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
> Fix For: Impala 4.1.0
>
> Attachments: orc_1.7.0-p3_random_int32.svg, orc_1.7.0-p3_random_int64.svg, orc_1.7.0-p4_random_int32.svg, orc_1.7.0-p4_random_int64.svg, perf-report-ORC-1020-tpch-42.txt
>
>
> ORC-1020 improves read performance of the ORC library in scanning random integers. Columns that encoded into integers, e.g. dictionary encoded strings, will also benifit from this.
> This Jira aims to add ORC-1020 to our native-toolchain and bump our orc version to 1.7-p4 to contain it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org