You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2017/09/18 19:54:00 UTC
[jira] [Resolved] (IMPALA-5860) Upgrade LLVM to 3.9.0 or later
[ https://issues.apache.org/jira/browse/IMPALA-5860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-5860.
-----------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.11.0
IMPALA-5860: upgrade to LLVM 3.9.1
LLVM made a few API changes:
* Misc minor changes to function and type signatures
* The CloneFunction() API changed semantics (http://reviews.llvm.org/D18628)
Needed to fix a few new clang-tidy warnings.
Testing:
Ran core and ASAN tests.
Perf:
Ran single node TPC-H and targeted perf with scale factor 60. Both
improved on average.
+----------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+-----------------------+---------+------------+------------+----------------+
| TPCH(60) | parquet / none / none | 17.82 | -5.01% | 11.64 | -4.23% |
+----------+-----------------------+---------+------------+------------+----------------+
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Num Clients | Iters |
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| TPCH(60) | TPCH-Q1 | parquet / none / none | 27.97 | 27.59 | +1.36% | 0.39% | 0.41% | 1 | 5 |
| TPCH(60) | TPCH-Q20 | parquet / none / none | 5.81 | 5.78 | +0.44% | 0.73% | 0.21% | 1 | 5 |
| TPCH(60) | TPCH-Q21 | parquet / none / none | 62.98 | 62.98 | +0.01% | 5.56% | 1.07% | 1 | 5 |
| TPCH(60) | TPCH-Q15 | parquet / none / none | 8.45 | 8.46 | -0.20% | 0.40% | 0.38% | 1 | 5 |
| TPCH(60) | TPCH-Q4 | parquet / none / none | 5.57 | 5.59 | -0.41% | 0.43% | 0.80% | 1 | 5 |
| TPCH(60) | TPCH-Q6 | parquet / none / none | 3.16 | 3.17 | -0.45% | 0.78% | 1.70% | 1 | 5 |
| TPCH(60) | TPCH-Q5 | parquet / none / none | 7.41 | 7.47 | -0.92% | 0.71% | 1.06% | 1 | 5 |
| TPCH(60) | TPCH-Q9 | parquet / none / none | 33.45 | 33.78 | -0.99% | 1.15% | 0.85% | 1 | 5 |
| TPCH(60) | TPCH-Q11 | parquet / none / none | 2.00 | 2.03 | -1.34% | 1.71% | 2.24% | 1 | 5 |
| TPCH(60) | TPCH-Q2 | parquet / none / none | 4.71 | 4.79 | -1.60% | 1.49% | 1.95% | 1 | 5 |
| TPCH(60) | TPCH-Q18 | parquet / none / none | 46.48 | 47.71 | -2.58% | 1.04% | 0.38% | 1 | 5 |
| TPCH(60) | TPCH-Q14 | parquet / none / none | 5.85 | 6.02 | -2.84% | 0.44% | 0.70% | 1 | 5 |
| TPCH(60) | TPCH-Q22 | parquet / none / none | 6.51 | 6.76 | -3.71% | 2.29% | 2.42% | 1 | 5 |
| TPCH(60) | TPCH-Q19 | parquet / none / none | 7.27 | 7.63 | -4.69% | 1.33% | 0.78% | 1 | 5 |
| TPCH(60) | TPCH-Q10 | parquet / none / none | 13.19 | 13.84 | -4.73% | 0.42% | 1.44% | 1 | 5 |
| TPCH(60) | TPCH-Q13 | parquet / none / none | 21.95 | 23.12 | -5.03% | 0.25% | 1.19% | 1 | 5 |
| TPCH(60) | TPCH-Q16 | parquet / none / none | 5.29 | 5.57 | -5.04% | 0.85% | 0.78% | 1 | 5 |
| TPCH(60) | TPCH-Q7 | parquet / none / none | 42.05 | 44.33 | -5.16% | 2.07% | 2.28% | 1 | 5 |
| TPCH(60) | TPCH-Q12 | parquet / none / none | 19.77 | 21.00 | -5.87% | 8.14% | 5.09% | 1 | 5 |
| TPCH(60) | TPCH-Q3 | parquet / none / none | 11.46 | 12.32 | -6.94% | 0.76% | 0.53% | 1 | 5 |
| TPCH(60) | TPCH-Q17 | parquet / none / none | 40.09 | 49.28 | -18.64% | 2.09% | 0.67% | 1 | 5 |
| TPCH(60) | TPCH-Q8 | parquet / none / none | 10.63 | 13.47 | I -21.08% | * 12.34% * | * 21.09% * | 1 | 5 |
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
+-------------------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+-------------------+-----------------------+---------+------------+------------+----------------+
| TARGETED-PERF(60) | parquet / none / none | 22.38 | -1.24% | 4.17 | +0.81% |
+-------------------+-----------------------+---------+------------+------------+----------------+
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Num Clients | Iters |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| TARGETED-PERF(60) | primitive_conjunct_ordering_1 | parquet / none / none | 0.12 | 0.10 | R +22.38% | 0.81% | * 27.26% * | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_decimal_highndv | parquet / none / none | 29.86 | 25.46 | +17.31% | 6.18% | 3.83% | 1 | 5 |
| TARGETED-PERF(60) | PERF_LIMIT-Q1 | parquet / none / none | 0.01 | 0.01 | +13.41% | * 15.35% * | 2.95% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_bigint_non_selective | parquet / none / none | 0.88 | 0.82 | +7.17% | 9.52% | 3.59% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_decimal_non_selective | parquet / none / none | 1.48 | 1.41 | +4.94% | 4.23% | 1.86% | 1 | 5 |
| TARGETED-PERF(60) | primitive_small_join_1 | parquet / none / none | 0.18 | 0.18 | +4.26% | * 11.92% * | 2.43% | 1 | 5 |
| TARGETED-PERF(60) | primitive_broadcast_join_3 | parquet / none / none | 7.29 | 7.03 | +3.77% | 5.98% | 9.35% | 1 | 5 |
| TARGETED-PERF(60) | primitive_exchange_broadcast | parquet / none / none | 38.41 | 37.02 | +3.77% | 8.59% | 1.31% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q6 | parquet / none / none | 1.93 | 1.89 | +2.14% | 2.22% | 1.75% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_2 | parquet / none / none | 7.26 | 7.17 | +1.29% | 2.28% | 4.54% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q1 | parquet / none / none | 2.79 | 2.75 | +1.28% | 0.52% | 0.76% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q3 | parquet / none / none | 3.51 | 3.47 | +1.01% | 0.63% | 0.57% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_string_selective | parquet / none / none | 1.05 | 1.04 | +0.76% | 3.03% | 2.40% | 1 | 5 |
| TARGETED-PERF(60) | primitive_orderby_bigint | parquet / none / none | 4.88 | 4.84 | +0.75% | 0.58% | 0.97% | 1 | 5 |
| TARGETED-PERF(60) | primitive_top-n_all | parquet / none / none | 38.56 | 38.28 | +0.73% | 0.20% | 0.24% | 1 | 5 |
| TARGETED-PERF(60) | primitive_orderby_all | parquet / none / none | 25.68 | 25.54 | +0.55% | 0.27% | 0.40% | 1 | 5 |
| TARGETED-PERF(60) | primitive_shuffle_join_union_all_with_groupby | parquet / none / none | 54.02 | 53.74 | +0.53% | 0.35% | 0.23% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q5 | parquet / none / none | 4.28 | 4.26 | +0.43% | 0.68% | 0.47% | 1 | 5 |
| TARGETED-PERF(60) | primitive_empty_build_join_1 | parquet / none / none | 16.25 | 16.19 | +0.42% | 0.33% | 0.42% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_bigint_highndv | parquet / none / none | 32.49 | 32.36 | +0.42% | 0.23% | 0.88% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q1 | parquet / none / none | 2.22 | 2.21 | +0.34% | 1.82% | 1.88% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_bigint_pk | parquet / none / none | 112.73 | 112.50 | +0.21% | 0.75% | 0.99% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q4 | parquet / none / none | 3.52 | 3.51 | +0.13% | 0.58% | 0.65% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q2 | parquet / none / none | 3.06 | 3.06 | +0.03% | 0.69% | 0.76% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_decimal_selective | parquet / none / none | 1.20 | 1.20 | -0.01% | 2.35% | 1.24% | 1 | 5 |
| TARGETED-PERF(60) | primitive_broadcast_join_2 | parquet / none / none | 4.27 | 4.27 | -0.03% | 0.52% | 0.48% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_decimal_lowndv.test | parquet / none / none | 3.87 | 3.87 | -0.07% | 1.69% | 1.63% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q7 | parquet / none / none | 1.92 | 1.93 | -0.28% | 2.33% | 1.94% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q5 | parquet / none / none | 0.48 | 0.48 | -0.28% | 0.59% | 0.53% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q4 | parquet / none / none | 17.48 | 17.53 | -0.30% | 0.43% | 0.58% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q7 | parquet / none / none | 7.87 | 7.90 | -0.35% | 0.67% | 0.55% | 1 | 5 |
| TARGETED-PERF(60) | primitive_exchange_shuffle | parquet / none / none | 74.25 | 74.53 | -0.37% | 0.57% | 0.36% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_bigint_lowndv | parquet / none / none | 3.81 | 3.82 | -0.42% | 1.51% | 1.10% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q2 | parquet / none / none | 9.93 | 10.00 | -0.67% | 0.77% | 0.67% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_string_like | parquet / none / none | 14.63 | 14.74 | -0.72% | 0.24% | 0.02% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_4 | parquet / none / none | 0.23 | 0.23 | -0.82% | 0.59% | 1.31% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q6 | parquet / none / none | 9.87 | 10.03 | -1.55% | 0.39% | 0.22% | 1 | 5 |
| TARGETED-PERF(60) | primitive_shuffle_join_one_to_many_string_with_groupby | parquet / none / none | 262.13 | 268.18 | -2.26% | 0.31% | 0.27% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_string_non_selective | parquet / none / none | 1.23 | 1.26 | -2.26% | 1.72% | 2.15% | 1 | 5 |
| TARGETED-PERF(60) | primitive_broadcast_join_1 | parquet / none / none | 2.04 | 2.09 | -2.54% | 0.31% | 2.88% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_3 | parquet / none / none | 0.13 | 0.13 | -3.13% | 0.73% | 2.50% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_bigint_selective | parquet / none / none | 0.12 | 0.12 | -3.15% | 1.03% | 1.73% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_5 | parquet / none / none | 14.11 | 14.60 | -3.33% | 2.03% | 2.43% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q3 | parquet / none / none | 8.28 | 8.64 | -4.17% | 0.79% | 1.08% | 1 | 5 |
| TARGETED-PERF(60) | primitive_long_predicate | parquet / none / none | 215.27 | 227.90 | -5.54% | 0.06% | 0.08% | 1 | 5 |
| TARGETED-PERF(60) | primitive_topn_bigint | parquet / none / none | 4.48 | 4.81 | -6.90% | 8.50% | * 15.79% * | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_in_predicate | parquet / none / none | 1.84 | 1.99 | -7.51% | 3.98% | 5.29% | 1 | 5 |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
Change-Id: Ida873ddb15e393b0bd37486db24add8a32f43ad0
Reviewed-on: http://gerrit.cloudera.org:8080/7974
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins
---
M .clang-tidy
M be/src/benchmarks/atoi-benchmark.cc
M be/src/benchmarks/scheduler-benchmark.cc
M be/src/codegen/codegen-symbol-emitter.cc
M be/src/codegen/instruction-counter-test.cc
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/common/compiler-util.h
M be/src/udf/udf.cc
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
11 files changed, 45 insertions(+), 32 deletions(-)
Approvals:
Impala Public Jenkins: Verified
Tim Armstrong: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/7974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
> Upgrade LLVM to 3.9.0 or later
> ------------------------------
>
> Key: IMPALA-5860
> URL: https://issues.apache.org/jira/browse/IMPALA-5860
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Fix For: Impala 2.11.0
>
>
> LLVM 3.9 has some useful features, including support for the [[nodiscard]] attribute (https://clang.llvm.org/cxx_status.html) and support for GCC ABI tags (http://releases.llvm.org/3.9.0/docs/ReleaseNotes.html#gcc-abi-tag), which is required to support building against code generated by GCC 5 and later with the C++11 ABI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)