You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org> on 2022/08/30 12:13:53 UTC
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT DOP
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18929
Change subject: IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT_DOP
......................................................................
IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT_DOP
Before IMPALA-9655 scan ranges were statically assigned to intra-node
fragment instances based on Longest-Processing Time algorithm:
https://github.com/apache/impala/blame/a7866a94578be6289bbac31686de4d9032ad9261/be/src/scheduling/scheduler.cc#L499-L501
From IMPALA-9655 we use dynamic intra-node load balancing for HDFS
scans. It means fragment instances have a shared queue of scan ranges
and the fragment instances grab the next scan range to be read from
thisqueue.
IMPALA-9655 got rid of the LPT-algorithm which means the scan ranges
are in a random order in the queue. This can lead to a skew if there
are large scan ranges at the end.
This patch mixes the above two approaches by using a priority queue
for the scan ranges, so each fragment instance would grab the largest
scan range in the queue. This could further mitigate intra-node skewing.
Ranges that are marked to use the hdfs cache are still handled with
higher priority.
The patch intoduces a new class called ScanRangeQueueMt which implements
the above.
Testing:
* added e2e test in which MIN(bytes_read) / MAX(bytes_read) is greater
then 0.6 with this patch. Earlier we could see it less than 0.4.
Performance:
No significant perf change. Ran TPCH (scale 30) with mt_dop set to 10 on
a 3 node minicluster on my desktop.
+----------+------------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+------------------------+---------+------------+------------+----------------+
| TPCH(30) | parquet / snap / block | 3.16 | -0.37% | 2.29 | +0.40% |
+----------+------------------------+---------+------------+------------+----------------+
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval | Tval |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| TPCH(30) | TPCH-Q20 | parquet / snap / block | 1.58 | 1.55 | +2.04% | 2.26% | 3.15% | 20 | +3.14% | 2.53 | 2.34 |
| TPCH(30) | TPCH-Q11 | parquet / snap / block | 0.68 | 0.66 | +2.94% | 3.67% | 4.18% | 20 | +0.90% | 1.97 | 2.33 |
| TPCH(30) | TPCH-Q5 | parquet / snap / block | 2.02 | 1.99 | +1.55% | 1.48% | 1.38% | 20 | +2.27% | 2.53 | 3.40 |
| TPCH(30) | TPCH-Q1 | parquet / snap / block | 2.59 | 2.56 | +1.22% | 3.25% | 3.12% | 20 | +1.75% | 1.33 | 1.21 |
| TPCH(30) | TPCH-Q15 | parquet / snap / block | 2.15 | 2.13 | +1.20% | 1.53% | 1.39% | 20 | +0.91% | 2.38 | 2.58 |
| TPCH(30) | TPCH-Q4 | parquet / snap / block | 1.26 | 1.24 | +1.62% | 2.33% | 2.77% | 20 | +0.36% | 2.23 | 1.98 |
| TPCH(30) | TPCH-Q6 | parquet / snap / block | 0.74 | 0.73 | +1.31% | 3.23% | 3.24% | 20 | +0.54% | 1.24 | 1.27 |
| TPCH(30) | TPCH-Q7 | parquet / snap / block | 2.13 | 2.11 | +1.03% | 1.84% | 1.71% | 20 | +0.52% | 1.85 | 1.83 |
| TPCH(30) | TPCH-Q19 | parquet / snap / block | 2.02 | 2.00 | +0.91% | 1.26% | 1.32% | 20 | +0.34% | 1.80 | 2.22 |
| TPCH(30) | TPCH-Q2 | parquet / snap / block | 1.42 | 1.41 | +0.95% | 2.54% | 1.58% | 20 | +0.07% | 0.42 | 1.41 |
| TPCH(30) | TPCH-Q16 | parquet / snap / block | 0.93 | 0.92 | +0.59% | 4.06% | 4.53% | 20 | +0.27% | 0.66 | 0.43 |
| TPCH(30) | TPCH-Q12 | parquet / snap / block | 1.57 | 1.56 | +0.68% | 1.98% | 2.34% | 20 | +0.14% | 1.04 | 0.98 |
| TPCH(30) | TPCH-Q21 | parquet / snap / block | 10.69 | 10.66 | +0.28% | 0.54% | 0.59% | 20 | +0.36% | 1.56 | 1.59 |
| TPCH(30) | TPCH-Q8 | parquet / snap / block | 2.30 | 2.29 | +0.50% | 1.38% | 1.57% | 20 | +0.09% | 1.12 | 1.06 |
| TPCH(30) | TPCH-Q22 | parquet / snap / block | 1.01 | 1.00 | +0.51% | 3.40% | 4.25% | 20 | +0.08% | 0.60 | 0.42 |
| TPCH(30) | TPCH-Q13 | parquet / snap / block | 4.66 | 4.66 | +0.08% | 1.00% | 0.78% | 20 | -0.02% | -0.25 | 0.29 |
| TPCH(30) | TPCH-Q18 | parquet / snap / block | 6.26 | 6.25 | +0.13% | 2.23% | 1.49% | 20 | -0.07% | -0.16 | 0.21 |
| TPCH(30) | TPCH-Q14 | parquet / snap / block | 1.68 | 1.69 | -0.51% | 2.51% | 3.81% | 20 | -0.05% | -0.31 | -0.51 |
| TPCH(30) | TPCH-Q3 | parquet / snap / block | 1.86 | 1.88 | -0.72% | 1.83% | 2.10% | 20 | -0.07% | -0.95 | -1.15 |
| TPCH(30) | TPCH-Q9 | parquet / snap / block | 8.84 | 9.02 | -1.93% | 1.78% | 2.55% | 20 | -2.27% | -2.50 | -2.79 |
| TPCH(30) | TPCH-Q17 | parquet / snap / block | 8.27 | 8.45 | -2.09% | 1.16% | 1.32% | 20 | -2.33% | -4.01 | -5.35 |
| TPCH(30) | TPCH-Q10 | parquet / snap / block | 4.95 | 5.13 | -3.60% | 6.86% | 9.75% | 20 | -2.53% | -2.70 | -1.37 |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
---
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
A be/src/exec/scan-range-queue-mt.h
M tests/query_test/test_scanners.py
4 files changed, 165 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18929/1
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 5: Code-Review+1
Carry +1 from Csaba and Gergely.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Aug 2022 14:07:58 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
Before IMPALA-9655 scan ranges were statically assigned to intra-node
fragment instances based on Longest-Processing Time algorithm:
https://github.com/apache/impala/blame/a7866a94578be6289bbac31686de4d9032ad9261/be/src/scheduling/scheduler.cc#L499-L501
From IMPALA-9655 we use dynamic intra-node load balancing for file
scans. It means fragment instances have a shared queue of scan ranges
and the fragment instances grab the next scan range to be read from
this queue.
IMPALA-9655 got rid of the LPT-algorithm which means the scan ranges
are in a random order in the queue. This can lead to a skew if there
are large scan ranges at the end.
This patch mixes the above two approaches by using a priority queue
for the scan ranges, so each fragment instance would grab the largest
scan range in the queue. This could further mitigate intra-node skewing.
Ranges that are marked to use the hdfs cache are still handled with
higher priority.
The patch intoduces a new class called ScanRangeQueueMt which implements
the above.
Testing:
* added e2e test in which MIN(bytes_read) / MAX(bytes_read) is greater
then 0.6 with this patch. Earlier we could see it less than 0.4.
Performance:
No significant perf change. Ran TPCH (scale 30) with mt_dop set to 10 on
a 3 node minicluster on my desktop.
+----------+------------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+------------------------+---------+------------+------------+----------------+
| TPCH(30) | parquet / snap / block | 3.16 | -0.37% | 2.29 | +0.40% |
+----------+------------------------+---------+------------+------------+----------------+
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval | Tval |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| TPCH(30) | TPCH-Q20 | parquet / snap / block | 1.58 | 1.55 | +2.04% | 2.26% | 3.15% | 20 | +3.14% | 2.53 | 2.34 |
| TPCH(30) | TPCH-Q11 | parquet / snap / block | 0.68 | 0.66 | +2.94% | 3.67% | 4.18% | 20 | +0.90% | 1.97 | 2.33 |
| TPCH(30) | TPCH-Q5 | parquet / snap / block | 2.02 | 1.99 | +1.55% | 1.48% | 1.38% | 20 | +2.27% | 2.53 | 3.40 |
| TPCH(30) | TPCH-Q1 | parquet / snap / block | 2.59 | 2.56 | +1.22% | 3.25% | 3.12% | 20 | +1.75% | 1.33 | 1.21 |
| TPCH(30) | TPCH-Q15 | parquet / snap / block | 2.15 | 2.13 | +1.20% | 1.53% | 1.39% | 20 | +0.91% | 2.38 | 2.58 |
| TPCH(30) | TPCH-Q4 | parquet / snap / block | 1.26 | 1.24 | +1.62% | 2.33% | 2.77% | 20 | +0.36% | 2.23 | 1.98 |
| TPCH(30) | TPCH-Q6 | parquet / snap / block | 0.74 | 0.73 | +1.31% | 3.23% | 3.24% | 20 | +0.54% | 1.24 | 1.27 |
| TPCH(30) | TPCH-Q7 | parquet / snap / block | 2.13 | 2.11 | +1.03% | 1.84% | 1.71% | 20 | +0.52% | 1.85 | 1.83 |
| TPCH(30) | TPCH-Q19 | parquet / snap / block | 2.02 | 2.00 | +0.91% | 1.26% | 1.32% | 20 | +0.34% | 1.80 | 2.22 |
| TPCH(30) | TPCH-Q2 | parquet / snap / block | 1.42 | 1.41 | +0.95% | 2.54% | 1.58% | 20 | +0.07% | 0.42 | 1.41 |
| TPCH(30) | TPCH-Q16 | parquet / snap / block | 0.93 | 0.92 | +0.59% | 4.06% | 4.53% | 20 | +0.27% | 0.66 | 0.43 |
| TPCH(30) | TPCH-Q12 | parquet / snap / block | 1.57 | 1.56 | +0.68% | 1.98% | 2.34% | 20 | +0.14% | 1.04 | 0.98 |
| TPCH(30) | TPCH-Q21 | parquet / snap / block | 10.69 | 10.66 | +0.28% | 0.54% | 0.59% | 20 | +0.36% | 1.56 | 1.59 |
| TPCH(30) | TPCH-Q8 | parquet / snap / block | 2.30 | 2.29 | +0.50% | 1.38% | 1.57% | 20 | +0.09% | 1.12 | 1.06 |
| TPCH(30) | TPCH-Q22 | parquet / snap / block | 1.01 | 1.00 | +0.51% | 3.40% | 4.25% | 20 | +0.08% | 0.60 | 0.42 |
| TPCH(30) | TPCH-Q13 | parquet / snap / block | 4.66 | 4.66 | +0.08% | 1.00% | 0.78% | 20 | -0.02% | -0.25 | 0.29 |
| TPCH(30) | TPCH-Q18 | parquet / snap / block | 6.26 | 6.25 | +0.13% | 2.23% | 1.49% | 20 | -0.07% | -0.16 | 0.21 |
| TPCH(30) | TPCH-Q14 | parquet / snap / block | 1.68 | 1.69 | -0.51% | 2.51% | 3.81% | 20 | -0.05% | -0.31 | -0.51 |
| TPCH(30) | TPCH-Q3 | parquet / snap / block | 1.86 | 1.88 | -0.72% | 1.83% | 2.10% | 20 | -0.07% | -0.95 | -1.15 |
| TPCH(30) | TPCH-Q9 | parquet / snap / block | 8.84 | 9.02 | -1.93% | 1.78% | 2.55% | 20 | -2.27% | -2.50 | -2.79 |
| TPCH(30) | TPCH-Q17 | parquet / snap / block | 8.27 | 8.45 | -2.09% | 1.16% | 1.32% | 20 | -2.33% | -4.01 | -5.35 |
| TPCH(30) | TPCH-Q10 | parquet / snap / block | 4.95 | 5.13 | -3.60% | 6.86% | 9.75% | 20 | -2.53% | -2.70 | -1.37 |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Reviewed-on: http://gerrit.cloudera.org:8080/18929
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
A be/src/exec/scan-range-queue-mt.h
M tests/query_test/test_scanners.py
4 files changed, 171 insertions(+), 16 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 10
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT_DOP
......................................................................
Patch Set 1:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py@368
PS1, Line 368: class TestHdfsScannerSkew(ImpalaTestSuite):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py@387
PS1, Line 387: l
flake8: E741 ambiguous variable name 'l'
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py@388
PS1, Line 388:
flake8: E261 at least two spaces before inline comment
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py@397
PS1, Line 397: =
flake8: E711 comparison to None should be 'if cond is None:'
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py@429
PS1, Line 429: /
flake8: E226 missing whitespace around arithmetic operator
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 30 Aug 2022 12:15:23 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 9:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8538/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 9
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Sep 2022 11:42:34 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello Gergely Fürnstáhl, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18929
to look at the new patch set (#6).
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
Before IMPALA-9655 scan ranges were statically assigned to intra-node
fragment instances based on Longest-Processing Time algorithm:
https://github.com/apache/impala/blame/a7866a94578be6289bbac31686de4d9032ad9261/be/src/scheduling/scheduler.cc#L499-L501
From IMPALA-9655 we use dynamic intra-node load balancing for file
scans. It means fragment instances have a shared queue of scan ranges
and the fragment instances grab the next scan range to be read from
this queue.
IMPALA-9655 got rid of the LPT-algorithm which means the scan ranges
are in a random order in the queue. This can lead to a skew if there
are large scan ranges at the end.
This patch mixes the above two approaches by using a priority queue
for the scan ranges, so each fragment instance would grab the largest
scan range in the queue. This could further mitigate intra-node skewing.
Ranges that are marked to use the hdfs cache are still handled with
higher priority.
The patch intoduces a new class called ScanRangeQueueMt which implements
the above.
Testing:
* added e2e test in which MIN(bytes_read) / MAX(bytes_read) is greater
then 0.6 with this patch. Earlier we could see it less than 0.4.
Performance:
No significant perf change. Ran TPCH (scale 30) with mt_dop set to 10 on
a 3 node minicluster on my desktop.
+----------+------------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+------------------------+---------+------------+------------+----------------+
| TPCH(30) | parquet / snap / block | 3.16 | -0.37% | 2.29 | +0.40% |
+----------+------------------------+---------+------------+------------+----------------+
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval | Tval |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| TPCH(30) | TPCH-Q20 | parquet / snap / block | 1.58 | 1.55 | +2.04% | 2.26% | 3.15% | 20 | +3.14% | 2.53 | 2.34 |
| TPCH(30) | TPCH-Q11 | parquet / snap / block | 0.68 | 0.66 | +2.94% | 3.67% | 4.18% | 20 | +0.90% | 1.97 | 2.33 |
| TPCH(30) | TPCH-Q5 | parquet / snap / block | 2.02 | 1.99 | +1.55% | 1.48% | 1.38% | 20 | +2.27% | 2.53 | 3.40 |
| TPCH(30) | TPCH-Q1 | parquet / snap / block | 2.59 | 2.56 | +1.22% | 3.25% | 3.12% | 20 | +1.75% | 1.33 | 1.21 |
| TPCH(30) | TPCH-Q15 | parquet / snap / block | 2.15 | 2.13 | +1.20% | 1.53% | 1.39% | 20 | +0.91% | 2.38 | 2.58 |
| TPCH(30) | TPCH-Q4 | parquet / snap / block | 1.26 | 1.24 | +1.62% | 2.33% | 2.77% | 20 | +0.36% | 2.23 | 1.98 |
| TPCH(30) | TPCH-Q6 | parquet / snap / block | 0.74 | 0.73 | +1.31% | 3.23% | 3.24% | 20 | +0.54% | 1.24 | 1.27 |
| TPCH(30) | TPCH-Q7 | parquet / snap / block | 2.13 | 2.11 | +1.03% | 1.84% | 1.71% | 20 | +0.52% | 1.85 | 1.83 |
| TPCH(30) | TPCH-Q19 | parquet / snap / block | 2.02 | 2.00 | +0.91% | 1.26% | 1.32% | 20 | +0.34% | 1.80 | 2.22 |
| TPCH(30) | TPCH-Q2 | parquet / snap / block | 1.42 | 1.41 | +0.95% | 2.54% | 1.58% | 20 | +0.07% | 0.42 | 1.41 |
| TPCH(30) | TPCH-Q16 | parquet / snap / block | 0.93 | 0.92 | +0.59% | 4.06% | 4.53% | 20 | +0.27% | 0.66 | 0.43 |
| TPCH(30) | TPCH-Q12 | parquet / snap / block | 1.57 | 1.56 | +0.68% | 1.98% | 2.34% | 20 | +0.14% | 1.04 | 0.98 |
| TPCH(30) | TPCH-Q21 | parquet / snap / block | 10.69 | 10.66 | +0.28% | 0.54% | 0.59% | 20 | +0.36% | 1.56 | 1.59 |
| TPCH(30) | TPCH-Q8 | parquet / snap / block | 2.30 | 2.29 | +0.50% | 1.38% | 1.57% | 20 | +0.09% | 1.12 | 1.06 |
| TPCH(30) | TPCH-Q22 | parquet / snap / block | 1.01 | 1.00 | +0.51% | 3.40% | 4.25% | 20 | +0.08% | 0.60 | 0.42 |
| TPCH(30) | TPCH-Q13 | parquet / snap / block | 4.66 | 4.66 | +0.08% | 1.00% | 0.78% | 20 | -0.02% | -0.25 | 0.29 |
| TPCH(30) | TPCH-Q18 | parquet / snap / block | 6.26 | 6.25 | +0.13% | 2.23% | 1.49% | 20 | -0.07% | -0.16 | 0.21 |
| TPCH(30) | TPCH-Q14 | parquet / snap / block | 1.68 | 1.69 | -0.51% | 2.51% | 3.81% | 20 | -0.05% | -0.31 | -0.51 |
| TPCH(30) | TPCH-Q3 | parquet / snap / block | 1.86 | 1.88 | -0.72% | 1.83% | 2.10% | 20 | -0.07% | -0.95 | -1.15 |
| TPCH(30) | TPCH-Q9 | parquet / snap / block | 8.84 | 9.02 | -1.93% | 1.78% | 2.55% | 20 | -2.27% | -2.50 | -2.79 |
| TPCH(30) | TPCH-Q17 | parquet / snap / block | 8.27 | 8.45 | -2.09% | 1.16% | 1.32% | 20 | -2.33% | -4.01 | -5.35 |
| TPCH(30) | TPCH-Q10 | parquet / snap / block | 4.95 | 5.13 | -3.60% | 6.86% | 9.75% | 20 | -2.53% | -2.70 | -1.37 |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
---
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
A be/src/exec/scan-range-queue-mt.h
M tests/query_test/test_scanners.py
4 files changed, 174 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18929/6
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11298/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Sep 2022 12:01:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 9: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 9
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Sep 2022 11:42:33 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11269/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Aug 2022 14:24:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT DOP
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT_DOP
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py@419
PS3, Line 419: '
> nit: Lets
oops, I was wrong - let's comes from "let us"
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 30 Aug 2022 16:11:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 3:
(14 comments)
Thanks for the comments!
http://gerrit.cloudera.org:8080/#/c/18929/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18929/3//COMMIT_MSG@13
PS3, Line 13: for HDFS
> not just HDFS, right? do we also use the same logic for files in cloud stor
Yeah, I just copied the title here, and also used HDFS scans in the title of this CR. Though I see that it can be confusing so I changed it to 'file scans'.
http://gerrit.cloudera.org:8080/#/c/18929/3//COMMIT_MSG@16
PS3, Line 16: thisqueue.
> typo
Done
http://gerrit.cloudera.org:8080/#/c/18929/3//COMMIT_MSG@25
PS3, Line 25: Ranges that are marked to use the hdfs cache are still handled with
: higher priority.
> Probably out of scope for this patch, but data cache / file handle cache co
File handle cache: it would require a more invasive change plus testing, so might want to deal with it in another patch.
Remote data cache: we have the file splits here, not the actual ranges that we need to read, i.e. it's not easy to tell if the data is cached or not. We can only assume that every file split scheduled here is potentially cached.
About scans that doesn't fit the data cache: maybe we could use a different scheduling policy. I guess it causes a problem when the same query is executed repeatedly. It can harm benchmark results, but I don't know if it actually happens in real-world use cases. Again, we see the file splits here, and if only a subset of the columns are read then it's hard to tell the actual size of the scans.
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h
File be/src/exec/scan-range-queue-mt.h:
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h@30
PS3, Line 30: Only used for MT scans where the scan ranges are dynamically assigned
: /// to the fragment instances using this queue.
> Why is it treated differently than the MT_DOP=0 case? I would assume that s
The MT_DOP=0 uses different methods and I didn't want to change it in this CR as it would require a big refactor. Maybe we can unify the two in the future.
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h@39
PS3, Line 39: at_front
> typo: high_prio
Done
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h@75
PS3, Line 75: scan_range_queue_.Reserve(capacity);
> This should also take the lock IMO, as reserve can cause the underlying vec
Done
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h@83
PS3, Line 83: if (!lhs->UseHdfsCache() && rhs->UseHdfsCache()) return true;
> Shouldn't we return false in the opposite case?
Good catch, done.
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py@368
PS1, Line 368:
> flake8: E302 expected 2 blank lines, found 1
Done
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py@387
PS1, Line 387: (
> flake8: E741 ambiguous variable name 'l'
Done
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py@388
PS1, Line 388: f
> flake8: E261 at least two spaces before inline comment
Done
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py@397
PS1, Line 397: a
> flake8: E711 comparison to None should be 'if cond is None:'
Done
http://gerrit.cloudera.org:8080/#/c/18929/1/tests/query_test/test_scanners.py@429
PS1, Line 429: b
> flake8: E226 missing whitespace around arithmetic operator
Done
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py@378
PS3, Line 378: 'text', 'parquet
> Does it add to the coverage if we execute with both fileformats? My guess w
Interestingly I got similar min/max ratios with text and parquet. I was thinking that it's good to run the tests on different file formats, but I can limit the test to 'text' if you want.
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py@419
PS3, Line 419: '
> oops, I was wrong - let's comes from "let us"
Done
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Aug 2022 11:29:08 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 4: Code-Review+1
(3 comments)
http://gerrit.cloudera.org:8080/#/c/18929/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18929/3//COMMIT_MSG@25
PS3, Line 25: Ranges that are marked to use the hdfs cache are still handled with
: higher priority.
> File handle cache: it would require a more invasive change plus testing, so
Data cache:
If we could get the number of bytes from a file in data cache, it could be subtracted from the size during the ordering.
This would probably also help with the file handle cache, as files with lot of chunks in data cache are more likely to be also in the file handle cache.
Note that this is just a brain dump, I wouldn't complicate this change with it.
Also realized that while a fix order for files can potentially cause regression in full scans not fitting to cache, it can also improve things for other queries: if not all files are read in the end (the query has LIMIT), then it is better for caching to not randomize the order.
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h
File be/src/exec/scan-range-queue-mt.h:
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h@30
PS3, Line 30: Only used for MT scans where the scan ranges are dynamically assigned
: /// to the fragment instances using this queue.
> The MT_DOP=0 uses different methods and I didn't want to change it in this
ok
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py@378
PS3, Line 378: 'text', 'parquet
> Interestingly I got similar min/max ratios with text and parquet. I was thi
It is ok like this, maybe we can look into it deeper if the test turn out to be flaky.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Aug 2022 11:45:13 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 4:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/11266/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Aug 2022 11:35:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 8: Code-Review+2
Will create a Jira about different ideas to improve scheduling.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 8
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Sep 2022 11:36:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 9: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 9
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Sep 2022 16:28:36 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 8:
(1 comment)
Good to see you again, Qifan! Thanks for the suggestions, probably it would be the best to track these ideas in Jira.
I fixed a test error in the latest patch.
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py@378
PS3, Line 378: 'text'))
> It is ok like this, maybe we can look into it deeper if the test turn out t
Sorry I had a test error as the tests used a text table from my local database.
Parquet scanning is still improved by this test but I get different ratios and higher variance, so I'm limiting the test to 'text' to make it more stable, as you originally suggested.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 8
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Sep 2022 10:51:52 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 8:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11306/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 8
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Sep 2022 11:01:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello Gergely Fürnstáhl, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18929
to look at the new patch set (#5).
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
Before IMPALA-9655 scan ranges were statically assigned to intra-node
fragment instances based on Longest-Processing Time algorithm:
https://github.com/apache/impala/blame/a7866a94578be6289bbac31686de4d9032ad9261/be/src/scheduling/scheduler.cc#L499-L501
From IMPALA-9655 we use dynamic intra-node load balancing for file
scans. It means fragment instances have a shared queue of scan ranges
and the fragment instances grab the next scan range to be read from
this queue.
IMPALA-9655 got rid of the LPT-algorithm which means the scan ranges
are in a random order in the queue. This can lead to a skew if there
are large scan ranges at the end.
This patch mixes the above two approaches by using a priority queue
for the scan ranges, so each fragment instance would grab the largest
scan range in the queue. This could further mitigate intra-node skewing.
Ranges that are marked to use the hdfs cache are still handled with
higher priority.
The patch intoduces a new class called ScanRangeQueueMt which implements
the above.
Testing:
* added e2e test in which MIN(bytes_read) / MAX(bytes_read) is greater
then 0.6 with this patch. Earlier we could see it less than 0.4.
Performance:
No significant perf change. Ran TPCH (scale 30) with mt_dop set to 10 on
a 3 node minicluster on my desktop.
+----------+------------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+------------------------+---------+------------+------------+----------------+
| TPCH(30) | parquet / snap / block | 3.16 | -0.37% | 2.29 | +0.40% |
+----------+------------------------+---------+------------+------------+----------------+
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval | Tval |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| TPCH(30) | TPCH-Q20 | parquet / snap / block | 1.58 | 1.55 | +2.04% | 2.26% | 3.15% | 20 | +3.14% | 2.53 | 2.34 |
| TPCH(30) | TPCH-Q11 | parquet / snap / block | 0.68 | 0.66 | +2.94% | 3.67% | 4.18% | 20 | +0.90% | 1.97 | 2.33 |
| TPCH(30) | TPCH-Q5 | parquet / snap / block | 2.02 | 1.99 | +1.55% | 1.48% | 1.38% | 20 | +2.27% | 2.53 | 3.40 |
| TPCH(30) | TPCH-Q1 | parquet / snap / block | 2.59 | 2.56 | +1.22% | 3.25% | 3.12% | 20 | +1.75% | 1.33 | 1.21 |
| TPCH(30) | TPCH-Q15 | parquet / snap / block | 2.15 | 2.13 | +1.20% | 1.53% | 1.39% | 20 | +0.91% | 2.38 | 2.58 |
| TPCH(30) | TPCH-Q4 | parquet / snap / block | 1.26 | 1.24 | +1.62% | 2.33% | 2.77% | 20 | +0.36% | 2.23 | 1.98 |
| TPCH(30) | TPCH-Q6 | parquet / snap / block | 0.74 | 0.73 | +1.31% | 3.23% | 3.24% | 20 | +0.54% | 1.24 | 1.27 |
| TPCH(30) | TPCH-Q7 | parquet / snap / block | 2.13 | 2.11 | +1.03% | 1.84% | 1.71% | 20 | +0.52% | 1.85 | 1.83 |
| TPCH(30) | TPCH-Q19 | parquet / snap / block | 2.02 | 2.00 | +0.91% | 1.26% | 1.32% | 20 | +0.34% | 1.80 | 2.22 |
| TPCH(30) | TPCH-Q2 | parquet / snap / block | 1.42 | 1.41 | +0.95% | 2.54% | 1.58% | 20 | +0.07% | 0.42 | 1.41 |
| TPCH(30) | TPCH-Q16 | parquet / snap / block | 0.93 | 0.92 | +0.59% | 4.06% | 4.53% | 20 | +0.27% | 0.66 | 0.43 |
| TPCH(30) | TPCH-Q12 | parquet / snap / block | 1.57 | 1.56 | +0.68% | 1.98% | 2.34% | 20 | +0.14% | 1.04 | 0.98 |
| TPCH(30) | TPCH-Q21 | parquet / snap / block | 10.69 | 10.66 | +0.28% | 0.54% | 0.59% | 20 | +0.36% | 1.56 | 1.59 |
| TPCH(30) | TPCH-Q8 | parquet / snap / block | 2.30 | 2.29 | +0.50% | 1.38% | 1.57% | 20 | +0.09% | 1.12 | 1.06 |
| TPCH(30) | TPCH-Q22 | parquet / snap / block | 1.01 | 1.00 | +0.51% | 3.40% | 4.25% | 20 | +0.08% | 0.60 | 0.42 |
| TPCH(30) | TPCH-Q13 | parquet / snap / block | 4.66 | 4.66 | +0.08% | 1.00% | 0.78% | 20 | -0.02% | -0.25 | 0.29 |
| TPCH(30) | TPCH-Q18 | parquet / snap / block | 6.26 | 6.25 | +0.13% | 2.23% | 1.49% | 20 | -0.07% | -0.16 | 0.21 |
| TPCH(30) | TPCH-Q14 | parquet / snap / block | 1.68 | 1.69 | -0.51% | 2.51% | 3.81% | 20 | -0.05% | -0.31 | -0.51 |
| TPCH(30) | TPCH-Q3 | parquet / snap / block | 1.86 | 1.88 | -0.72% | 1.83% | 2.10% | 20 | -0.07% | -0.95 | -1.15 |
| TPCH(30) | TPCH-Q9 | parquet / snap / block | 8.84 | 9.02 | -1.93% | 1.78% | 2.55% | 20 | -2.27% | -2.50 | -2.79 |
| TPCH(30) | TPCH-Q17 | parquet / snap / block | 8.27 | 8.45 | -2.09% | 1.16% | 1.32% | 20 | -2.33% | -4.01 | -5.35 |
| TPCH(30) | TPCH-Q10 | parquet / snap / block | 4.95 | 5.13 | -3.60% | 6.86% | 9.75% | 20 | -2.53% | -2.70 | -1.37 |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
---
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
A be/src/exec/scan-range-queue-mt.h
M tests/query_test/test_scanners.py
4 files changed, 170 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18929/5
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT DOP
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT_DOP
......................................................................
Patch Set 3:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/18929/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18929/3//COMMIT_MSG@13
PS3, Line 13: for HDFS
not just HDFS, right? do we also use the same logic for files in cloud storage?
http://gerrit.cloudera.org:8080/#/c/18929/3//COMMIT_MSG@16
PS3, Line 16: thisqueue.
typo
http://gerrit.cloudera.org:8080/#/c/18929/3//COMMIT_MSG@25
PS3, Line 25: Ranges that are marked to use the hdfs cache are still handled with
: higher priority.
Probably out of scope for this patch, but data cache / file handle cache could be also considered, as postponing the processing of cached files increases the chance of being evicted in the meantime.
e.g. if files have a similar length, then this algorithm creates a fix order for them, which is kind of worst case if the scan doesn't fit to the cache (assuming LRU cache).
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h
File be/src/exec/scan-range-queue-mt.h:
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h@30
PS3, Line 30: Only used for MT scans where the scan ranges are dynamically assigned
: /// to the fragment instances using this queue.
Why is it treated differently than the MT_DOP=0 case? I would assume that scanner threads can have similar issues with skew - the only difference is that that number of scanner threads is not fixed, so we cannot do a static assignment, but this algorithm would work without issue IMO.
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h@75
PS3, Line 75: scan_range_queue_.Reserve(capacity);
This should also take the lock IMO, as reserve can cause the underlying vector to be moved.
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h@83
PS3, Line 83: if (!lhs->UseHdfsCache() && rhs->UseHdfsCache()) return true;
Shouldn't we return false in the opposite case?
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py@378
PS3, Line 378: 'text', 'parquet
Does it add to the coverage if we execute with both fileformats? My guess would be is that text is more reliably, as the size should be linear to the number of rows.
http://gerrit.cloudera.org:8080/#/c/18929/3/tests/query_test/test_scanners.py@419
PS3, Line 419: '
nit: Lets
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 30 Aug 2022 13:05:44 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Gergely Fürnstáhl (Code Review)" <ge...@cloudera.org>.
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 4: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Aug 2022 13:31:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT DOP
Posted by "Gergely Fürnstáhl (Code Review)" <ge...@cloudera.org>.
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT_DOP
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h
File be/src/exec/scan-range-queue-mt.h:
http://gerrit.cloudera.org:8080/#/c/18929/3/be/src/exec/scan-range-queue-mt.h@39
PS3, Line 39: at_front
typo: high_prio
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 30 Aug 2022 13:25:32 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT_DOP
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11254/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 30 Aug 2022 12:36:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT_DOP
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11255/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 30 Aug 2022 12:38:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello Gergely Fürnstáhl, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18929
to look at the new patch set (#4).
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
Before IMPALA-9655 scan ranges were statically assigned to intra-node
fragment instances based on Longest-Processing Time algorithm:
https://github.com/apache/impala/blame/a7866a94578be6289bbac31686de4d9032ad9261/be/src/scheduling/scheduler.cc#L499-L501
From IMPALA-9655 we use dynamic intra-node load balancing for file
scans. It means fragment instances have a shared queue of scan ranges
and the fragment instances grab the next scan range to be read from
this queue.
IMPALA-9655 got rid of the LPT-algorithm which means the scan ranges
are in a random order in the queue. This can lead to a skew if there
are large scan ranges at the end.
This patch mixes the above two approaches by using a priority queue
for the scan ranges, so each fragment instance would grab the largest
scan range in the queue. This could further mitigate intra-node skewing.
Ranges that are marked to use the hdfs cache are still handled with
higher priority.
The patch intoduces a new class called ScanRangeQueueMt which implements
the above.
Testing:
* added e2e test in which MIN(bytes_read) / MAX(bytes_read) is greater
then 0.6 with this patch. Earlier we could see it less than 0.4.
Performance:
No significant perf change. Ran TPCH (scale 30) with mt_dop set to 10 on
a 3 node minicluster on my desktop.
+----------+------------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+------------------------+---------+------------+------------+----------------+
| TPCH(30) | parquet / snap / block | 3.16 | -0.37% | 2.29 | +0.40% |
+----------+------------------------+---------+------------+------------+----------------+
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval | Tval |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| TPCH(30) | TPCH-Q20 | parquet / snap / block | 1.58 | 1.55 | +2.04% | 2.26% | 3.15% | 20 | +3.14% | 2.53 | 2.34 |
| TPCH(30) | TPCH-Q11 | parquet / snap / block | 0.68 | 0.66 | +2.94% | 3.67% | 4.18% | 20 | +0.90% | 1.97 | 2.33 |
| TPCH(30) | TPCH-Q5 | parquet / snap / block | 2.02 | 1.99 | +1.55% | 1.48% | 1.38% | 20 | +2.27% | 2.53 | 3.40 |
| TPCH(30) | TPCH-Q1 | parquet / snap / block | 2.59 | 2.56 | +1.22% | 3.25% | 3.12% | 20 | +1.75% | 1.33 | 1.21 |
| TPCH(30) | TPCH-Q15 | parquet / snap / block | 2.15 | 2.13 | +1.20% | 1.53% | 1.39% | 20 | +0.91% | 2.38 | 2.58 |
| TPCH(30) | TPCH-Q4 | parquet / snap / block | 1.26 | 1.24 | +1.62% | 2.33% | 2.77% | 20 | +0.36% | 2.23 | 1.98 |
| TPCH(30) | TPCH-Q6 | parquet / snap / block | 0.74 | 0.73 | +1.31% | 3.23% | 3.24% | 20 | +0.54% | 1.24 | 1.27 |
| TPCH(30) | TPCH-Q7 | parquet / snap / block | 2.13 | 2.11 | +1.03% | 1.84% | 1.71% | 20 | +0.52% | 1.85 | 1.83 |
| TPCH(30) | TPCH-Q19 | parquet / snap / block | 2.02 | 2.00 | +0.91% | 1.26% | 1.32% | 20 | +0.34% | 1.80 | 2.22 |
| TPCH(30) | TPCH-Q2 | parquet / snap / block | 1.42 | 1.41 | +0.95% | 2.54% | 1.58% | 20 | +0.07% | 0.42 | 1.41 |
| TPCH(30) | TPCH-Q16 | parquet / snap / block | 0.93 | 0.92 | +0.59% | 4.06% | 4.53% | 20 | +0.27% | 0.66 | 0.43 |
| TPCH(30) | TPCH-Q12 | parquet / snap / block | 1.57 | 1.56 | +0.68% | 1.98% | 2.34% | 20 | +0.14% | 1.04 | 0.98 |
| TPCH(30) | TPCH-Q21 | parquet / snap / block | 10.69 | 10.66 | +0.28% | 0.54% | 0.59% | 20 | +0.36% | 1.56 | 1.59 |
| TPCH(30) | TPCH-Q8 | parquet / snap / block | 2.30 | 2.29 | +0.50% | 1.38% | 1.57% | 20 | +0.09% | 1.12 | 1.06 |
| TPCH(30) | TPCH-Q22 | parquet / snap / block | 1.01 | 1.00 | +0.51% | 3.40% | 4.25% | 20 | +0.08% | 0.60 | 0.42 |
| TPCH(30) | TPCH-Q13 | parquet / snap / block | 4.66 | 4.66 | +0.08% | 1.00% | 0.78% | 20 | -0.02% | -0.25 | 0.29 |
| TPCH(30) | TPCH-Q18 | parquet / snap / block | 6.26 | 6.25 | +0.13% | 2.23% | 1.49% | 20 | -0.07% | -0.16 | 0.21 |
| TPCH(30) | TPCH-Q14 | parquet / snap / block | 1.68 | 1.69 | -0.51% | 2.51% | 3.81% | 20 | -0.05% | -0.31 | -0.51 |
| TPCH(30) | TPCH-Q3 | parquet / snap / block | 1.86 | 1.88 | -0.72% | 1.83% | 2.10% | 20 | -0.07% | -0.95 | -1.15 |
| TPCH(30) | TPCH-Q9 | parquet / snap / block | 8.84 | 9.02 | -1.93% | 1.78% | 2.55% | 20 | -2.27% | -2.50 | -2.79 |
| TPCH(30) | TPCH-Q17 | parquet / snap / block | 8.27 | 8.45 | -2.09% | 1.16% | 1.32% | 20 | -2.33% | -4.01 | -5.35 |
| TPCH(30) | TPCH-Q10 | parquet / snap / block | 4.95 | 5.13 | -3.60% | 6.86% | 9.75% | 20 | -2.53% | -2.70 | -1.37 |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
---
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
A be/src/exec/scan-range-queue-mt.h
M tests/query_test/test_scanners.py
4 files changed, 170 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18929/4
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 6: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Sep 2022 12:54:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello Qifan Chen, Gergely Fürnstáhl, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18929
to look at the new patch set (#8).
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
Before IMPALA-9655 scan ranges were statically assigned to intra-node
fragment instances based on Longest-Processing Time algorithm:
https://github.com/apache/impala/blame/a7866a94578be6289bbac31686de4d9032ad9261/be/src/scheduling/scheduler.cc#L499-L501
From IMPALA-9655 we use dynamic intra-node load balancing for file
scans. It means fragment instances have a shared queue of scan ranges
and the fragment instances grab the next scan range to be read from
this queue.
IMPALA-9655 got rid of the LPT-algorithm which means the scan ranges
are in a random order in the queue. This can lead to a skew if there
are large scan ranges at the end.
This patch mixes the above two approaches by using a priority queue
for the scan ranges, so each fragment instance would grab the largest
scan range in the queue. This could further mitigate intra-node skewing.
Ranges that are marked to use the hdfs cache are still handled with
higher priority.
The patch intoduces a new class called ScanRangeQueueMt which implements
the above.
Testing:
* added e2e test in which MIN(bytes_read) / MAX(bytes_read) is greater
then 0.6 with this patch. Earlier we could see it less than 0.4.
Performance:
No significant perf change. Ran TPCH (scale 30) with mt_dop set to 10 on
a 3 node minicluster on my desktop.
+----------+------------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+------------------------+---------+------------+------------+----------------+
| TPCH(30) | parquet / snap / block | 3.16 | -0.37% | 2.29 | +0.40% |
+----------+------------------------+---------+------------+------------+----------------+
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval | Tval |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| TPCH(30) | TPCH-Q20 | parquet / snap / block | 1.58 | 1.55 | +2.04% | 2.26% | 3.15% | 20 | +3.14% | 2.53 | 2.34 |
| TPCH(30) | TPCH-Q11 | parquet / snap / block | 0.68 | 0.66 | +2.94% | 3.67% | 4.18% | 20 | +0.90% | 1.97 | 2.33 |
| TPCH(30) | TPCH-Q5 | parquet / snap / block | 2.02 | 1.99 | +1.55% | 1.48% | 1.38% | 20 | +2.27% | 2.53 | 3.40 |
| TPCH(30) | TPCH-Q1 | parquet / snap / block | 2.59 | 2.56 | +1.22% | 3.25% | 3.12% | 20 | +1.75% | 1.33 | 1.21 |
| TPCH(30) | TPCH-Q15 | parquet / snap / block | 2.15 | 2.13 | +1.20% | 1.53% | 1.39% | 20 | +0.91% | 2.38 | 2.58 |
| TPCH(30) | TPCH-Q4 | parquet / snap / block | 1.26 | 1.24 | +1.62% | 2.33% | 2.77% | 20 | +0.36% | 2.23 | 1.98 |
| TPCH(30) | TPCH-Q6 | parquet / snap / block | 0.74 | 0.73 | +1.31% | 3.23% | 3.24% | 20 | +0.54% | 1.24 | 1.27 |
| TPCH(30) | TPCH-Q7 | parquet / snap / block | 2.13 | 2.11 | +1.03% | 1.84% | 1.71% | 20 | +0.52% | 1.85 | 1.83 |
| TPCH(30) | TPCH-Q19 | parquet / snap / block | 2.02 | 2.00 | +0.91% | 1.26% | 1.32% | 20 | +0.34% | 1.80 | 2.22 |
| TPCH(30) | TPCH-Q2 | parquet / snap / block | 1.42 | 1.41 | +0.95% | 2.54% | 1.58% | 20 | +0.07% | 0.42 | 1.41 |
| TPCH(30) | TPCH-Q16 | parquet / snap / block | 0.93 | 0.92 | +0.59% | 4.06% | 4.53% | 20 | +0.27% | 0.66 | 0.43 |
| TPCH(30) | TPCH-Q12 | parquet / snap / block | 1.57 | 1.56 | +0.68% | 1.98% | 2.34% | 20 | +0.14% | 1.04 | 0.98 |
| TPCH(30) | TPCH-Q21 | parquet / snap / block | 10.69 | 10.66 | +0.28% | 0.54% | 0.59% | 20 | +0.36% | 1.56 | 1.59 |
| TPCH(30) | TPCH-Q8 | parquet / snap / block | 2.30 | 2.29 | +0.50% | 1.38% | 1.57% | 20 | +0.09% | 1.12 | 1.06 |
| TPCH(30) | TPCH-Q22 | parquet / snap / block | 1.01 | 1.00 | +0.51% | 3.40% | 4.25% | 20 | +0.08% | 0.60 | 0.42 |
| TPCH(30) | TPCH-Q13 | parquet / snap / block | 4.66 | 4.66 | +0.08% | 1.00% | 0.78% | 20 | -0.02% | -0.25 | 0.29 |
| TPCH(30) | TPCH-Q18 | parquet / snap / block | 6.26 | 6.25 | +0.13% | 2.23% | 1.49% | 20 | -0.07% | -0.16 | 0.21 |
| TPCH(30) | TPCH-Q14 | parquet / snap / block | 1.68 | 1.69 | -0.51% | 2.51% | 3.81% | 20 | -0.05% | -0.31 | -0.51 |
| TPCH(30) | TPCH-Q3 | parquet / snap / block | 1.86 | 1.88 | -0.72% | 1.83% | 2.10% | 20 | -0.07% | -0.95 | -1.15 |
| TPCH(30) | TPCH-Q9 | parquet / snap / block | 8.84 | 9.02 | -1.93% | 1.78% | 2.55% | 20 | -2.27% | -2.50 | -2.79 |
| TPCH(30) | TPCH-Q17 | parquet / snap / block | 8.27 | 8.45 | -2.09% | 1.16% | 1.32% | 20 | -2.33% | -4.01 | -5.35 |
| TPCH(30) | TPCH-Q10 | parquet / snap / block | 4.95 | 5.13 | -3.60% | 6.86% | 9.75% | 20 | -2.53% | -2.70 | -1.37 |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
---
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
A be/src/exec/scan-range-queue-mt.h
M tests/query_test/test_scanners.py
4 files changed, 171 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18929/8
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 8
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 7:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8532/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Sep 2022 13:11:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 7: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Sep 2022 13:11:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT DOP
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18929
to look at the new patch set (#3).
Change subject: IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT_DOP
......................................................................
IMPALA-11539: Mitigate intra-node skew of HDFS scans with MT_DOP
Before IMPALA-9655 scan ranges were statically assigned to intra-node
fragment instances based on Longest-Processing Time algorithm:
https://github.com/apache/impala/blame/a7866a94578be6289bbac31686de4d9032ad9261/be/src/scheduling/scheduler.cc#L499-L501
From IMPALA-9655 we use dynamic intra-node load balancing for HDFS
scans. It means fragment instances have a shared queue of scan ranges
and the fragment instances grab the next scan range to be read from
thisqueue.
IMPALA-9655 got rid of the LPT-algorithm which means the scan ranges
are in a random order in the queue. This can lead to a skew if there
are large scan ranges at the end.
This patch mixes the above two approaches by using a priority queue
for the scan ranges, so each fragment instance would grab the largest
scan range in the queue. This could further mitigate intra-node skewing.
Ranges that are marked to use the hdfs cache are still handled with
higher priority.
The patch intoduces a new class called ScanRangeQueueMt which implements
the above.
Testing:
* added e2e test in which MIN(bytes_read) / MAX(bytes_read) is greater
then 0.6 with this patch. Earlier we could see it less than 0.4.
Performance:
No significant perf change. Ran TPCH (scale 30) with mt_dop set to 10 on
a 3 node minicluster on my desktop.
+----------+------------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+------------------------+---------+------------+------------+----------------+
| TPCH(30) | parquet / snap / block | 3.16 | -0.37% | 2.29 | +0.40% |
+----------+------------------------+---------+------------+------------+----------------+
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval | Tval |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
| TPCH(30) | TPCH-Q20 | parquet / snap / block | 1.58 | 1.55 | +2.04% | 2.26% | 3.15% | 20 | +3.14% | 2.53 | 2.34 |
| TPCH(30) | TPCH-Q11 | parquet / snap / block | 0.68 | 0.66 | +2.94% | 3.67% | 4.18% | 20 | +0.90% | 1.97 | 2.33 |
| TPCH(30) | TPCH-Q5 | parquet / snap / block | 2.02 | 1.99 | +1.55% | 1.48% | 1.38% | 20 | +2.27% | 2.53 | 3.40 |
| TPCH(30) | TPCH-Q1 | parquet / snap / block | 2.59 | 2.56 | +1.22% | 3.25% | 3.12% | 20 | +1.75% | 1.33 | 1.21 |
| TPCH(30) | TPCH-Q15 | parquet / snap / block | 2.15 | 2.13 | +1.20% | 1.53% | 1.39% | 20 | +0.91% | 2.38 | 2.58 |
| TPCH(30) | TPCH-Q4 | parquet / snap / block | 1.26 | 1.24 | +1.62% | 2.33% | 2.77% | 20 | +0.36% | 2.23 | 1.98 |
| TPCH(30) | TPCH-Q6 | parquet / snap / block | 0.74 | 0.73 | +1.31% | 3.23% | 3.24% | 20 | +0.54% | 1.24 | 1.27 |
| TPCH(30) | TPCH-Q7 | parquet / snap / block | 2.13 | 2.11 | +1.03% | 1.84% | 1.71% | 20 | +0.52% | 1.85 | 1.83 |
| TPCH(30) | TPCH-Q19 | parquet / snap / block | 2.02 | 2.00 | +0.91% | 1.26% | 1.32% | 20 | +0.34% | 1.80 | 2.22 |
| TPCH(30) | TPCH-Q2 | parquet / snap / block | 1.42 | 1.41 | +0.95% | 2.54% | 1.58% | 20 | +0.07% | 0.42 | 1.41 |
| TPCH(30) | TPCH-Q16 | parquet / snap / block | 0.93 | 0.92 | +0.59% | 4.06% | 4.53% | 20 | +0.27% | 0.66 | 0.43 |
| TPCH(30) | TPCH-Q12 | parquet / snap / block | 1.57 | 1.56 | +0.68% | 1.98% | 2.34% | 20 | +0.14% | 1.04 | 0.98 |
| TPCH(30) | TPCH-Q21 | parquet / snap / block | 10.69 | 10.66 | +0.28% | 0.54% | 0.59% | 20 | +0.36% | 1.56 | 1.59 |
| TPCH(30) | TPCH-Q8 | parquet / snap / block | 2.30 | 2.29 | +0.50% | 1.38% | 1.57% | 20 | +0.09% | 1.12 | 1.06 |
| TPCH(30) | TPCH-Q22 | parquet / snap / block | 1.01 | 1.00 | +0.51% | 3.40% | 4.25% | 20 | +0.08% | 0.60 | 0.42 |
| TPCH(30) | TPCH-Q13 | parquet / snap / block | 4.66 | 4.66 | +0.08% | 1.00% | 0.78% | 20 | -0.02% | -0.25 | 0.29 |
| TPCH(30) | TPCH-Q18 | parquet / snap / block | 6.26 | 6.25 | +0.13% | 2.23% | 1.49% | 20 | -0.07% | -0.16 | 0.21 |
| TPCH(30) | TPCH-Q14 | parquet / snap / block | 1.68 | 1.69 | -0.51% | 2.51% | 3.81% | 20 | -0.05% | -0.31 | -0.51 |
| TPCH(30) | TPCH-Q3 | parquet / snap / block | 1.86 | 1.88 | -0.72% | 1.83% | 2.10% | 20 | -0.07% | -0.95 | -1.15 |
| TPCH(30) | TPCH-Q9 | parquet / snap / block | 8.84 | 9.02 | -1.93% | 1.78% | 2.55% | 20 | -2.27% | -2.50 | -2.79 |
| TPCH(30) | TPCH-Q17 | parquet / snap / block | 8.27 | 8.45 | -2.09% | 1.16% | 1.32% | 20 | -2.33% | -4.01 | -5.35 |
| TPCH(30) | TPCH-Q10 | parquet / snap / block | 4.95 | 5.13 | -3.60% | 6.86% | 9.75% | 20 | -2.53% | -2.70 | -1.37 |
+----------+----------+------------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+-------+
Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
---
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
A be/src/exec/scan-range-queue-mt.h
M tests/query_test/test_scanners.py
4 files changed, 166 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18929/3
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 5: Code-Review+1
(1 comment)
This makes sense to me.
I don't think this would have an impact on data caching except in pretty extreme cases. It's possible that we could get smarter with this logic in future, but this seems like an improvement on its own.
We are moving from O(1) to O(log(N)), but that doesn't seem like a big problem for this data structure. Other costs should dwarf the cost of accessing this structure.
http://gerrit.cloudera.org:8080/#/c/18929/5/be/src/exec/scan-range-queue-mt.h
File be/src/exec/scan-range-queue-mt.h:
http://gerrit.cloudera.org:8080/#/c/18929/5/be/src/exec/scan-range-queue-mt.h@37
PS5, Line 37: class ScanRangeQueueMt {
Nit: Maybe add DISALLOW_COPY_AND_ASSIGN
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 01 Sep 2022 19:28:12 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 5:
I'm ok to go to +2
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 01 Sep 2022 19:31:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 6:
(1 comment)
Thanks for the comments. Yeah we can improve this queue further in the future if really needed, e.g. improving concurrency, but for now I just wanted to keep it very simple.
http://gerrit.cloudera.org:8080/#/c/18929/5/be/src/exec/scan-range-queue-mt.h
File be/src/exec/scan-range-queue-mt.h:
http://gerrit.cloudera.org:8080/#/c/18929/5/be/src/exec/scan-range-queue-mt.h@37
PS5, Line 37: class ScanRangeQueueMt {
> Nit: Maybe add DISALLOW_COPY_AND_ASSIGN
Done
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Sep 2022 11:42:56 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 7: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8532/
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Sep 2022 18:00:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Qifan Chen (Code Review)" <ge...@cloudera.org>.
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 7:
Seems like a very good optimization and maybe we could also consider some other de-skewing opportunities.
1. Allow the fragments to process scan ranges residing on different disks at the same time;
2. For scan ranges from the scan disks, apply the size-based approach outlined in this patch, or a modification of it that can pick the most CPU extensive scan first. This basically asks for a CPU cost for a scan range. Not sure if such info is available these days.
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Sep 2022 20:22:26 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )
Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................
Patch Set 9:
Create a Jira with the improvements that came up during review:
IMPALA-11539
--
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 9
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Sep 2022 14:53:42 +0000
Gerrit-HasComments: No