You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2020/06/27 00:28:39 UTC
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16118
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
IMPALA-9902: add rewrite of TPC-DS q38
I generated the query with dsqgen and then
rewrote it to avoid intersect.
Testing:
Compared results to hive running the original version of the
query.
Change-Id: I81807683aa265a946729e15156bd2e33724103e1
---
A testdata/workloads/tpcds/queries/tpcds-q38.test
M testdata/workloads/tpcds/queries/tpcds-q8.test
M tests/query_test/test_tpcds_queries.py
3 files changed, 39 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/16118/1
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6459/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 23:43:34 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Fang-Yu Rao, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16118
to look at the new patch set (#3).
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
IMPALA-9902: add rewrite of TPC-DS q38
I generated the query with dsqgen and then
rewrote it to avoid intersect.
Testing:
Compared results to hive running the original version of the
query.
Change-Id: I81807683aa265a946729e15156bd2e33724103e1
---
A testdata/workloads/tpcds/queries/tpcds-decimal_v2-q38-rewrite.test
M testdata/workloads/tpcds/queries/tpcds-q8.test
M tests/query_test/test_tpcds_queries.py
M tests/util/parse_util.py
4 files changed, 40 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/16118/3
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 06 Jul 2020 23:47:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6505/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 06 Jul 2020 18:04:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 2:
(3 comments)
Thanks for pointing out that I need to update the query count, I missed that originally.
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q38.test
File testdata/workloads/tpcds/queries/tpcds-q38.test:
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q38.test@1
PS1, Line 1:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
> I just found that HS2 is not able to compile the rewritten Q38 due to the f
Yep, I also compared this to the hive results from the original query.
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q8.test
File testdata/workloads/tpcds/queries/tpcds-q8.test:
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q8.test@4
PS1, Line 4: -- INTERSECT.
> Thanks Tim for catching this! After checking $IMPALA_HOME/toolchain/toolcha
Yeah I was going to add a rewrite of this cause it also used INTERSECT, then discovered that we'd already done it.
http://gerrit.cloudera.org:8080/#/c/16118/1/tests/query_test/test_tpcds_queries.py
File tests/query_test/test_tpcds_queries.py:
http://gerrit.cloudera.org:8080/#/c/16118/1/tests/query_test/test_tpcds_queries.py@127
PS1, Line 127: def test_tpcds_q39_1(self, vector):
> I am wondering if we need to add the modified Q38 under the class of TestTp
I moved it to the decimal_v2 tests, I think that seems reasonable since that's the default behaviour. It shouldn't affect the result of the query.
I added a -rewrite suffix to indicate that it was a rewrite. 38a makes it sound like an official variant. I think unfortunately a lot of these TPC-DS queries have undocumented tweaks to them, but we do have the Unmodified suite below at least.
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 23:15:58 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6095/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 06 Jul 2020 18:44:33 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 2: Code-Review+1
(3 comments)
Thanks Tim for addressing my previous comments! I do not have any other suggestion.
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q38.test
File testdata/workloads/tpcds/queries/tpcds-q38.test:
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q38.test@1
PS1, Line 1:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
> Yep, I also compared this to the hive results from the original query.
Ack
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q8.test
File testdata/workloads/tpcds/queries/tpcds-q8.test:
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q8.test@4
PS1, Line 4: -- INTERSECT.
> Yeah I was going to add a rewrite of this cause it also used INTERSECT, the
Ack
http://gerrit.cloudera.org:8080/#/c/16118/1/tests/query_test/test_tpcds_queries.py
File tests/query_test/test_tpcds_queries.py:
http://gerrit.cloudera.org:8080/#/c/16118/1/tests/query_test/test_tpcds_queries.py@127
PS1, Line 127: def test_tpcds_q39_1(self, vector):
> I moved it to the decimal_v2 tests, I think that seems reasonable since tha
Ack
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 30 Jun 2020 02:16:19 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6438/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sat, 27 Jun 2020 00:47:33 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 3: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 06 Jul 2020 17:37:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 1:
(3 comments)
Hi Tim, thanks for this patch! I only have two minor comments. I left one comment in test_tpcds_queries.py and the other one regarding EXPECTED_TPCDS_QUERIES_COUNT in the following.
Specifically, I am wondering if we need to change EXPECTED_TPCDS_QUERIES_COUNT from 72 to 73 at https://github.com/apache/impala/blob/master/tests/util/parse_util.py#L25 since we add one more query in this patch. I recall that the following 2 tests depend on the value of EXPECTED_TPCDS_QUERIES_COUNT.
1) tests/infra/test_stress_infra.py::TestStressInfra::test_stress_finds_workloads: https://github.com/apache/impala/blob/master/tests/infra/test_stress_infra.py#L66, and
2) tests/infra/test_perf_infra.py::TestPerfInfra::test_run_workload_finds_queries: https://github.com/apache/impala/blob/master/tests/infra/test_perf_infra.py#L44
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q38.test
File testdata/workloads/tpcds/queries/tpcds-q38.test:
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q38.test@1
PS1, Line 1: ====
: ---- QUERY: TPCDS-Q38
: -- This is an unofficial variant that is rewritten to use EXISTS subqueries instead of
: -- INTERSECT
: select count(*) from (
: select distinct c_last_name, c_first_name, d_date
: from store_sales, date_dim dd1, customer c1
: where store_sales.ss_sold_date_sk = dd1.d_date_sk
: and store_sales.ss_customer_sk = c1.c_customer_sk
: and d_month_seq between 1189 and 1189 + 11
: and exists (
: select distinct c_last_name, c_first_name, d_date
: from catalog_sales, date_dim dd2, customer c2
: where catalog_sales.cs_sold_date_sk = dd2.d_date_sk
: and catalog_sales.cs_bill_customer_sk = c2.c_customer_sk
: and d_month_seq between 1189 and 1189 + 11
: and c1.c_last_name <=> c2.c_last_name and c1.c_first_name <=> c2.c_first_name and dd1.d_date <=> dd2.d_date
: )
: and exists (
: select distinct c_last_name, c_first_name, d_date
: from web_sales, date_dim dd3, customer c3
: where web_sales.ws_sold_date_sk = dd3.d_date_sk
: and web_sales.ws_bill_customer_sk = c3.c_customer_sk
: and d_month_seq between 1189 and 1189 + 11
: and c1.c_last_name <=> c3.c_last_name and c1.c_first_name <=> c3.c_first_name and dd1.d_date <=> dd3.d_date
: )
: ) hot_cust
: limit 100
: ---- RESULTS
: 108
: ---- TYPES
: BIGINT
: ====
I just found that HS2 is not able to compile the rewritten Q38 due to the following error.
Error while compiling statement: FAILED: SemanticException [Error 10249]: Line 15:9 Unsupported SubQuery Expression 'd_date': Only 1 SubQuery expression is supported.
But I am still able to use HS2 to verify the correctness by using the unmodified query and I found the results from Impala and HS2 match.
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q8.test
File testdata/workloads/tpcds/queries/tpcds-q8.test:
http://gerrit.cloudera.org:8080/#/c/16118/1/testdata/workloads/tpcds/queries/tpcds-q8.test@4
PS1, Line 4: -- INTERSECT.
Thanks Tim for catching this! After checking $IMPALA_HOME/toolchain/toolchain-packages-gcc7.5.0/tpc-ds-2.1.0/share/tpc-ds/query_templates/query8.tpl, I just realized that Q8 is indeed a variant.
http://gerrit.cloudera.org:8080/#/c/16118/1/tests/query_test/test_tpcds_queries.py
File tests/query_test/test_tpcds_queries.py:
http://gerrit.cloudera.org:8080/#/c/16118/1/tests/query_test/test_tpcds_queries.py@127
PS1, Line 127: def test_tpcds_q38(self, vector):
I am wondering if we need to add the modified Q38 under the class of TestTpcdsDecimalV2Query (https://github.com/apache/impala/blob/master/tests/query_test/test_tpcds_queries.py#L269).
On the other hand, I am also wondering if it is better to change the name of the test to test_tpcds_q38a so that one can tell from its name that the query is a variant.
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sun, 28 Jun 2020 19:19:12 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Fang-Yu Rao, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16118
to look at the new patch set (#2).
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
IMPALA-9902: add rewrite of TPC-DS q38
I generated the query with dsqgen and then
rewrote it to avoid intersect.
Testing:
Compared results to hive running the original version of the
query.
Change-Id: I81807683aa265a946729e15156bd2e33724103e1
---
A testdata/workloads/tpcds/queries/tpcds-decimal_v2-q38-rewrite.test
M testdata/workloads/tpcds/queries/tpcds-q8.test
M tests/query_test/test_tpcds_queries.py
M tests/util/parse_util.py
4 files changed, 40 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/16118/2
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 3: Code-Review+2
This makes sense to me.
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 06 Jul 2020 18:08:10 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
IMPALA-9902: add rewrite of TPC-DS q38
I generated the query with dsqgen and then
rewrote it to avoid intersect.
Testing:
Compared results to hive running the original version of the
query.
Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Reviewed-on: http://gerrit.cloudera.org:8080/16118
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
A testdata/workloads/tpcds/queries/tpcds-decimal_v2-q38-rewrite.test
M testdata/workloads/tpcds/queries/tpcds-q8.test
M tests/query_test/test_tpcds_queries.py
M tests/util/parse_util.py
4 files changed, 40 insertions(+), 2 deletions(-)
Approvals:
Tim Armstrong: Looks good to me, but someone else must approve
Joe McDonnell: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-9902: add rewrite of TPC-DS q38
Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/16118 )
Change subject: IMPALA-9902: add rewrite of TPC-DS q38
......................................................................
Patch Set 1:
Hi Tim, I am answering one of my questions regarding whether to update EXPECTED_TPCDS_QUERIES_COUNT from 72 to 73 at https://github.com/apache/impala/blob/master/tests/util/parse_util.py#L25.
According to the logic in the function of load_tpc_queries() at https://github.com/apache/impala/blob/master/tests/util/test_file_parser.py#L348-L350, it seems that we only need to update EXPECTED_TPCDS_QUERIES_COUNT when adding tests with names containing "tpcds-demical_v2" and thus we do not need to change EXPECTED_TPCDS_QUERIES_COUNT if we do not add tests under the class of TestTpcdsDecimalV2Query.
--
To view, visit http://gerrit.cloudera.org:8080/16118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I81807683aa265a946729e15156bd2e33724103e1
Gerrit-Change-Number: 16118
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sun, 28 Jun 2020 21:35:40 +0000
Gerrit-HasComments: No