You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2016/12/01 07:21:49 UTC
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Tim Armstrong has uploaded a new change for review.
http://gerrit.cloudera.org:8080/5302
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
Fix a test bug where we need to skip nested types tests for the old aggs
and joins.
Fix a product bug where *eos is not initialised by the MT scan node.
This causes incorrect results when the calling ExecNode does not
initialise the eos variable, e.g. the sort node and the old agg and join
nodes.
Testing:
Added a test that reproduces the incorrect results with the sort node
when run under ASAN
Tested the mt_dop tests locally with old aggs and joins to ensure they
pass.
Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
---
M be/src/exec/hdfs-scan-node-mt.cc
A testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet-nested.test
M testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet.test
M tests/query_test/test_mt_dop.py
4 files changed, 50 insertions(+), 29 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/5302/1
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Michael Ho, Dan Hecht,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/5302
to look at the new patch set (#2).
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
Fix a test bug where we need to skip nested types tests for the old aggs
and joins.
Fix a product bug where *eos is not initialised by the MT scan node.
This causes incorrect results when the calling ExecNode does not
initialise the eos variable, e.g. the sort node and the old agg and join
nodes.
Testing:
Added a test that reproduces the incorrect results with the sort node
when run under ASAN
Tested the mt_dop tests locally with old aggs and joins to ensure they
pass.
Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
---
M be/src/exec/hdfs-scan-node-mt.cc
A testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet-nested.test
M testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet.test
M testdata/workloads/functional-query/queries/QueryTest/mt-dop.test
M tests/query_test/test_mt_dop.py
5 files changed, 54 insertions(+), 33 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/5302/2
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change.
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/5302/1/testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet.test
File testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet.test:
Line 9: # IMPALA-4565: incorrect results because mt scan node does not set eos
> move this into mt-dop.test since it's not specific to Parquet and getting c
Done
Line 11: set disable_outermost_topn=true;
> I'm a little wary of having this test coverage depend on this 'exotic' quer
Done
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Alex Behm (Code Review)" <ge...@cloudera.org>.
Alex Behm has posted comments on this change.
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/5302/1/testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet.test
File testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet.test:
Line 9: # IMPALA-4565: incorrect results because mt scan node does not set eos
move this into mt-dop.test since it's not specific to Parquet and getting coverage over all file formats seems not bad
Line 11: set disable_outermost_topn=true;
I'm a little wary of having this test coverage depend on this 'exotic' query option. Maybe the same can be achieved by using an order by without limit on alltypestiny, but setting the batch size to 1?
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change.
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Michael Ho has posted comments on this change.
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
Patch Set 1: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change.
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
Patch Set 4:
Build started: http://jenkins.impala.io:8080/job/gerrit-verify-dryrun/65/
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change.
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
Patch Set 1:
Build started: http://jenkins.impala.io:8080/job/gerrit-verify-dryrun/63/
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Alex Behm (Code Review)" <ge...@cloudera.org>.
Alex Behm has posted comments on this change.
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Dan Hecht (Code Review)" <ge...@cloudera.org>.
Dan Hecht has posted comments on this change.
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged.
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
Fix a test bug where we need to skip nested types tests for the old aggs
and joins.
Fix a product bug where *eos is not initialised by the MT scan node.
This causes incorrect results when the calling ExecNode does not
initialise the eos variable, e.g. the sort node and the old agg and join
nodes.
Testing:
Added a test that reproduces the incorrect results with the sort node
when run under ASAN
Tested the mt_dop tests locally with old aggs and joins to ensure they
pass.
Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Reviewed-on: http://gerrit.cloudera.org:8080/5302
Reviewed-by: Alex Behm <al...@cloudera.com>
Tested-by: Impala Public Jenkins
---
M be/src/exec/hdfs-scan-node-mt.cc
A testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet-nested.test
M testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet.test
M testdata/workloads/functional-query/queries/QueryTest/mt-dop.test
M tests/query_test/test_mt_dop.py
5 files changed, 60 insertions(+), 33 deletions(-)
Approvals:
Impala Public Jenkins: Verified
Alex Behm: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-4564,IMPALA-4565: mt dop fixes for old aggs and joins
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Michael Ho, Dan Hecht,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/5302
to look at the new patch set (#3).
Change subject: IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
......................................................................
IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins
Fix a test bug where we need to skip nested types tests for the old aggs
and joins.
Fix a product bug where *eos is not initialised by the MT scan node.
This causes incorrect results when the calling ExecNode does not
initialise the eos variable, e.g. the sort node and the old agg and join
nodes.
Testing:
Added a test that reproduces the incorrect results with the sort node
when run under ASAN
Tested the mt_dop tests locally with old aggs and joins to ensure they
pass.
Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
---
M be/src/exec/hdfs-scan-node-mt.cc
A testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet-nested.test
M testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet.test
M testdata/workloads/functional-query/queries/QueryTest/mt-dop.test
M tests/query_test/test_mt_dop.py
5 files changed, 60 insertions(+), 33 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/5302/3
--
To view, visit http://gerrit.cloudera.org:8080/5302
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>