You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2019/05/09 00:12:35 UTC

[Impala-ASF-CR] expr-test: use gtest parameterization

Hello Tim Armstrong,

I'd like you to do a code review. Please visit

    http://gerrit.cloudera.org:8080/13289

to review the following change.


Change subject: expr-test: use gtest parameterization
......................................................................

expr-test: use gtest parameterization

Instead of running the tests three times with different flags from
main(), this uses gtest's parameterization feature to accomplish the
same.

The advantage here is that we end up with different test names for each
of the runs. Additionally, this moves the setup code into a proper setup
method so that executing expr-test --gtest_list_tests doesn't waste time
starting a cluster.

This is prep work towards adding multi-threaded test execution for
long-running tests. expr-test seems to currently be one of the worst
offenders.

Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
---
M be/src/exprs/expr-test.cc
1 file changed, 126 insertions(+), 125 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/13289/1
-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13289/1/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:

http://gerrit.cloudera.org:8080/#/c/13289/1/be/src/exprs/expr-test.cc@4811
PS1, Line 4811: TEST_P(ExprTest, StringParseUrlFunction) { // TODO: For now, our parse_url my not behave exactly like Hive
line too long (106 > 90)



-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 May 2019 00:13:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 1:

(1 comment)

This seems like an improvement. I think the previous approach was just from lack of knowledge about gtest. I know I did something similar with buffer-pool-test too :)

http://gerrit.cloudera.org:8080/#/c/13289/1/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:

http://gerrit.cloudera.org:8080/#/c/13289/1/be/src/exprs/expr-test.cc@282
PS1, Line 282:       // TODO(todd): why are these settings only for this case?
> I think this only needs to be for (!disable_codegen_). These are important 
Actually this can just be unconditional because it has no effect if codegen is disabled. I looked at the original code and it was an artefact of how that code was structured instead of intentional.



-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 May 2019 00:55:56 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................

expr-test: use gtest parameterization

Instead of running the tests three times with different flags from
main(), this uses gtest's parameterization feature to accomplish the
same.

The advantage here is that we end up with different test names for each
of the runs. Additionally, this moves the setup code into a proper setup
method so that executing expr-test --gtest_list_tests doesn't waste time
starting a cluster.

This is prep work towards adding multi-threaded test execution for
long-running tests. expr-test seems to currently be one of the worst
offenders.

Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Reviewed-on: http://gerrit.cloudera.org:8080/13289
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exprs/expr-test.cc
1 file changed, 135 insertions(+), 125 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 2: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 May 2019 06:18:44 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 1:

This is pretty mechanical so I'm probably OK to +2 once the TODOs are removed.


-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 May 2019 00:56:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4230/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 10 May 2019 19:03:25 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13289/1/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:

http://gerrit.cloudera.org:8080/#/c/13289/1/be/src/exprs/expr-test.cc@282
PS1, Line 282:       // TODO(todd): why are these settings only for this case?
I think this only needs to be for (!disable_codegen_). These are important in that case so codegen doesn't get auto-disabled. We could just set them unconditionally though even so that the planner doesn't do anything clever in any cases.


http://gerrit.cloudera.org:8080/#/c/13289/1/be/src/exprs/expr-test.cc@9296
PS1, Line 9296: // TODO(todd) why don't we test the false/true case?
IIRC just because it didn't add much coverage and adds a lot to test runtime (running this test with codegen enabled is really slow). It was a bit of a fight to even convince people that we should run it in one configuration with codegen enabled.

Mostly we get the best backend codegened/interpreted coverage from running the unmodified (i.e. more complex) expr trees enabled. Running the trees in interpreted mode should be enough to validate correctness of the rewrites. So enabling the remaining combination might provide some additional incidental coverage from codegening some additional expr tree shapes but mostly it isn't that interesting since the majorit y of expressions get folded to a constant anyway.



-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 May 2019 00:53:53 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/3140/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 May 2019 00:58:50 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 11 May 2019 01:28:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 10 May 2019 19:03:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4201/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 May 2019 06:23:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 2: -Verified

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/3145/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 May 2019 08:11:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13289

to look at the new patch set (#2).

Change subject: expr-test: use gtest parameterization
......................................................................

expr-test: use gtest parameterization

Instead of running the tests three times with different flags from
main(), this uses gtest's parameterization feature to accomplish the
same.

The advantage here is that we end up with different test names for each
of the runs. Additionally, this moves the setup code into a proper setup
method so that executing expr-test --gtest_list_tests doesn't waste time
starting a cluster.

This is prep work towards adding multi-threaded test execution for
long-running tests. expr-test seems to currently be one of the worst
offenders.

Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
---
M be/src/exprs/expr-test.cc
1 file changed, 135 insertions(+), 125 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/13289/2
-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] expr-test: use gtest parameterization

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13289 )

Change subject: expr-test: use gtest parameterization
......................................................................


Patch Set 2: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4201/


-- 
To view, visit http://gerrit.cloudera.org:8080/13289
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc9fb24ad62b4aa2e120a99d74ae04bb221c034b
Gerrit-Change-Number: 13289
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 May 2019 06:26:55 +0000
Gerrit-HasComments: No