You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2022/09/07 19:38:39 UTC
[kudu-CR] [tools] mark long-running scenarios in kudu-tool-test
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18947
Change subject: [tools] mark long-running scenarios in kudu-tool-test
......................................................................
[tools] mark long-running scenarios in kudu-tool-test
I noticed that it takes very long time for kudu-tool-test to complete.
It turned out the test suite contains many scenarios that run with
HMS integration enabled and it takes a long time to start up the
HMS process.
This patch marks the scenarios that tend to run over 3 seconds
as slow, so they are skipped unless the KUDU_ALLOW_SLOW_TESTS
environment variable is set to true/yes/1.
The run time of the kudu-tool-test with this patch looks like the
following on a contemporary Linux machine:
Without KUDU_ALLOW_SLOW_TESTS:
[==========] 128 tests from 8 test suites ran. (86355 ms total)
With KUDU_ALLOW_SLOW_TESTS=1:
[==========] 128 tests from 8 test suites ran. (842684 ms total)
Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
---
M src/kudu/tools/kudu-tool-test.cc
1 file changed, 28 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/47/18947/1
--
To view, visit http://gerrit.cloudera.org:8080/18947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
Gerrit-Change-Number: 18947
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
[kudu-CR] [tools] mark long-running scenarios in kudu-tool-test
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/18947 )
Change subject: [tools] mark long-running scenarios in kudu-tool-test
......................................................................
Patch Set 2: Verified+1
unrelated test failure (TSAN):
org.apache.kudu.client.TestScanToken
--
To view, visit http://gerrit.cloudera.org:8080/18947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
Gerrit-Change-Number: 18947
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy <mr...@cloudera.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Thu, 08 Sep 2022 01:03:13 +0000
Gerrit-HasComments: No
[kudu-CR] [tools] mark long-running scenarios in kudu-tool-test
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/18947 )
Change subject: [tools] mark long-running scenarios in kudu-tool-test
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18947/3/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/18947/3/src/kudu/tools/kudu-tool-test.cc@1225
PS3, Line 1225: SKIP_IF_SLOW_NOT_ALLOWED();
> I'm a bit of confused why this help info could cost so much time, maybe we
With SKIP_IF_SLOW_NOT_ALLOWED() those test scenarios are simply exit early and not run unless KUDU_ALLOW_SLOW_TESTS is set 'true', saving a lot of time in total.
The majority of time in those long scenario is spent to wait for HMS starting up -- that process hogs a lot of memory and runs very slow (that's Java). With that, I don't think running those multi-thread way could be able to help there.
--
To view, visit http://gerrit.cloudera.org:8080/18947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
Gerrit-Change-Number: 18947
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy <mr...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Sun, 11 Sep 2022 16:11:47 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] mark long-running scenarios in kudu-tool-test
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed a vote on this change.
Change subject: [tools] mark long-running scenarios in kudu-tool-test
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/18947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
Gerrit-Change-Number: 18947
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy <mr...@cloudera.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [tools] mark long-running scenarios in kudu-tool-test
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18947 )
Change subject: [tools] mark long-running scenarios in kudu-tool-test
......................................................................
Patch Set 3: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18947/3/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/18947/3/src/kudu/tools/kudu-tool-test.cc@1225
PS3, Line 1225: SKIP_IF_SLOW_NOT_ALLOWED();
I'm a bit of confused why this help info could cost so much time, maybe we can run them in multi-threads to reduce time later.
--
To view, visit http://gerrit.cloudera.org:8080/18947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
Gerrit-Change-Number: 18947
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy <mr...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Sun, 11 Sep 2022 14:52:16 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] mark long-running scenarios in kudu-tool-test
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Yingchun Lai, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18947
to look at the new patch set (#2).
Change subject: [tools] mark long-running scenarios in kudu-tool-test
......................................................................
[tools] mark long-running scenarios in kudu-tool-test
I noticed that it takes very long time for kudu-tool-test to complete.
It turned out the test suite contains many scenarios that run with
HMS integration enabled and it takes a long time to start up the
HMS process.
This patch marks the scenarios that tend to run over 3 seconds
as slow, so they are skipped unless the KUDU_ALLOW_SLOW_TESTS
environment variable is set to true/yes/1.
The run time of the kudu-tool-test with this patch looks like the
following on a contemporary Linux machine:
Without KUDU_ALLOW_SLOW_TESTS:
[==========] 128 tests from 8 test suites ran. (86355 ms total)
[ PASSED ] 88 tests.
[ SKIPPED ] 40 tests, listed below:
...
With KUDU_ALLOW_SLOW_TESTS=1:
[==========] 128 tests from 8 test suites ran. (842684 ms total)
[ PASSED ] 128 tests.
Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
---
M src/kudu/tools/kudu-tool-test.cc
1 file changed, 26 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/47/18947/2
--
To view, visit http://gerrit.cloudera.org:8080/18947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
Gerrit-Change-Number: 18947
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [tools] mark long-running scenarios in kudu-tool-test
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18947 )
Change subject: [tools] mark long-running scenarios in kudu-tool-test
......................................................................
[tools] mark long-running scenarios in kudu-tool-test
I noticed that it takes very long time for kudu-tool-test to complete.
It turned out the test suite contains many scenarios that run with
HMS integration enabled and it takes a long time to start up the
HMS process.
This patch marks the scenarios that tend to run over 3 seconds
as slow, so they are skipped unless the KUDU_ALLOW_SLOW_TESTS
environment variable is set to true/yes/1.
The run time of the kudu-tool-test with this patch looks like the
following on a contemporary Linux machine:
Without KUDU_ALLOW_SLOW_TESTS:
[==========] 128 tests from 8 test suites ran. (86355 ms total)
[ PASSED ] 88 tests.
[ SKIPPED ] 40 tests, listed below:
...
With KUDU_ALLOW_SLOW_TESTS=1:
[==========] 128 tests from 8 test suites ran. (842684 ms total)
[ PASSED ] 128 tests.
Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
Reviewed-on: http://gerrit.cloudera.org:8080/18947
Reviewed-by: Mahesh Reddy <mr...@cloudera.com>
Tested-by: Alexey Serbin <al...@apache.org>
Reviewed-by: Yifan Zhang <ch...@163.com>
---
M src/kudu/tools/kudu-tool-test.cc
1 file changed, 26 insertions(+), 0 deletions(-)
Approvals:
Mahesh Reddy: Looks good to me, but someone else must approve
Alexey Serbin: Verified
Yifan Zhang: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/18947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
Gerrit-Change-Number: 18947
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy <mr...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [tools] mark long-running scenarios in kudu-tool-test
Posted by "Mahesh Reddy (Code Review)" <ge...@cloudera.org>.
Mahesh Reddy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18947 )
Change subject: [tools] mark long-running scenarios in kudu-tool-test
......................................................................
Patch Set 2: Code-Review+1
lgtm, unrelated test failure
--
To view, visit http://gerrit.cloudera.org:8080/18947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
Gerrit-Change-Number: 18947
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy <mr...@cloudera.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Wed, 07 Sep 2022 22:27:53 +0000
Gerrit-HasComments: No
[kudu-CR] [tools] mark long-running scenarios in kudu-tool-test
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18947 )
Change subject: [tools] mark long-running scenarios in kudu-tool-test
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1893741fada185d819128396c88f4ac99f43ed40
Gerrit-Change-Number: 18947
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy <mr...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Sun, 11 Sep 2022 05:44:54 +0000
Gerrit-HasComments: No