You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2018/06/29 22:43:47 UTC
[Impala-ASF-CR] IMPALA-7185: low statestore custom cluster interval
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10845
Change subject: IMPALA-7185: low statestore custom cluster interval
......................................................................
IMPALA-7185: low statestore custom cluster interval
This changes the default statestore interval for the custom cluster
tests. This can reduce the time taken for the cluster to start and
metadata to load. On some tests this resulted in saving 5+ seconds
per test. Overall it shaved around a minute off the custom cluster
tests.
Testing:
Ran 10 iterations of the tests.
Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
---
M tests/authorization/test_grant_revoke.py
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_admission_controller.py
M tests/run-custom-cluster-tests.sh
4 files changed, 16 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/10845/1
--
To view, visit http://gerrit.cloudera.org:8080/10845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Gerrit-Change-Number: 10845
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7185: low statestore custom cluster interval
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10845 )
Change subject: IMPALA-7185: low statestore custom cluster interval
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2764/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/10845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Gerrit-Change-Number: 10845
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Jul 2018 18:39:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7185: low statestore custom cluster interval
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/10845
to look at the new patch set (#2).
Change subject: IMPALA-7185: low statestore custom cluster interval
......................................................................
IMPALA-7185: low statestore custom cluster interval
This changes the default statestore interval for the custom cluster
tests. This can reduce the time taken for the cluster to start and
metadata to load. On some tests this resulted in saving 5+ seconds
per test. Overall it shaved around a minute off the custom cluster
tests.
Testing:
Ran 10 iterations of the tests.
Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
---
M tests/authorization/test_grant_revoke.py
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_admission_controller.py
3 files changed, 13 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/10845/2
--
To view, visit http://gerrit.cloudera.org:8080/10845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Gerrit-Change-Number: 10845
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-7185: low statestore custom cluster interval
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10845 )
Change subject: IMPALA-7185: low statestore custom cluster interval
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/10845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Gerrit-Change-Number: 10845
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Jul 2018 18:39:13 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7185: low statestore custom cluster interval
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/10845 )
Change subject: IMPALA-7185: low statestore custom cluster interval
......................................................................
Patch Set 3: Code-Review+2
This looks good to me.
--
To view, visit http://gerrit.cloudera.org:8080/10845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Gerrit-Change-Number: 10845
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Jul 2018 18:03:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7185: low statestore custom cluster interval
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10845 )
Change subject: IMPALA-7185: low statestore custom cluster interval
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/10845/2/tests/custom_cluster/test_admission_controller.py
File tests/custom_cluster/test_admission_controller.py:
http://gerrit.cloudera.org:8080/#/c/10845/2/tests/custom_cluster/test_admission_controller.py@287
PS2, Line 287: "-default_query_options=mem_limit=200000000"))
> Most of the tests in this file use _STATESTORED_ARGS (all but test_mem_limi
I'm not sure why I made this change, I think it was unintended. It probably doesn't matter much either way for this test though.
--
To view, visit http://gerrit.cloudera.org:8080/10845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Gerrit-Change-Number: 10845
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 30 Jun 2018 00:28:54 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7185: low statestore custom cluster interval
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10845 )
Change subject: IMPALA-7185: low statestore custom cluster interval
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/10845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Gerrit-Change-Number: 10845
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Jul 2018 22:06:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7185: low statestore custom cluster interval
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10845 )
Change subject: IMPALA-7185: low statestore custom cluster interval
......................................................................
IMPALA-7185: low statestore custom cluster interval
This changes the default statestore interval for the custom cluster
tests. This can reduce the time taken for the cluster to start and
metadata to load. On some tests this resulted in saving 5+ seconds
per test. Overall it shaved around a minute off the custom cluster
tests.
Testing:
Ran 10 iterations of the tests.
Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Reviewed-on: http://gerrit.cloudera.org:8080/10845
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/authorization/test_grant_revoke.py
M tests/common/custom_cluster_test_suite.py
2 files changed, 12 insertions(+), 7 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/10845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Gerrit-Change-Number: 10845
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7185: low statestore custom cluster interval
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/10845
to look at the new patch set (#3).
Change subject: IMPALA-7185: low statestore custom cluster interval
......................................................................
IMPALA-7185: low statestore custom cluster interval
This changes the default statestore interval for the custom cluster
tests. This can reduce the time taken for the cluster to start and
metadata to load. On some tests this resulted in saving 5+ seconds
per test. Overall it shaved around a minute off the custom cluster
tests.
Testing:
Ran 10 iterations of the tests.
Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
---
M tests/authorization/test_grant_revoke.py
M tests/common/custom_cluster_test_suite.py
2 files changed, 12 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/10845/3
--
To view, visit http://gerrit.cloudera.org:8080/10845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Gerrit-Change-Number: 10845
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-7185: low statestore custom cluster interval
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/10845 )
Change subject: IMPALA-7185: low statestore custom cluster interval
......................................................................
Patch Set 2:
(1 comment)
Thanks for putting this together.
http://gerrit.cloudera.org:8080/#/c/10845/2/tests/custom_cluster/test_admission_controller.py
File tests/custom_cluster/test_admission_controller.py:
http://gerrit.cloudera.org:8080/#/c/10845/2/tests/custom_cluster/test_admission_controller.py@287
PS2, Line 287: "-default_query_options=mem_limit=200000000"))
Most of the tests in this file use _STATESTORED_ARGS (all but test_mem_limit_upper_bound prior to this change, and that one might not be intentional). Is there something unique about this test that lets you remove it here? (Or can we remove it for other tests here?)
--
To view, visit http://gerrit.cloudera.org:8080/10845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5d1612283ff420d95b0dd0ca5a2a67f56765f79
Gerrit-Change-Number: 10845
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Jun 2018 23:24:13 +0000
Gerrit-HasComments: Yes