You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Jason Fehr (Code Review)" <ge...@cloudera.org> on 2024/03/28 17:47:39 UTC

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Jason Fehr has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21221


Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................

IMPALA-12944: Fixes Workload Management Test Flakiness

The custom cluster workload management tests are flaky
because the tests can actually run before the completed
queries table has been fully created by the Impala startup
process. The table create sql runs asynchronously during
startup and thus can take longer to finish than the custom
cluster tests take to execute.

This change adds checks at the beginning of each test to
ensure the completed queries table sql has finished before
any of the test code runs.

Change-Id: I428702a210e024db95808dc2518da497426922f8
---
M tests/custom_cluster/test_query_log.py
1 file changed, 87 insertions(+), 29 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/21221/1
-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 1
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py@73
PS3, Line 73: 
> I just remember, if you inspect logs like this, you need to ensure that eac
Will it? It looks specifically at impalad.INFO, which should be new for every custom cluster test.



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 6
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 22:37:26 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py@22
PS3, Line 22: import re
flake8: F401 're' imported but unused



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 20:44:00 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 7:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/15733/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Mar 2024 01:28:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 3:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/15725/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 21:04:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 6:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/15729/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 6
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 21:24:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21221

to look at the new patch set (#3).

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................

IMPALA-12944: Fixes Workload Management Test Flakiness

The custom cluster workload management tests are flaky
because the tests can actually run before the completed
queries table has been fully created by the Impala startup
process. The table create sql runs asynchronously during
startup and thus can take longer to finish than the custom
cluster tests take to execute.

This change adds checks at the beginning of each test to
ensure the completed queries table sql has finished before
any of the test code runs.

Change-Id: I428702a210e024db95808dc2518da497426922f8
---
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_query_log.py
2 files changed, 43 insertions(+), 22 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/21221/3
-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 7: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Mar 2024 05:54:37 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 11:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/15740/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 11
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Mar 2024 23:02:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21221

to look at the new patch set (#4).

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................

IMPALA-12944: Fixes Workload Management Test Flakiness

The custom cluster workload management tests are flaky
because the tests can actually run before the completed
queries table has been fully created by the Impala startup
process. The table create sql runs asynchronously during
startup and thus can take longer to finish than the custom
cluster tests take to execute.

This change adds checks at the beginning of each test to
ensure the completed queries table sql has finished before
any of the test code runs.

Change-Id: I428702a210e024db95808dc2518da497426922f8
---
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_query_log.py
2 files changed, 42 insertions(+), 22 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/21221/4
-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 4
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py@73
PS3, Line 73: 
> Will it? It looks specifically at impalad.INFO, which should be new for eve
test_breakpad has a custom matcher that uses wildcard matching. That's different.



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 6
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 22:38:34 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 6: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 6
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 22:38:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 6: Code-Review+1

(3 comments)

http://gerrit.cloudera.org:8080/#/c/21221/6/tests/common/impala_test_suite.py
File tests/common/impala_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/21221/6/tests/common/impala_test_suite.py@1270
PS6, Line 1270: return \
              :         self.assert_log_contains(
nit: looks better if breaking after parenthesis. No need for backslash.


http://gerrit.cloudera.org:8080/#/c/21221/6/tests/common/impala_test_suite.py@1293
PS6, Line 1293: Returns the result of the very last call to line_regex.search.
nit: "Returns the result of the very last call to line_regex.search or None if expected_count=0."


http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py@73
PS3, Line 73: 
> Will it? It looks specifically at impalad.INFO, which should be new for eve
Right, nevermind.
I look again and test_breakpad.py has its own assert_logs() method. I did stumble upon that back then.
assert_impalad_log_contains() looks safe since it resolve symlink and target just 1 file.



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 6
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 22:53:53 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21221/5/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/5/tests/custom_cluster/test_query_log.py@22
PS5, Line 22: import re
flake8: F401 're' imported but unused



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 5
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 20:55:36 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21221

to look at the new patch set (#7).

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................

IMPALA-12944: Fixes Workload Management Test Flakiness

The custom cluster workload management tests are flaky
because the tests can actually run before the completed
queries table has been fully created by the Impala startup
process. The table create sql runs asynchronously during
startup and thus can take longer to finish than the custom
cluster tests take to execute.

This change adds checks at the beginning of each test to
ensure the completed queries table sql has finished before
any of the test code runs.

Change-Id: I428702a210e024db95808dc2518da497426922f8
---
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_query_log.py
2 files changed, 43 insertions(+), 22 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/21221/7
-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 1:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/15716/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 1
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 18:07:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21221

to look at the new patch set (#5).

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................

IMPALA-12944: Fixes Workload Management Test Flakiness

The custom cluster workload management tests are flaky
because the tests can actually run before the completed
queries table has been fully created by the Impala startup
process. The table create sql runs asynchronously during
startup and thus can take longer to finish than the custom
cluster tests take to execute.

This change adds checks at the beginning of each test to
ensure the completed queries table sql has finished before
any of the test code runs.

Change-Id: I428702a210e024db95808dc2518da497426922f8
---
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_query_log.py
2 files changed, 43 insertions(+), 22 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/21221/5
-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 5
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/21221/1/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/1/tests/custom_cluster/test_query_log.py@53
PS1, Line 53:  
flake8: E251 unexpected spaces around keyword / parameter equals


http://gerrit.cloudera.org:8080/#/c/21221/1/tests/custom_cluster/test_query_log.py@53
PS1, Line 53:  
flake8: E251 unexpected spaces around keyword / parameter equals



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 1
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 17:48:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 4: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 4
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 20:49:32 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py@73
PS3, Line 73: 
I just remember, if you inspect logs like this, you need to ensure that each custom cluster start with their own empty log dir. Otherwise, this can mistakenly grep log lines from previous cluster. This is why test_breakpad.py start each custom cluster run with tmp_dir as log dir.



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 6
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 20:59:03 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 9:

gerrit-verify-dryrun


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 9
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Mar 2024 15:27:04 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 10: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 10
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Mar 2024 20:49:57 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 10:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10469/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 10
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Mar 2024 15:49:19 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21221/2/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/2/tests/custom_cluster/test_query_log.py@64
PS2, Line 64:     create_re = r'\]\s+(\w+:\w+)\]\s+Analyzing query: CREATE TABLE IF NOT EXISTS {}' \
> I thought you might grep the output of `show tables in sys`.
You can also extend "def setup_method(self, method)" for TestQueryLogTableBase to wait for any precondition necessary before running test method. See test_breakpad.py for example.



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 18:34:51 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21221/2/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/2/tests/custom_cluster/test_query_log.py@64
PS2, Line 64:     create_re = r'\]\s+(\w+:\w+)\]\s+Analyzing query: CREATE TABLE IF NOT EXISTS {}' \
I thought you might grep the output of `show tables in sys`.

If you want to keep the log approach, use assert_log_contains, which retries until reaching a timeout. It also knows where the log dir is, so you don't need all the extra changes.



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 18:26:13 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 7:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/21221/6/tests/common/impala_test_suite.py
File tests/common/impala_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/21221/6/tests/common/impala_test_suite.py@1270
PS6, Line 1270: return self.assert_log_contains(
              :         "impalad", level, line_re
> nit: looks better if breaking after parenthesis. No need for backslash.
Done


http://gerrit.cloudera.org:8080/#/c/21221/6/tests/common/impala_test_suite.py@1293
PS6, Line 1293: Returns the result of the very last call to line_regex.search 
> nit: "Returns the result of the very last call to line_regex.search or None
Done


http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/3/tests/custom_cluster/test_query_log.py@73
PS3, Line 73: 
> Right, nevermind.
I specifically tested this scenario and found that the impalad.INFO softlink does indeed change for each test.



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Mar 2024 01:03:44 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 7: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Mar 2024 01:08:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 10: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 10
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Mar 2024 15:34:58 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................

IMPALA-12944: Fixes Workload Management Test Flakiness

The custom cluster workload management tests are flaky
because the tests can actually run before the completed
queries table has been fully created by the Impala startup
process. The table create sql runs asynchronously during
startup and thus can take longer to finish than the custom
cluster tests take to execute.

This change adds checks at the beginning of each test to
ensure the completed queries table sql has finished before
any of the test code runs.

Change-Id: I428702a210e024db95808dc2518da497426922f8
Reviewed-on: http://gerrit.cloudera.org:8080/21221
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_query_log.py
2 files changed, 43 insertions(+), 22 deletions(-)

Approvals:
  Michael Smith: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 12
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21221

to look at the new patch set (#2).

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................

IMPALA-12944: Fixes Workload Management Test Flakiness

The custom cluster workload management tests are flaky
because the tests can actually run before the completed
queries table has been fully created by the Impala startup
process. The table create sql runs asynchronously during
startup and thus can take longer to finish than the custom
cluster tests take to execute.

This change adds checks at the beginning of each test to
ensure the completed queries table sql has finished before
any of the test code runs.

Change-Id: I428702a210e024db95808dc2518da497426922f8
---
M tests/custom_cluster/test_query_log.py
1 file changed, 87 insertions(+), 29 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/21221/2
-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 2:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/15717/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 18:11:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21221/2/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21221/2/tests/custom_cluster/test_query_log.py@64
PS2, Line 64:        table sql has finished."""
> You can also extend "def setup_method(self, method)" for TestQueryLogTableB
I started down the path of running the `show tables in sys` query but quickly ran into a problem since most of the tests expect a defined number of queries to be executed.  Trying to modify each test to handle the variable number of queries that could be executed to verify the completed queries table existed was going to require a large amount of effort.

I switched to assert_log_contains after making a modification to it.

I wish I could use use setup_method, but I cannot because the cluster has not yet started when that method is called.



-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 20:43:05 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21221

to look at the new patch set (#6).

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................

IMPALA-12944: Fixes Workload Management Test Flakiness

The custom cluster workload management tests are flaky
because the tests can actually run before the completed
queries table has been fully created by the Impala startup
process. The table create sql runs asynchronously during
startup and thus can take longer to finish than the custom
cluster tests take to execute.

This change adds checks at the beginning of each test to
ensure the completed queries table sql has finished before
any of the test code runs.

Change-Id: I428702a210e024db95808dc2518da497426922f8
---
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_query_log.py
2 files changed, 42 insertions(+), 22 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/21221/6
-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 6
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 4:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/15726/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 4
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 21:08:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12944: Fixes Workload Management Test Flakiness

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21221 )

Change subject: IMPALA-12944: Fixes Workload Management Test Flakiness
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/15728/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21221
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I428702a210e024db95808dc2518da497426922f8
Gerrit-Change-Number: 21221
Gerrit-PatchSet: 5
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 28 Mar 2024 21:22:19 +0000
Gerrit-HasComments: No