You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Jason Fehr (Code Review)" <ge...@cloudera.org> on 2024/11/25 21:51:10 UTC

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Jason Fehr has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22119


Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................

IMPALA-13536: Fix Workload Management Init Tests Issues

Several problems with the workload management code and
test_workload_mgmt_init.py tests have been uncovered by the Ozone
tests.

* test_create_on_version_1_0_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_create_on_version_1_1_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_invalid_* - All four of these tests run the same internal
      function to execute the test. This internal function was not
      waiting long enough for the expected failure to appear. The
      fixed internal function waits longer for the expected failure.

Additionally, custom_cluster_test_suite.py now resolves the daemon
log file symlinks to their actual paths which are then logged. This
change enables easier Jenkins build debugging by enabling failed
tests to be traced to the exact log files for that test.

Modified tests passed on Ozone builds both with and without erasure
coding enabled.

Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
---
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
2 files changed, 23 insertions(+), 5 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/22119/1
-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 1
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 2: -Code-Review

(1 comment)

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py@510
PS2, Line 510:       LOG.info("Log Files for Test:\n{}".format(file_list))
> This will be printed for all custom cluster tests and becomes a noise.
Yeah, I'm also not sure this is that helpful. Finding the right file by timestamp isn't too hard either.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 22:54:09 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/22119/7/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/7/tests/custom_cluster/test_workload_mgmt_init.py@150
PS7, Line 150:     self.restart_cluster(vector, schema_version="1.1.0", log_symlinks=True)
             :     self.check_schema("1.1.0", vector)
> When will we stop writing "schema_version" property? Impala 5 or WM 2.0.0?
We can stop writing it with Impala 5.  IMPALA-13595 has been created and linked to that epic.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 8
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Dec 2024 21:02:18 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/22119/5/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/5/tests/common/custom_cluster_test_suite.py@460
PS5, Line 460:     # List log files from this cluster startup to identify log files for failed tests.
I don't see how this is limited to failed tests.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 5
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 17:21:48 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 7:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/22119/7/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/7/tests/custom_cluster/test_workload_mgmt_init.py@150
PS7, Line 150:     self.restart_cluster(vector, schema_version="1.1.0", log_symlinks=True)
             :     self.check_schema("1.1.0", vector)
> Why schema_version property still exist in no_upgrade scenario?
The "schema_version" property is the old table property.  Since previous versions of workload management did not handle changes in this property value, I had to add the new property named "wm_schema_version" to maintain backwards compatibility.  For more details, see the comment at https://github.com/apache/impala/blob/79eb48e9f63a87bf240080949c11e90b47c1367f/be/src/catalog/workload-management-init.cc#L416-L418



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Dec 2024 19:55:09 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 8:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/11154/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 8
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Dec 2024 07:02:16 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py@510
PS2, Line 510:       LOG.info("Log Files for Test:\n{}".format(file_list))
This will be printed for all custom cluster tests and becomes a noise.
Is it still helpful even though assert.*log_contains function already print out the actual log file path?

For example, 10 impalads cluster may print 40 file paths for all INFO, WARNING, ERROR, and FATAL.
If test does set specific impalad_log_dir, it may print all resolved paths for old symlinks left behind under /tmp/.


http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py@193
PS2, Line 193: cluster_size=10
Now that workload management init done by CatalogD, is it still important to vary the cluster size?



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 22:52:33 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/11172/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 22:44:31 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py@510
PS2, Line 510:       LOG.info("Log Files for Test:\n{}".format(file_list))
> Yeah, I'm also not sure this is that helpful. Finding the right file by tim
This log message will only be printed when a test fails.

When the logfile assertion functions fail, they print the log file that was examined but do not print out other files for the cluster.

When diagnosing build failures, I have frequently found myself looking across log files, and identifying the exact files to review is difficult especially for tests that run close together.

The start-impala-cluster.py script removes old symlinks.  since impala_log_dir is also passed to the start-impala-cluster.py script, there will not be any old sym links listed here.


http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py@193
PS2, Line 193: cluster_size=10
> Now that workload management init done by CatalogD, is it still important t
It's not as important, but still good to test to ensure clusters greater than 3 nodes do not have communication issues.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 23:58:13 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................

IMPALA-13536: Fix Workload Management Init Tests Issues

Several problems with the workload management code and
test_workload_mgmt_init.py tests have been uncovered by the Ozone
tests.

* test_create_on_version_1_0_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_create_on_version_1_1_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_invalid_* - All four of these tests run the same internal
      function to execute the test. This internal function was not
      waiting long enough for the expected failure to appear. The
      fixed internal function waits longer for the expected failure.

Additionally, custom_cluster_test_suite.py now resolves the daemon
log file symlinks to their actual paths which are then logged. This
change enables easier Jenkins build debugging by enabling failed
tests to be traced to the exact log files for that test.

Modified tests passed on Ozone builds both with and without erasure
coding enabled.

Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
---
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
2 files changed, 25 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/22119/2
-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 8:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/17639/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 8
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Dec 2024 21:29:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................

IMPALA-13536: Fix Workload Management Init Tests Issues

Several problems with the workload management code and
test_workload_mgmt_init.py tests have been uncovered by the Ozone
tests.

* test_create_on_version_1_0_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_create_on_version_1_1_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_invalid_* - All four of these tests run the same internal
      function to execute the test. This internal function was not
      waiting long enough for the expected failure to appear. The
      fixed internal function waits longer for the expected failure.

Additionally, the @CustomClusterTestSuite annotation has a new option
named 'log_symlinks', which, if set to True will resolve all daemon
log symlinks and output their actual paths to the log. Failed tests
can then be easily traced to the exact log files for that test.

The existing workload management tests in testdata have been expanded
to also assert the expected table properties are present.

Modified tests passed on Ozone builds both with and without erasure
coding enabled.

Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Reviewed-on: http://gerrit.cloudera.org:8080/22119
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
D testdata/workloads/functional-query/queries/QueryTest/workload-management-log-v1.0.0.test
D testdata/workloads/functional-query/queries/QueryTest/workload-management-log-v1.1.0.test
R testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_live-v1.0.0.test
R testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_live-v1.1.0.test
A testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_log-v1.0.0.test
A testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_log-v1.1.0.test
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
M tests/util/workload_management.py
9 files changed, 265 insertions(+), 247 deletions(-)

Approvals:
  Michael Smith: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 10
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/22119/1/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/1/tests/common/custom_cluster_test_suite.py@377
PS1, Line 377:     Waits for the catalog to report the workload management initialization process has
Did we talk about replacing this with a check like `describe sys.impala_query_log` to see that the table was available?


http://gerrit.cloudera.org:8080/#/c/22119/1/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/1/tests/custom_cluster/test_workload_mgmt_init.py@186
PS1, Line 186:        Cluster sizes of 1 are used to speed up the initial setup."""
This comment no longer applies.


http://gerrit.cloudera.org:8080/#/c/22119/1/tests/custom_cluster/test_workload_mgmt_init.py@201
PS1, Line 201:        Cluster sizes of 1 are used to speed up the initial setup."""
This comment no longer applies.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 1
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 22:07:59 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 6:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/17606/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 6
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 22:42:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................

IMPALA-13536: Fix Workload Management Init Tests Issues

Several problems with the workload management code and
test_workload_mgmt_init.py tests have been uncovered by the Ozone
tests.

* test_create_on_version_1_0_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_create_on_version_1_1_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_invalid_* - All four of these tests run the same internal
      function to execute the test. This internal function was not
      waiting long enough for the expected failure to appear. The
      fixed internal function waits longer for the expected failure.

Additionally, custom_cluster_test_suite.py now resolves the daemon
log file symlinks to their actual paths which are then logged. This
change enables easier Jenkins build debugging by enabling failed
tests to be traced to the exact log files for that test.

Modified tests passed on Ozone builds both with and without erasure
coding enabled.

Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
---
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
2 files changed, 87 insertions(+), 44 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/22119/3
-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 9: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 9
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Dec 2024 19:26:29 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 7:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/22119/7/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/7/tests/custom_cluster/test_workload_mgmt_init.py@150
PS7, Line 150:     self.restart_cluster(vector, schema_version="1.1.0", log_symlinks=True)
             :     self.check_schema("1.1.0", vector)
> The "schema_version" property is the old table property.  Since previous ve
When will we stop writing "schema_version" property? Impala 5 or WM 2.0.0?
Please file a JIRA to remind us to stop writing "schema_version" and link it to IMPALA-12674 epic.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Dec 2024 20:28:02 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 7:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/22119/7/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/7/tests/custom_cluster/test_workload_mgmt_init.py@150
PS7, Line 150:     self.restart_cluster(vector, schema_version="1.1.0", log_symlinks=True)
             :     self.check_schema("1.1.0", vector)
Why schema_version property still exist in no_upgrade scenario?
I thought wm_schema_version replace that since 1.1.0.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 23:15:37 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 2: Code-Review+1

(2 comments)

http://gerrit.cloudera.org:8080/#/c/22119/1/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/1/tests/custom_cluster/test_workload_mgmt_init.py@186
PS1, Line 186:     self.check_schema_version("1.0.0")
> Done
Ack


http://gerrit.cloudera.org:8080/#/c/22119/1/tests/custom_cluster/test_workload_mgmt_init.py@201
PS1, Line 201: 
> Done
Ack



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 22:43:11 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 5:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/22119/5/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/5/tests/common/custom_cluster_test_suite.py@460
PS5, Line 460:     # List log files from this cluster startup to identify log files for failed tests.
> I don't see how this is limited to failed tests.
It's only limited by the fact that succeeding tests hide all stdout/stderr.  Comment removed.


http://gerrit.cloudera.org:8080/#/c/22119/5/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/5/tests/custom_cluster/test_workload_mgmt_init.py@123
PS5, Line 123:   def check_schema_version(self, schema_version, protocol=None):
> Can this be a .test file instead? Please check some examples in testdata/
Good idea.  I was able to expand the existing .test files to also validate the table properties.  This change in turn allowed for the elimination of even more code.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 5
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 22:15:46 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 9:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/11194/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 9
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Dec 2024 19:26:57 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 3:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py@510
PS2, Line 510:     finally:
> When start-impala-cluster.py runs, it removes all existing symlinks (at lea
Done


http://gerrit.cloudera.org:8080/#/c/22119/3/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/3/tests/common/custom_cluster_test_suite.py@512
PS3, Line 512:         # List log files from this cluster startup to identify log files for failed tests.
             :         file_list = ""
             :         for pattern in ["*.INFO", "*.WARNING", "*.ERROR", "*.FATAL"]:
             :           matching_files = glob(os.path.join(impala_log_dir, pattern))
             :           for matching_file in sorted(matching_files):
             :             file_list += "  * {} - {}\n".format(matching_file.split(os.path.sep)[-1],
             :                 os.path.realpath(matching_file))
             :         LOG.info("Log Files for Test:\n{}".format(file_list))
nit: might want to contain this in its own method.


http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py@193
PS2, Line 193: le sql was used
> I think it's a good idea to connect to different coordinators.  I modified 
Sorry if I'm not clear before. You don't need to add create_random_impala_client() for this. run_test_case method already facilitate that.

  def run_test_case(self, test_file_name, vector, use_db=None, multiple_impalad=False,
      encoding=None, test_file_vars=None):
    """
    Runs the queries in the specified test based on the vector values

    Runs the query using targeting the file format/compression specified in the test
    vector and the exec options specified in the test vector. If multiple_impalad=True
    a connection to a random impalad will be chosen to execute each test section.
    Otherwise, the default impalad client will be used. If 'protocol' (either 'hs2' or
    'beeswax') is set in the vector, a client for that protocol is used. Otherwise we
    use the default: beeswax.

So instead, you can simply do:

    # Asserts the correct create table sql was used.
    self.run_test_case('QueryTest/workload-management-log-v1.1.0', vector, self.WM_DB, multiple_impalad=True)
    self.run_test_case('QueryTest/workload-management-live-v1.1.0', vector, self.WM_DB, multiple_impalad=True)



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 26 Nov 2024 20:56:14 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/17599/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 4
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 26 Nov 2024 21:13:07 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/22119/1/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/1/tests/common/custom_cluster_test_suite.py@377
PS1, Line 377:     Waits for the catalog to report the workload management initialization process has
> Did we talk about replacing this with a check like `describe sys.impala_que
Yeah, we talked about it.  I went with log message assertions because some tests assert on the number of queries executed, and "describe sys.impala_query_..." would have to be executed a non-deterministic number of times.


http://gerrit.cloudera.org:8080/#/c/22119/1/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/1/tests/custom_cluster/test_workload_mgmt_init.py@186
PS1, Line 186:     self.check_schema_version("1.0.0")
> This comment no longer applies.
Done


http://gerrit.cloudera.org:8080/#/c/22119/1/tests/custom_cluster/test_workload_mgmt_init.py@201
PS1, Line 201: 
> This comment no longer applies.
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 22:42:44 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 4:

Patch 4 rebases https://gerrit.cloudera.org/#/c/22118 onto this change.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 4
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 26 Nov 2024 20:50:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Andrew Sherman, Riza Suminto, Abhishek Rawat, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/22119

to look at the new patch set (#4).

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................

IMPALA-13536: Fix Workload Management Init Tests Issues

Several problems with the workload management code and
test_workload_mgmt_init.py tests have been uncovered by the Ozone
tests.

* test_create_on_version_1_0_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_create_on_version_1_1_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_invalid_* - All four of these tests run the same internal
      function to execute the test. This internal function was not
      waiting long enough for the expected failure to appear. The
      fixed internal function waits longer for the expected failure.

Additionally, custom_cluster_test_suite.py now resolves the daemon
log file symlinks to their actual paths which are then logged. This
change enables easier Jenkins build debugging by enabling failed
tests to be traced to the exact log files for that test.

Modified tests passed on Ozone builds both with and without erasure
coding enabled.

Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
---
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
2 files changed, 86 insertions(+), 43 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/22119/4
-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 4
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/22119/3/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/3/tests/common/custom_cluster_test_suite.py@512
PS3, Line 512:         # List log files from this cluster startup to identify log files for failed tests.
             :         file_list = ""
             :         for pattern in ["*.INFO", "*.WARNING", "*.ERROR", "*.FATAL"]:
             :           matching_files = glob(os.path.join(impala_log_dir, pattern))
             :           for matching_file in sorted(matching_files):
             :             file_list += "  * {} - {}\n".format(matching_file.split(os.path.sep)[-1],
             :                 os.path.realpath(matching_file))
             :         LOG.info("Log Files for Test:\n{}".format(file_list))
> nit: might want to contain this in its own method.
good idea, done


http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py@193
PS2, Line 193: le sql was used
> Sorry if I'm not clear before. You don't need to add create_random_impala_c
Ah, I'm understanding now.  You were clear, I just wasn't getting how to apply it to these tests.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 16:43:59 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/11179/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 5
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 16:44:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 5: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 5
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 22:12:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/17605/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 5
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 17:12:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 9: Code-Review+2

Carry Riza's +1.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 9
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Dec 2024 19:26:38 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 8: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/22119/7/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/7/tests/custom_cluster/test_workload_mgmt_init.py@150
PS7, Line 150:     self.restart_cluster(vector, schema_version="1.1.0", log_symlinks=True)
             :     self.check_schema("1.1.0", vector)
> We can stop writing it with Impala 5.  IMPALA-13595 has been created and li
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 8
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Dec 2024 21:15:19 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/17580/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 1
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 22:17:23 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py@510
PS2, Line 510:       LOG.info("Log Files for Test:\n{}".format(file_list))
> Do you mean when test fail or when start-impala-cluster.py fail? The code s
When start-impala-cluster.py runs, it removes all existing symlinks (at least that is what it did yesterday on my local dev machine when I tested it).

I was thinking more about this topic last night, and I agree that printing the resolved symlinks will create noise as the majority of tests don't need to trace errors across multiple log files.  I created a new parameter on CustomClusterTestSuite that will log the symlinks if specified.


http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py@193
PS2, Line 193: cluster_size=10
> Seems unnecessary to start 10 nodes to only run the test file against just 
I think it's a good idea to connect to different coordinators.  I modified the code to randomly choose a coordinator in situations where more than 1 coordinator is configured.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 26 Nov 2024 20:30:48 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/22119/5/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/5/tests/custom_cluster/test_workload_mgmt_init.py@123
PS5, Line 123:   def check_schema_version(self, schema_version, protocol=None):
Can this be a .test file instead? Please check some examples in testdata/

git grep -in 'describe formatted' testdata/

If possible, then create_random_impala_client is not needed anymore.



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 5
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 17:32:18 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/17597/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 26 Nov 2024 20:57:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Andrew Sherman, Riza Suminto, Abhishek Rawat, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/22119

to look at the new patch set (#8).

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................

IMPALA-13536: Fix Workload Management Init Tests Issues

Several problems with the workload management code and
test_workload_mgmt_init.py tests have been uncovered by the Ozone
tests.

* test_create_on_version_1_0_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_create_on_version_1_1_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_invalid_* - All four of these tests run the same internal
      function to execute the test. This internal function was not
      waiting long enough for the expected failure to appear. The
      fixed internal function waits longer for the expected failure.

Additionally, the @CustomClusterTestSuite annotation has a new option
named 'log_symlinks', which, if set to True will resolve all daemon
log symlinks and output their actual paths to the log. Failed tests
can then be easily traced to the exact log files for that test.

The existing workload management tests in testdata have been expanded
to also assert the expected table properties are present.

Modified tests passed on Ozone builds both with and without erasure
coding enabled.

Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
---
D testdata/workloads/functional-query/queries/QueryTest/workload-management-log-v1.0.0.test
D testdata/workloads/functional-query/queries/QueryTest/workload-management-log-v1.1.0.test
R testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_live-v1.0.0.test
R testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_live-v1.1.0.test
A testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_log-v1.0.0.test
A testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_log-v1.1.0.test
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
M tests/util/workload_management.py
9 files changed, 265 insertions(+), 247 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/22119/8
-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 8
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/17582/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 23:09:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Andrew Sherman, Riza Suminto, Abhishek Rawat, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/22119

to look at the new patch set (#5).

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................

IMPALA-13536: Fix Workload Management Init Tests Issues

Several problems with the workload management code and
test_workload_mgmt_init.py tests have been uncovered by the Ozone
tests.

* test_create_on_version_1_0_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_create_on_version_1_1_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_invalid_* - All four of these tests run the same internal
      function to execute the test. This internal function was not
      waiting long enough for the expected failure to appear. The
      fixed internal function waits longer for the expected failure.

Additionally, custom_cluster_test_suite.py now resolves the daemon
log file symlinks to their actual paths which are then logged. This
change enables easier Jenkins build debugging by enabling failed
tests to be traced to the exact log files for that test.

Modified tests passed on Ozone builds both with and without erasure
coding enabled.

Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
---
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
2 files changed, 111 insertions(+), 49 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/22119/5
-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 5
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/common/custom_cluster_test_suite.py@510
PS2, Line 510:       LOG.info("Log Files for Test:\n{}".format(file_list))
> This log message will only be printed when a test fails.
Do you mean when test fail or when start-impala-cluster.py fail? The code seems to print in any case since it is in finally block.

Also, if 2 test run and both log to same $IMPALA_LOG_DIR, where the first has cluster size 10 and the second has cluster size 3, then I think impalad_node4.INFO to impalad_node10.INFO will still be there in $IMPALA_LOG_DIR.


http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:

http://gerrit.cloudera.org:8080/#/c/22119/2/tests/custom_cluster/test_workload_mgmt_init.py@193
PS2, Line 193: cluster_size=10
> It's not as important, but still good to test to ensure clusters greater th
Seems unnecessary to start 10 nodes to only run the test file against just one of them.

I think it is good to randomly pick 1 of impalad as coordinator by setting multiple_impalad=True parameter in run_test_case. What do you think?



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 26 Nov 2024 00:20:08 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 2: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/11172/


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 26 Nov 2024 04:16:38 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 9: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 9
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Dec 2024 01:01:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 7:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/17607/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 22:59:32 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Andrew Sherman, Riza Suminto, Abhishek Rawat, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/22119

to look at the new patch set (#6).

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................

IMPALA-13536: Fix Workload Management Init Tests Issues

Several problems with the workload management code and
test_workload_mgmt_init.py tests have been uncovered by the Ozone
tests.

* test_create_on_version_1_0_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_create_on_version_1_1_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_invalid_* - All four of these tests run the same internal
      function to execute the test. This internal function was not
      waiting long enough for the expected failure to appear. The
      fixed internal function waits longer for the expected failure.

Additionally, custom_cluster_test_suite.py now resolves the daemon
log file symlinks to their actual paths which are then logged. This
change enables easier Jenkins build debugging by enabling failed
tests to be traced to the exact log files for that test.

Modified tests passed on Ozone builds both with and without erasure
coding enabled.

Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
---
D testdata/workloads/functional-query/queries/QueryTest/workload-management-log-v1.0.0.test
D testdata/workloads/functional-query/queries/QueryTest/workload-management-log-v1.1.0.test
R testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_live-v1.0.0.test
R testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_live-v1.1.0.test
A testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_log-v1.0.0.test
A testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_log-v1.1.0.test
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
M tests/util/workload_management.py
9 files changed, 265 insertions(+), 244 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/22119/6
-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 6
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22119 )

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................


Patch Set 6:

(1 comment)

gerrit-auto-critic failed. You can reproduce it locally using command:

  python2 bin/jenkins/critique-gerrit-review.py --dryrun

To run it, you might need a virtual env with virtualenv installed.

http://gerrit.cloudera.org:8080/#/c/22119/6/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/22119/6/tests/common/custom_cluster_test_suite.py@33
PS6, Line 33: from random import randint
flake8: F401 'random.randint' imported but unused



-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 6
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 22:16:49 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init Tests Issues

Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Andrew Sherman, Riza Suminto, Abhishek Rawat, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/22119

to look at the new patch set (#7).

Change subject: IMPALA-13536: Fix Workload Management Init Tests Issues
......................................................................

IMPALA-13536: Fix Workload Management Init Tests Issues

Several problems with the workload management code and
test_workload_mgmt_init.py tests have been uncovered by the Ozone
tests.

* test_create_on_version_1_0_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_create_on_version_1_1_0 - Test comment said it ran on 10
      nodes, test configuration specified 1 node. Fix was to modify
      the test configuration.
* test_invalid_* - All four of these tests run the same internal
      function to execute the test. This internal function was not
      waiting long enough for the expected failure to appear. The
      fixed internal function waits longer for the expected failure.

Additionally, the @CustomClusterTestSuite annotation has a new option
named 'log_symlinks', which, if set to True will resolve all daemon
log symlinks and output their actual paths to the log. Failed tests
can then be easily traced to the exact log files for that test.

The existing workload management tests in testdata have been expanded
to also assert the expected table properties are present.

Modified tests passed on Ozone builds both with and without erasure
coding enabled.

Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
---
D testdata/workloads/functional-query/queries/QueryTest/workload-management-log-v1.0.0.test
D testdata/workloads/functional-query/queries/QueryTest/workload-management-log-v1.1.0.test
R testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_live-v1.0.0.test
R testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_live-v1.1.0.test
A testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_log-v1.0.0.test
A testdata/workloads/functional-query/queries/QueryTest/workload-mgmt-impala_query_log-v1.1.0.test
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
M tests/util/workload_management.py
9 files changed, 263 insertions(+), 244 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/22119/7
-- 
To view, visit http://gerrit.cloudera.org:8080/22119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie3f34088d1d925f30abb63471387e6fdb62b95a7
Gerrit-Change-Number: 22119
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>