You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Jason Fehr (Code Review)" <ge...@cloudera.org> on 2024/11/25 21:01:38 UTC
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Jason Fehr has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22118
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
IMPALA-13536: Fix Workload Management Init with Catalog HA
When running an Impala cluster with catalogd HA enabled, the standby
catalogd would go into a loop waiting for the first catalog update to
arrive repeatedly logging the same error and never joining the server
thread defined in catalogd-main.cc.
When the standby daemon became active, the first catalogd update was
finally received, and the workload management initialization process
ran a second time in the newly active daemon because this daemon
saw that it was active.
The catalogd workload management initialization code now waits until
the active catalogd has been determined. At that point, the standby
daemon skips workload management initialization while the
active daemon runs it after it receives the first catalog update.
Testing was accomplished by modifying the workload management
initialization custom cluster tests to assert that the init process
is not re-run when a catalogd switches from standby to active and
also to remove the assumption that the first catalogd was active. The
test_catalog_ha test was deleted since all its assertions are handled
by the setup_method of the new TestWorkloadManagementCatalogHA class.
Ozone tests with and without erasure coding were also ran and passed.
Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
---
M be/src/catalog/catalog-server.cc
M be/src/catalog/catalog-server.h
M be/src/catalog/workload-management-init.cc
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
5 files changed, 101 insertions(+), 65 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/22118/1
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 1
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/22118 )
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
IMPALA-13536: Fix Workload Management Init with Catalog HA
When running an Impala cluster with catalogd HA enabled, the standby
catalogd would go into a loop waiting for the first catalog update to
arrive repeatedly logging the same error and never joining the server
thread defined in catalogd-main.cc.
Before this patch, when the standby daemon became active, the first
catalogd update was finally received, and the workload management
initialization process ran a second time in the newly active daemon
because this daemon saw that it was active.
This patch modifies the catalogd workload management initialization
code so it waits until the active catalogd has been determined. At
that point, the standby daemon skips workload management
initialization while the active daemon runs it after it receives the
first catalog update.
Testing was accomplished by modifying the workload management
initialization custom cluster tests to assert that the init process
is not re-run when a catalogd switches from standby to active and
also to remove the assumption that the first catalogd was active. The
test_catalog_ha test was deleted since all its assertions are handled
by the setup_method of the new TestWorkloadManagementCatalogHA class.
Ozone tests with and without erasure coding were also ran and passed.
Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
---
M be/src/catalog/catalog-server.cc
M be/src/catalog/catalog-server.h
M be/src/catalog/catalogd-main.cc
M be/src/catalog/workload-management-init.cc
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
6 files changed, 132 insertions(+), 73 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/22118/3
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/22118 )
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/22118/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/22118/1//COMMIT_MSG@14
PS1, Line 14: Before this patch, when the standby daemon became active, the first
: catalogd update was finally received, and the workload management
: initialization process ran a second time in the newly active daemon
: because this daemon saw
> This is the bad behavior?
Yes, updated the commit message to clarify.
http://gerrit.cloudera.org:8080/#/c/22118/1//COMMIT_MSG@19
PS1, Line 19: This patch modifies the catalogd workload management initialization
: code so it waits until the active catalogd has been determined. At
: that point, the standby daemon skips workload management
: initialization while the active daemon runs it after it receives
> This is the corrected behavior?
Yes, updated the commit message to clarify.
http://gerrit.cloudera.org:8080/#/c/22118/1/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:
http://gerrit.cloudera.org:8080/#/c/22118/1/tests/custom_cluster/test_workload_mgmt_init.py@583
PS1, Line 583: assert len(init_logs) == 2, "Expected length of catalogd matches to be '2' but " \
: "was '{}'".format(len(init_logs))
:
:
> I think Standby CatalogD should explicitly log that it skip workload manage
Done
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 22:37:23 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22118 )
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/22118/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/22118/1//COMMIT_MSG@14
PS1, Line 14: When the standby daemon became active, the first catalogd update was
: finally received, and the workload management initialization process
: ran a second time in the newly active daemon because this daemon
: saw that it was active.
This is the bad behavior?
http://gerrit.cloudera.org:8080/#/c/22118/1//COMMIT_MSG@19
PS1, Line 19: The catalogd workload management initialization code now waits until
: the active catalogd has been determined. At that point, the standby
: daemon skips workload management initialization while the
: active daemon runs it after it receives the first catalog update.
This is the corrected behavior?
http://gerrit.cloudera.org:8080/#/c/22118/1/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:
http://gerrit.cloudera.org:8080/#/c/22118/1/tests/custom_cluster/test_workload_mgmt_init.py@583
PS1, Line 583: # Assert only 1 catalog ran workload management initialization.
: assert init_logs[0] is None or init_logs[1] is None, "Both catalogds ran workload " \
: "management initialization"
:
I think Standby CatalogD should explicitly log that it skip workload management initialization because it is not the first CatalogD that selected as an active one.
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 1
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 21:21:31 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22118 )
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Nov 2024 02:31:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22118 )
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/17579/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 1
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 21:28:47 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Jason Fehr (Code Review)" <ge...@cloudera.org>.
Hello Andrew Sherman, Riza Suminto, Abhishek Rawat, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22118
to look at the new patch set (#2).
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
IMPALA-13536: Fix Workload Management Init with Catalog HA
When running an Impala cluster with catalogd HA enabled, the standby
catalogd would go into a loop waiting for the first catalog update to
arrive repeatedly logging the same error and never joining the server
thread defined in catalogd-main.cc.
Before this patch, when the standby daemon became active, the first
catalogd update was finally received, and the workload management
initialization process ran a second time in the newly active daemon
because this daemon saw that it was active.
This patch modifies the catalogd workload management initialization
code so it waits until the active catalogd has been determined. At
that point, the standby daemon skips workload management
initialization while the active daemon runs it after it receives the
first catalog update.
Testing was accomplished by modifying the workload management
initialization custom cluster tests to assert that the init process
is not re-run when a catalogd switches from standby to active and
also to remove the assumption that the first catalogd was active. The
test_catalog_ha test was deleted since all its assertions are handled
by the setup_method of the new TestWorkloadManagementCatalogHA class.
Ozone tests with and without erasure coding were also ran and passed.
Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
---
M be/src/catalog/catalog-server.cc
M be/src/catalog/catalog-server.h
M be/src/catalog/catalogd-main.cc
M be/src/catalog/workload-management-init.cc
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
6 files changed, 130 insertions(+), 73 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/22118/2
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22118 )
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/11177/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 26 Nov 2024 20:51:15 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22118 )
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
Patch Set 2:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/22118/2/be/src/catalog/workload-management-init.cc
File be/src/catalog/workload-management-init.cc:
http://gerrit.cloudera.org:8080/#/c/22118/2/be/src/catalog/workload-management-init.cc@502
PS2, Line 502: The first expression evaluates to true when the first catalog update is sent.
The first expression evaluates to true when the active CatalogD has sent its first catalog update.
http://gerrit.cloudera.org:8080/#/c/22118/2/be/src/catalog/workload-management-init.cc@503
PS2, Line 503: coordinator
CatalogD
http://gerrit.cloudera.org:8080/#/c/22118/2/tests/common/impala_test_suite.py
File tests/common/impala_test_suite.py:
http://gerrit.cloudera.org:8080/#/c/22118/2/tests/common/impala_test_suite.py@1411
PS2, Line 1411: matches.append(None)
Instead of artificially appending None, you can make log line for both Active and Standby CatalogD to have common prefix and capture the prefix instead. For example:
Active:
Workload management initialization is complete. This is an Active Catalog.
Standby:
Workload management initialization is skipped. This is a Standby Catalog.
http://gerrit.cloudera.org:8080/#/c/22118/1/tests/custom_cluster/test_workload_mgmt_init.py
File tests/custom_cluster/test_workload_mgmt_init.py:
http://gerrit.cloudera.org:8080/#/c/22118/1/tests/custom_cluster/test_workload_mgmt_init.py@583
PS1, Line 583: assert len(init_logs) == 2, "Expected length of catalogd matches to be '2' but " \
: "was '{}'".format(len(init_logs))
:
:
> Done
Done
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 23:14:01 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22118 )
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/17581/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 2
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 25 Nov 2024 23:04:31 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22118 )
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/17598/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 3
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 26 Nov 2024 21:01:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13536: Fix Workload Management Init with Catalog HA
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22118 )
Change subject: IMPALA-13536: Fix Workload Management Init with Catalog HA
......................................................................
IMPALA-13536: Fix Workload Management Init with Catalog HA
When running an Impala cluster with catalogd HA enabled, the standby
catalogd would go into a loop waiting for the first catalog update to
arrive repeatedly logging the same error and never joining the server
thread defined in catalogd-main.cc.
Before this patch, when the standby daemon became active, the first
catalogd update was finally received, and the workload management
initialization process ran a second time in the newly active daemon
because this daemon saw that it was active.
This patch modifies the catalogd workload management initialization
code so it waits until the active catalogd has been determined. At
that point, the standby daemon skips workload management
initialization while the active daemon runs it after it receives the
first catalog update.
Testing was accomplished by modifying the workload management
initialization custom cluster tests to assert that the init process
is not re-run when a catalogd switches from standby to active and
also to remove the assumption that the first catalogd was active. The
test_catalog_ha test was deleted since all its assertions are handled
by the setup_method of the new TestWorkloadManagementCatalogHA class.
Ozone tests with and without erasure coding were also ran and passed.
Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Reviewed-on: http://gerrit.cloudera.org:8080/22118
Reviewed-by: Riza Suminto <ri...@cloudera.com>
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/catalog/catalog-server.cc
M be/src/catalog/catalog-server.h
M be/src/catalog/catalogd-main.cc
M be/src/catalog/workload-management-init.cc
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_workload_mgmt_init.py
6 files changed, 132 insertions(+), 73 deletions(-)
Approvals:
Riza Suminto: Looks good to me, but someone else must approve
Michael Smith: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/22118
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id3797a0a9cf0b8ae844d9b7d46b607d93824f69a
Gerrit-Change-Number: 22118
Gerrit-PatchSet: 4
Gerrit-Owner: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jason Fehr <jf...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>