You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "ttttttz (Code Review)" <ge...@cloudera.org> on 2023/10/24 12:35:42 UTC
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
ttttttz has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20614
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
IMPALA-12513: Reset metadata when the CatalogD become active
Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
---
M be/src/catalog/catalog-server.cc
1 file changed, 11 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/20614/1
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 1
Gerrit-Owner: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20614
to look at the new patch set (#8).
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
When switching active catalogd, the loaded metadata in the standby
catalogd may not be the latest. A backend flag should be provided
to control whether to reset metadata when the catalogd become active.
Adds the following startup flags for catalogd:
'catalogd_ha_reset_metadata_on_failover'. Default is false. If true,
reset all metadata when the catalogd becomes active.
Testing:
- Added a test case to start both catalogds with flag
'catalogd_ha_reset_metadata_on_failover' as true.
- Passed core tests
Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
---
M be/src/catalog/catalog-server.cc
M tests/custom_cluster/test_catalogd_ha.py
2 files changed, 53 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/20614/8
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 8
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
ttttttz has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 4:
(1 comment)
> Patch Set 2:
>
> (1 comment)
http://gerrit.cloudera.org:8080/#/c/20614/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20614/2//COMMIT_MSG@9
PS2, Line 9: the loaded metadata in the candidate
: CatalogD may not be the latest.
> What are the causes for the loaded metadata in standby catalogd legging beh
There are serveral causes why the metadata loaded in the standby catalogd legging behind, such as:
1?The catalogd enables load metadata in the background.
2?After multiple statestored restarts, the catalog's HA state changes as follows: active > stanbdy > active.
3?Newly created user-defined functions will not be recognized after failover.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 4
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 25 Oct 2023 09:07:15 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
Patch Set 8:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py
File tests/custom_cluster/test_catalogd_ha.py:
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py@411
PS8, Line 411:
nit: two more spaces
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py@414
PS8, Line 414:
nit: two more spaces
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py@428
PS8, Line 428:
nit: two more spaces.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 8
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Tue, 07 Nov 2023 19:00:39 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14242/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 3
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 25 Oct 2023 09:03:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14234/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 1
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Tue, 24 Oct 2023 13:06:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20614/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20614/2//COMMIT_MSG@9
PS2, Line 9: the loaded metadata in the candidate
: CatalogD may not be the latest.
> There are serveral causes why the metadata loaded in the standby catalogd l
For "The catalogd enables load metadata in the background", we can enable "load_catalog_in_background" for standby catalogd, though it increase the network traffic. For "Newly created user-defined functions will not be recognized after failover", this issue is tracked by IMPALA-10273 (Support function event).
Since most of the users don't turn on load_catalog_in_background and don't create UDFs in ETL jobs, I prefer to add a flag reset_catalog_when_active.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 4
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 01 Nov 2023 03:32:08 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 5:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/14349/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 5
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Tue, 07 Nov 2023 03:43:10 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD become active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20614
to look at the new patch set (#7).
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD become active
......................................................................
IMPALA-12513: Allow to reset metadata when the CatalogD become active
When switching active catalogd, the loaded metadata in the standby
catalogd may not be the latest. A backend flag should be provided
to control whether to reset metadata when the catalogd become active.
Adds the following startup flags for catalogd:
'catalogd_ha_reset_metadata_on_failover'. Default is false. If true,
reset all metadata when the catalogd become active.
Testing:
- Added a test case to start both catalogds with flag
'catalogd_ha_reset_metadata_on_failover' as true.
- Passed core tests
Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
---
M be/src/catalog/catalog-server.cc
M tests/custom_cluster/test_catalogd_ha.py
2 files changed, 52 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/20614/7
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 7
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py
File tests/custom_cluster/test_catalogd_ha.py:
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py@395
PS8, Line 395: statestored_args="--enable_catalogd_ha=true "
: "--use_subscriber_id_as_catalogd_priority=true",
: catalogd_args="--enable_catalogd_ha=true "
: "--catalogd_ha_reset_metadata_on_failover=true"
The new test failed on my local machine. Changes the setting as:
statestored_args="--use_subscriber_id_as_catalogd_priority=true",
catalogd_args="--catalogd_ha_reset_metadata_on_failover=true",
start_args="--enable_catalogd_ha"
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 8
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Tue, 07 Nov 2023 17:38:43 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 4:
> > Patch Set 4:
> >
> > (1 comment)
>
> I understand. Indeed, in order to pursue performance, the standby
> catalog is likely to enable background metadata loading, despite
> the risk of metadata expiration.
Could you add a flag to turn on resetting metadata when CatalogD becomes active?
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 4
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 01 Nov 2023 21:14:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20614
to look at the new patch set (#2).
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
IMPALA-12513: Reset metadata when the CatalogD become active
When switching active CatalogD, the loaded metadata in the candidate
CatalogD may not be the latest. We should reset the metadata to ensure
that there are no metadata inconsistency exceptions after failover.
Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
---
M be/src/catalog/catalog-server.cc
1 file changed, 11 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/20614/2
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 2
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20614
to look at the new patch set (#3).
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
IMPALA-12513: Reset metadata when the CatalogD become active
When switching active CatalogD, the loaded metadata in the candidate
CatalogD may not be the latest. We should reset the metadata to ensure
that there are no metadata inconsistency exceptions after failover.
Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
---
M be/src/catalog/catalog-server.cc
M tests/custom_cluster/test_catalogd_ha.py
2 files changed, 46 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/20614/3
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 3
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
Patch Set 8:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14352/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 8
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Tue, 07 Nov 2023 06:15:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
ttttttz has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
Patch Set 9:
(4 comments)
Hi Wenzhe! I have submitted the latest code.
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py
File tests/custom_cluster/test_catalogd_ha.py:
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py@395
PS8, Line 395: statestored_args="--use_subscriber_id_as_catalogd_priority=true",
: catalogd_args="--catalogd_ha_reset_metadata_on_failover=true",
: start_args="--enable_catalogd_ha")
: def test_metadata_after_failover(self, unique_database):
> The new test failed on my local machine. Changes the setting as:
Done
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py@411
PS8, Line 411:
> nit: two more spaces
Done
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py@414
PS8, Line 414:
> nit: two more spaces
Done
http://gerrit.cloudera.org:8080/#/c/20614/8/tests/custom_cluster/test_catalogd_ha.py@428
PS8, Line 428:
> nit: two more spaces.
Done
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 9
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 08 Nov 2023 01:59:22 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14243/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 4
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 25 Oct 2023 09:37:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
ttttttz has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
Patch Set 8:
(4 comments)
Thanks Wenzhe!
http://gerrit.cloudera.org:8080/#/c/20614/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20614/7//COMMIT_MSG@11
PS7, Line 11: become
> nit: becomes
Done
http://gerrit.cloudera.org:8080/#/c/20614/7//COMMIT_MSG@14
PS7, Line 14: become
> nit: becomes
Done
http://gerrit.cloudera.org:8080/#/c/20614/7/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/20614/7/be/src/catalog/catalog-server.cc@548
PS7, Line 548: FLAGS
> upper case: FLAGS_
Done
http://gerrit.cloudera.org:8080/#/c/20614/7/tests/custom_cluster/test_catalogd_ha.py
File tests/custom_cluster/test_catalogd_ha.py:
http://gerrit.cloudera.org:8080/#/c/20614/7/tests/custom_cluster/test_catalogd_ha.py@395
PS7, Line 395: statestored_args="--enable_catalogd_ha=true "
> set statestored_args="--use_subscriber_id_as_catalogd_priority=true", other
Done
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 8
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Tue, 07 Nov 2023 05:50:11 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
ttttttz has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 4:
> Patch Set 4:
>
> > > Patch Set 4:
> > >
> > > (1 comment)
> >
> > I understand. Indeed, in order to pursue performance, the standby
> > catalog is likely to enable background metadata loading, despite
> > the risk of metadata expiration.
>
> Could you add a flag to turn on resetting metadata when CatalogD becomes active?
Okay, I am willing to do it!
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 4
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Thu, 02 Nov 2023 15:43:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20614
to look at the new patch set (#4).
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
IMPALA-12513: Reset metadata when the CatalogD become active
When switching active CatalogD, the loaded metadata in the candidate
CatalogD may not be the latest. We should reset the metadata to ensure
that there are no metadata inconsistency exceptions after failover.
Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
---
M be/src/catalog/catalog-server.cc
M tests/custom_cluster/test_catalogd_ha.py
2 files changed, 46 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/20614/4
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 4
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD become active
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD become active
......................................................................
Patch Set 7:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/20614/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20614/7//COMMIT_MSG@11
PS7, Line 11: become
nit: becomes
http://gerrit.cloudera.org:8080/#/c/20614/7//COMMIT_MSG@14
PS7, Line 14: become
nit: becomes
http://gerrit.cloudera.org:8080/#/c/20614/7/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/20614/7/be/src/catalog/catalog-server.cc@548
PS7, Line 548: Flags
upper case: FLAGS_
http://gerrit.cloudera.org:8080/#/c/20614/7/tests/custom_cluster/test_catalogd_ha.py
File tests/custom_cluster/test_catalogd_ha.py:
http://gerrit.cloudera.org:8080/#/c/20614/7/tests/custom_cluster/test_catalogd_ha.py@395
PS7, Line 395: statestored_args="--enable_catalogd_ha=true ",
set statestored_args="--use_subscriber_id_as_catalogd_priority=true", otherwise the statestore elect active catalogd randomly.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 7
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Tue, 07 Nov 2023 05:23:46 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20614/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20614/2//COMMIT_MSG@9
PS2, Line 9: the loaded metadata in the candidate
: CatalogD may not be the latest.
What are the causes for the loaded metadata in standby catalogd legging behind? How about adding a flag to keep metadata ready on standby catalogd if the flag is enabled, otherwise reset the metadata when it becomes active.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 2
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Tue, 24 Oct 2023 19:33:32 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20614
to look at the new patch set (#5).
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
IMPALA-12513: Reset metadata when the CatalogD become active
When switching active CatalogD, the loaded metadata in the candidate
CatalogD may not be the latest. We should reset the metadata to ensure
that there are no metadata inconsistency exceptions after failover.
Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
---
M be/src/catalog/catalog-server.cc
M tests/custom_cluster/test_catalogd_ha.py
2 files changed, 52 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/20614/5
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 5
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD become active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20614
to look at the new patch set (#6).
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD become active
......................................................................
IMPALA-12513: Allow to reset metadata when the CatalogD become active
When switching active catalogd, the loaded metadata in the standby
catalogd may not be the latest. A backend flag should be provided
to control whether to reset metadata when the catalogd become active.
Adds the following startup flags for catalogd:
'catalogd_ha_reset_metadata_on_failover'. Default is false. If true,
reset all metadata when the catalogd become active.
Testing:
- Added a test case to start both catalogds with flag
'catalogd_ha_reset_metadata_on_failover' as true.
- Passed core tests
Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
---
M be/src/catalog/catalog-server.cc
M tests/custom_cluster/test_catalogd_ha.py
2 files changed, 52 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/20614/6
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 6
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
Patch Set 10: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 10
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 08 Nov 2023 02:09:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
When switching active catalogd, the loaded metadata in the standby
catalogd may not be the latest. A backend flag should be provided
to control whether to reset metadata when the catalogd become active.
Adds the following startup flags for catalogd:
'catalogd_ha_reset_metadata_on_failover'. Default is false. If true,
reset all metadata when the catalogd becomes active.
Testing:
- Added a test case to start both catalogds with flag
'catalogd_ha_reset_metadata_on_failover' as true.
- Passed core tests
Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Reviewed-on: http://gerrit.cloudera.org:8080/20614
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/catalog/catalog-server.cc
M tests/custom_cluster/test_catalogd_ha.py
2 files changed, 52 insertions(+), 0 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 11
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD become active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD become active
......................................................................
Patch Set 7:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/14351/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 7
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Tue, 07 Nov 2023 04:42:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20614
to look at the new patch set (#9).
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
When switching active catalogd, the loaded metadata in the standby
catalogd may not be the latest. A backend flag should be provided
to control whether to reset metadata when the catalogd become active.
Adds the following startup flags for catalogd:
'catalogd_ha_reset_metadata_on_failover'. Default is false. If true,
reset all metadata when the catalogd becomes active.
Testing:
- Added a test case to start both catalogds with flag
'catalogd_ha_reset_metadata_on_failover' as true.
- Passed core tests
Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
---
M be/src/catalog/catalog-server.cc
M tests/custom_cluster/test_catalogd_ha.py
2 files changed, 52 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/20614/9
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 9
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
Patch Set 9: Code-Review+2
Zhi, thanks for your contribution.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 9
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 08 Nov 2023 02:08:56 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
ttttttz has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 3:
> Patch Set 2:
>
> (1 comment)
There are serveral causes why the metadata loaded in the standby catalogd legging behind, such as:
1、The catalogd enables load metadata in the background.
2、After multiple statestored restarts, the catalog's HA state changes as follows: active > stanbdy > active.
3、Newly created user-defined functions will not be recognized after failover.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 3
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 25 Oct 2023 09:04:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20614/3/tests/custom_cluster/test_catalogd_ha.py
File tests/custom_cluster/test_catalogd_ha.py:
http://gerrit.cloudera.org:8080/#/c/20614/3/tests/custom_cluster/test_catalogd_ha.py@426
PS3, Line 426:
flake8: W292 no newline at end of file
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 3
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 25 Oct 2023 08:33:50 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12513: Reset metadata when the CatalogD become active
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
ttttttz has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Reset metadata when the CatalogD become active
......................................................................
Patch Set 4:
> Patch Set 4:
>
> (1 comment)
I understand. Indeed, in order to pursue performance, the standby catalog is likely to enable background metadata loading, despite the risk of metadata expiration.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 4
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 01 Nov 2023 03:49:10 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
Patch Set 10:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9901/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 10
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 08 Nov 2023 02:09:28 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
Patch Set 9:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14365/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 9
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 08 Nov 2023 02:22:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20614 )
Change subject: IMPALA-12513: Allow to reset metadata when the CatalogD becomes active
......................................................................
Patch Set 10: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/20614
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b54f36f96e7901499105e51790d8e2300d7fea9
Gerrit-Change-Number: 20614
Gerrit-PatchSet: 10
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Wed, 08 Nov 2023 06:31:20 +0000
Gerrit-HasComments: No