You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2021/08/05 03:28:10 UTC
[Impala-ASF-CR] IMPALA-5476: Fix Catalogd restart bring about metadata is out of sync
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17645 )
Change subject: IMPALA-5476: Fix Catalogd restart bring about metadata is out of sync
......................................................................
Patch Set 14:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17645/10/tests/custom_cluster/test_restart_services.py
File tests/custom_cluster/test_restart_services.py:
http://gerrit.cloudera.org:8080/#/c/17645/10/tests/custom_cluster/test_restart_services.py@208
PS10, Line 208: LOG.info(str(e))
: if i == 5:
: self.cluster.catalogd.restart()
:
> I tried many times, but the WARNING log was not printed. It may be because
statestored will send topic update request to catalogd once it's registered. I ran the test and realized that we restart catalogd too quickly, i.e. catalogd hasn't been ready after restart but we restart it again. So impalad is actually talking to the second restarted catalogd.
The problem is restart() returns too early. Connected to statestored doesn't mean catalogd is ready: https://github.com/apache/impala/blob/ca17e307ab3abb2c95c27b3ba749adf6bf16efc7/tests/common/impala_cluster.py#L569-L571
We probably need to add a metric like "catalog-server.ready" to indicate the start is finish.
--
To view, visit http://gerrit.cloudera.org:8080/17645
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9fe25f5a2a42fb432e306ef08ae35750c8f3c50c
Gerrit-Change-Number: 17645
Gerrit-PatchSet: 14
Gerrit-Owner: liuyao <li...@sensorsdata.cn>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 05 Aug 2021 03:28:10 +0000
Gerrit-HasComments: Yes