[Impala-ASF-CR] WIP IMPALA-12155: Support CatalogD HA

["Wenzhe Zhou (Code Review)" <ge...@cloudera.org>]

Wenzhe Zhou has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/19914 )

Change subject: WIP IMPALA-12155: Support CatalogD HA
......................................................................

WIP IMPALA-12155: Support CatalogD HA

To support catalog HA, we allow two catalog instances in an Active-
Passive HA pair to be added to an Impala cluster.
We add the preemptive behavior for catalogd. When enabled, the
preemptive behavior allows the catalogd with the higher priority to
become active and the paired catalogd becomes standby. The active
catalogd acts as the source of metadata and provides catalog service
for the Impala cluster.

To enable catalog HA for a cluster, two catalogds in the HA pair and
statestore must bestarted with starting flag "enable_catalog_ha".

The catalogd in an Active-Passive HA pair can be assigned an instance
priority value to indicate a preference for which catalogd should assume
the active role. The registration ID which is assigned by statestore can
be used as instance priority value. The lower numerical value in
registration ID corresponds to a higher priority. The catalogd with the
higher priority is designated as active, the other catalogd is
designated as standby. Only the active catalogd propagates the
IMPALA_CATALOG_TOPIC to the cluster. This guarantees only one writer for
the IMPALA_CATALOG_TOPIC in a Impala cluster.

statestore which is the registration center of an Impala cluster assigns
the roles for the catalogd in the HA pair after both catalogd register
to statestore. When statestore detects the active catalogd is not
healthy, it fails over catalog service to standby catalogd. When
failover occurs, statestore sends notifications with the address of
active catalogd to all coordinators and catalogd in the cluster. The
events are logged in the statestore and catalogd logs. When the catalogd
with the higher priority recovers from a failure, statestore does not
resume it as active to avoid flip-flop between the two catalogd.

To make a specific catalogd in the HA pair as active instance, the
catalogd must be started with starting flag "force_catalog_active" so
that the catalogd will be assigned with active role when it registers
to statestore. This allows administrator to manually perform catalog
service failover.

Testing:
 - Passed the core tests.
 - TODO unit-test

Change-Id: I68ce7e57014e2a01133aede7853a212d90688ddd
---
M be/src/catalog/catalog-server.cc
M be/src/catalog/catalog-server.h
M be/src/common/global-flags.cc
M be/src/runtime/exec-env.cc
M be/src/statestore/statestore-subscriber-catalog.h
M be/src/statestore/statestore-subscriber.cc
M be/src/statestore/statestore-subscriber.h
M be/src/statestore/statestore.cc
M be/src/statestore/statestore.h
M common/thrift/StatestoreService.thrift
10 files changed, 404 insertions(+), 84 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/19914/4
-- 
To view, visit http://gerrit.cloudera.org:8080/19914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I68ce7e57014e2a01133aede7853a212d90688ddd
Gerrit-Change-Number: 19914
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>