[Impala-ASF-CR] WIP IMPALA-7626: Throttle catalog partial RPC requests

["Bharath Vissapragada (Code Review)" <ge...@cloudera.org>]

Hello Tianyi Wang, Philip Zeyliger, Impala Public Jenkins, Vuk Ercegovac, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11561

to look at the new patch set (#3).

Change subject: WIP IMPALA-7626: Throttle catalog partial RPC requests
......................................................................

WIP IMPALA-7626: Throttle catalog partial RPC requests

With more coordinators running in local catalog mode, the expected RPC
traffic on the Catalog server is higher compared to the non-local-catalog
mode. Each such RPC is handled in its own thread and consumes some
non-trivial CPU for serializing and deserializing the metadata.

With this change, the maximum number of performing the actual work are
capped to a certain limit at any point and the remaining requests (if any)
are blocked until the current requests are serviced. Adds the following
parameters for controlling this behavior.

--catalog_partial_rpc_max_parallel_runs
--catalog_partial_rpc_queue_timeout_s

--catalog_partial_rpc_queue_timeout_s controls the timeout for queued
requests.

Added some basic supportability to examine the queue length via metrics.

Marking this as WIP since this probably needs more testing. Suggestions welcome.

Change-Id: I11f77a16cfa38ada42d8b7c859850198ea7dd142
---
M be/src/catalog/catalog-server.cc
M be/src/catalog/catalog-server.h
M be/src/catalog/catalog.cc
M be/src/catalog/catalog.h
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M common/thrift/JniCatalog.thrift
M common/thrift/metrics.json
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/JniCatalog.java
11 files changed, 135 insertions(+), 8 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/11561/3
-- 
To view, visit http://gerrit.cloudera.org:8080/11561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I11f77a16cfa38ada42d8b7c859850198ea7dd142
Gerrit-Change-Number: 11561
Gerrit-PatchSet: 3
Gerrit-Owner: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>