[Impala-ASF-CR] IMPALA-4856, IMPALA-4872: Include KRPC services in plan fragment's destinations

["Michael Ho (Code Review)" <ge...@cloudera.org>]

Hello Sailesh Mukil,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/7760

to look at the new patch set (#5).

Change subject: IMPALA-4856, IMPALA-4872: Include KRPC services in plan fragment's destinations
......................................................................

IMPALA-4856, IMPALA-4872: Include KRPC services in plan fragment's destinations

This change allows Impala to publish the IP address and port
information of KRPC services if it's enabled via the flag
use_krpc. The information is included in a new field in the
backend descriptor published as statestore updates. Scheduler
will also include this information in the destinations of plan
fragments. Also updated the mini-cluster startup script to assign
KRPC ports to Impalad instances.

This patch also takes into account of a problem found in
IMPALA-5795. In particular, the backend descriptor of the
coordinator may not be found in the backend map in the
scheduler if coordinator is not an executor (i.e. dedicated
coordinator). The fix to also check against the local backend
descriptor.

This patch is partially based on an abandoned patch by Henry Robinson.

Testing done: ran core tests with a patch which ignores the use_krpc
flag to exercise the code in scheduler.

Change-Id: I8707bfb5028bbe81d2a042fcf3e6e19f4b719a72
---
M be/src/common/global-flags.cc
M be/src/runtime/exec-env.cc
M be/src/runtime/exec-env.h
M be/src/scheduling/backend-config.cc
M be/src/scheduling/backend-config.h
M be/src/scheduling/scheduler-test-util.cc
M be/src/scheduling/scheduler.cc
M be/src/scheduling/scheduler.h
M be/src/service/impala-server.cc
M be/src/testutil/in-process-servers.cc
M be/src/testutil/in-process-servers.h
M be/src/util/network-util.cc
M be/src/util/network-util.h
M bin/start-impala-cluster.py
M common/thrift/ImpalaInternalService.thrift
M common/thrift/StatestoreService.thrift
16 files changed, 158 insertions(+), 60 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/7760/5
-- 
To view, visit http://gerrit.cloudera.org:8080/7760
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8707bfb5028bbe81d2a042fcf3e6e19f4b719a72
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>