[kudu-CR] [sentry] add mini Sentry to the external mini cluster

["Hao Hao (Code Review)" <ge...@cloudera.org>]

Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11956

to look at the new patch set (#7).

Change subject: [sentry] add mini Sentry to the external mini cluster
......................................................................

[sentry] add mini Sentry to the external mini cluster

This commit enables external mini cluster to be able to start with both
mini sentry and mini hms. A major challenge is the circular dependency
between the HMS and Sentry in terms of configuring each with the other's
address, including port which is currently discovered by polling lsof.
To work around it, one approach would be to:

  1. Start the Sentry service. Find out which port it's on. At this
     stage, the Sentry service has no knowledge of any HMS service.
  2. Start the HMS, configured to talk to Sentry's port. Find out which
     port it's on.
  3. Restart the Sentry service on the same port, reconfigured to talk
     to the HMS's port.

However, there could be a race condition where another program binds to
the same port during the restart in step 3. One option is to use
SO_REUSEPORT socket option, which permits multiple sockets to be bound
to an identical socket address. Although both Sentry and HMS are written
in Java, only JDK9 (and above) supports this socket option[1]. Methods
such as Java reflection can be used to invoke private methods to set up
this option, but they are hacky.

This patch extends the UNIQUE_LOOPBACK usage in mini cluster, so that
external servers such as Sentry can pick up a unique bind address,
knowing that in doing so there'll never be any danger of a port
collision.

[1]: https://docs.oracle.com/javase/9/docs/api/java/net/StandardSocketOptions.html#SO_REUSEPORT

Change-Id: I7f02e6085bd239570d10ec629f48856d37ed6e59
---
M src/kudu/hms/mini_hms.cc
M src/kudu/hms/mini_hms.h
M src/kudu/integration-tests/CMakeLists.txt
A src/kudu/integration-tests/master_sentry-itest.cc
M src/kudu/mini-cluster/CMakeLists.txt
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/mini-cluster/external_mini_cluster.h
M src/kudu/mini-cluster/mini_cluster.cc
M src/kudu/mini-cluster/mini_cluster.h
M src/kudu/security/test/mini_kdc.cc
M src/kudu/sentry/mini_sentry.cc
M src/kudu/sentry/mini_sentry.h
M src/kudu/sentry/sentry-test-base.h
M src/kudu/tools/tool.proto
M src/kudu/util/test_util.cc
M src/kudu/util/test_util.h
16 files changed, 537 insertions(+), 69 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/11956/7
-- 
To view, visit http://gerrit.cloudera.org:8080/11956
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7f02e6085bd239570d10ec629f48856d37ed6e59
Gerrit-Change-Number: 11956
Gerrit-PatchSet: 7
Gerrit-Owner: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)