[kudu-CR] [ranger] Add refreshing Ranger policies

["Attila Bukor (Code Review)" <ge...@cloudera.org>]

Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Grant Henke, Greg Solovyev, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16160

to look at the new patch set (#5).

Change subject: [ranger] Add refreshing Ranger policies
......................................................................

[ranger] Add refreshing Ranger policies

Authorization tests depend on frequent polling of policies from the
Ranger server to make sure newly created policies are cached in the
authorizer plugin. We've seen some flaky tests due to this recently, so
a better way to handle this would be to manually refresh the policies in
the authorizer plugin.

When the Ranger integration was initially implemented we decided to not
use the existing ResetAuthzCache methods in master as it behaves in a
different way than it did with Sentry. Sentry's cache was within Kudu
and could be fully reset. Unfortunately Ranger's cache lives in the
Ranger plugin within the Ranger subprocess which has a void
refreshPolicies() method that doesn't throw any checked exceptions. This
means there's no way to guarantee the cache is cleared and the only
failure we can know about is if the request times out or other failures
between the master and the subprocess, but not between the subprocess
and the Ranger server. This means that even if Ranger itself is down, we
can still get an OK response when calling this method and there's no way
to guarantee clearing the cache.

When Sentry integration was removed from Kudu, this ResetAuthzCache()
method was also removed along with the tooling, as we didn't have any
authz providers that would support resetting the cache.

Now we need a way to do this to deflake the tests, and as there is no
longer a discrepancy between the behavior of different authz providers
and the best effort basis of refreshing policies should be enough, this
method is readded by this commit and the tests are changed to use it
instead of the timer-based policy refreshes.

Change-Id: Ie90b1c23b92060e081ab514fbae417b4f31c2b66
---
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/ranger/RangerProtocolHandler.java
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/ranger/authorization/RangerKuduAuthorizer.java
M src/kudu/integration-tests/master_authz-itest.cc
M src/kudu/integration-tests/ts_authz-itest.cc
M src/kudu/master/authz_provider.h
M src/kudu/master/default_authz_provider.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/master/master_service.h
M src/kudu/master/ranger_authz_provider.cc
M src/kudu/master/ranger_authz_provider.h
M src/kudu/ranger/mini_ranger.h
M src/kudu/ranger/ranger.proto
M src/kudu/ranger/ranger_client.cc
M src/kudu/ranger/ranger_client.h
M src/kudu/tools/kudu-admin-test.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_master.cc
18 files changed, 418 insertions(+), 59 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/60/16160/5
-- 
To view, visit http://gerrit.cloudera.org:8080/16160
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie90b1c23b92060e081ab514fbae417b4f31c2b66
Gerrit-Change-Number: 16160
Gerrit-PatchSet: 5
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Greg Solovyev <gs...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)