You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2019/03/04 19:02:59 UTC

[kudu-CR] WIP [master] introduced SentryAuthzCache

Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12653


Change subject: WIP [master] introduced SentryAuthzCache
......................................................................

WIP [master] introduced SentryAuthzCache

This patch incorporate a TTL-based cache into the SentryAuthzProvider.
The cache stores responses received from the authz provider (Sentry in
this case).  It's possible to enable or disable using the cache upon
creation of SentryAuthzProvider instance: set the newly introduced
`--sentry_cache_capacity` command-line flag to 0 to disable the cache.

WIP:
  * need to add tests
  * pre-process raw TListSentryPrivilegesResponse into some sort
    of structure to speed up lookups of certain privilege on an
    authorizable
  * address the 'thundering herd' request pattern (maybe in the
    implementation of the TTL cache itself?)

Change-Id: If377cd4a7bae481ffc9a6950cb26862b66e9bab5
---
M src/kudu/master/CMakeLists.txt
A src/kudu/master/sentry_authz_cache.cc
A src/kudu/master/sentry_authz_cache.h
M src/kudu/master/sentry_authz_provider.cc
M src/kudu/master/sentry_authz_provider.h
5 files changed, 227 insertions(+), 12 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/12653/1
-- 
To view, visit http://gerrit.cloudera.org:8080/12653
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If377cd4a7bae481ffc9a6950cb26862b66e9bab5
Gerrit-Change-Number: 12653
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>

[kudu-CR] WIP [master] introduced SentryAuthzCache

Posted by "Greg Solovyev (Code Review)" <ge...@cloudera.org>.
Greg Solovyev has posted comments on this change. ( http://gerrit.cloudera.org:8080/12653 )

Change subject: WIP [master] introduced SentryAuthzCache
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_provider.cc
File src/kudu/master/sentry_authz_provider.cc:

http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_provider.cc@319
PS1, Line 319:   if (PREDICT_TRUE(cache_)) {
If I am reading this correctly, when cache is enabled and a privilege is not in the cache, Kudu will deny the action. When Sentry has more privileges than Kudu's cache capacity Kudu will incorrectly denying actions due to cache miss rather than due to configured privilege. Is this the behavior we want by design?



-- 
To view, visit http://gerrit.cloudera.org:8080/12653
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If377cd4a7bae481ffc9a6950cb26862b66e9bab5
Gerrit-Change-Number: 12653
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Greg Solovyev <gs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Mon, 18 Mar 2019 18:40:08 +0000
Gerrit-HasComments: Yes

[kudu-CR] WIP [master] introduced SentryAuthzCache

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12653 )

Change subject: WIP [master] introduced SentryAuthzCache
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_provider.cc
File src/kudu/master/sentry_authz_provider.cc:

http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_provider.cc@319
PS1, Line 319:   if (PREDICT_TRUE(cache_)) {
> If I am reading this correctly, when cache is enabled and a privilege is no
Nope, that's not the case.  When cache is enabled and the information is not in the cache, the required information will be fetched from the authz provider.

The entries in the cache are replaced when the cache is at capacity -- that's how caches are works usually.



-- 
To view, visit http://gerrit.cloudera.org:8080/12653
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If377cd4a7bae481ffc9a6950cb26862b66e9bab5
Gerrit-Change-Number: 12653
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Greg Solovyev <gs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 19 Mar 2019 00:04:23 +0000
Gerrit-HasComments: Yes

[kudu-CR] WIP [master] introduced SentryAuthzCache

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has abandoned this change. ( http://gerrit.cloudera.org:8080/12653 )

Change subject: WIP [master] introduced SentryAuthzCache
......................................................................


Abandoned

An alternative patch is coming: it's based on the FIFO cache from util/cache.{cc,h}
-- 
To view, visit http://gerrit.cloudera.org:8080/12653
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: If377cd4a7bae481ffc9a6950cb26862b66e9bab5
Gerrit-Change-Number: 12653
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] WIP [master] introduced SentryAuthzCache

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12653 )

Change subject: WIP [master] introduced SentryAuthzCache
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_provider.cc
File src/kudu/master/sentry_authz_provider.cc:

http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_provider.cc@319
PS1, Line 319:   if (PREDICT_TRUE(cache_)) {
> Nope, that's not the case.  When cache is enabled and the information is no
I meant when some piece of information is not in the cache, the request is made to the source to get that piece of information.  Then the cache takes care of accommodating the newly fetched information: if the cache is at capacity, some existing entries are to be evicted.  The policy for the eviction of entries differ (and there are many), but the principle of evicting existing entries to accommodate new ones is common for the caches, and that's how they usually work.



-- 
To view, visit http://gerrit.cloudera.org:8080/12653
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If377cd4a7bae481ffc9a6950cb26862b66e9bab5
Gerrit-Change-Number: 12653
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Greg Solovyev <gs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 19 Mar 2019 00:10:44 +0000
Gerrit-HasComments: Yes

[kudu-CR] WIP [master] introduced SentryAuthzCache

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12653 )

Change subject: WIP [master] introduced SentryAuthzCache
......................................................................


Patch Set 1:

(2 comments)

Rough, high-level pass since this is still a WIP.

http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_cache.h
File src/kudu/master/sentry_authz_cache.h:

http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_cache.h@26
PS1, Line 26: /*
            : #include <gflags/gflags.h>
            : #include <gflags/gflags_declare.h>
            : */
Remove these.


http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_cache.h@50
PS1, Line 50: SentryAuthzCache
The API this exposes isn't very cache-like (e.g. no Put), so it probably deserves a different name (though I haven't come up with a good one. Maybe SentryPrivilegeGetter or something?).

Alternatively, I wonder if this can be put directly in the SentryAuthzProvider? Unless you foresee it sprouting new functionality that makes encapsulating it useful (e.g. the batching up of requests to Sentry).



-- 
To view, visit http://gerrit.cloudera.org:8080/12653
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If377cd4a7bae481ffc9a6950cb26862b66e9bab5
Gerrit-Change-Number: 12653
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 05 Mar 2019 08:27:03 +0000
Gerrit-HasComments: Yes

[kudu-CR] WIP [master] introduced SentryAuthzCache

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12653 )

Change subject: WIP [master] introduced SentryAuthzCache
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_cache.h
File src/kudu/master/sentry_authz_cache.h:

http://gerrit.cloudera.org:8080/#/c/12653/1/src/kudu/master/sentry_authz_cache.h@50
PS1, Line 50: SentryAuthzCache
> The API this exposes isn't very cache-like (e.g. no Put), so it probably de
These are good suggestions.  I'll try to incorporate those into the alternative patch that uses newly introduced FIFO cache from util/cache.{cc,h}



-- 
To view, visit http://gerrit.cloudera.org:8080/12653
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If377cd4a7bae481ffc9a6950cb26862b66e9bab5
Gerrit-Change-Number: 12653
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Mon, 18 Mar 2019 18:11:02 +0000
Gerrit-HasComments: Yes