[jira] [Updated] (CASSANDRA-6768) Refresh permissions cache in ClientState periodically to avoid cache miss stampede effect

["Michał Michalski (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CASSANDRA-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michał Michalski updated CASSANDRA-6768:
----------------------------------------

    Description: 
h2. Background

We want to password-protect Cassandra by using the built-in PasswordAuthenticator and PasswordAuthorizer. In general we are happy with this solution, but after reviewing the code we're a bit afraid of default  permissionsCache behaviour in org.apache.cassandra.service.ClientState.

h2. Problem

>From what I understand, at the moment cache expires every N seconds (2 by default) and it gets repopulated when permissionsCache.get() is being  called. However, as we're talking about at least a few hundreds requests to Cassandra per second, we're afraid of the "stampede" effect once the cache expires and a number of queries will "trigger" its reload simultaneously during the short period of time when it will be empty.

h2. Proposed Solution

Therefore, instead of the current solution, we'd prefer this cache to be reloaded "in background" every N seconds, so it's only a single request every N seconds, rather than tens (hundreds?) of them just after the cache expires during the period when it's empty.

In other words, we're thinking about two solutions (updated after comments from [~jjordan] and [~iamaleksey]):

h3. Make the ClientState's permissionsCache configurable. 

Let's define additional configurable variable called refreshPeriod and make it 0 by default (0 means no refresh - nothing changes in current code). If it's > 0, add the refreshAfterWrite to the existing code.

This way we keep the defaults "safe", but add possibility to "tune" the cache when someone needs it. Any cons?

h3. Make Authorizer responsible for its own cache

It changes the IAuthorizer

  was:
h3. Background

We want to password-protect Cassandra by using the built-in PasswordAuthenticator and PasswordAuthorizer. In general we are happy with this solution, but after reviewing the code we're a bit afraid of default  permissionsCache behaviour in org.apache.cassandra.service.ClientState.

h3. Problem

>From what I understand, at the moment cache expires every N seconds (2 by default) and it gets repopulated when permissionsCache.get() is being  called. However, as we're talking about at least a few hundreds requests to Cassandra per second, we're afraid of the "stampede" effect once the cache expires and a number of queries will "trigger" its reload simultaneously during the short period of time when it will be empty.

h3. Proposed Solution

Therefore, instead of the current solution, we'd prefer this cache to be reloaded "in background" every N seconds, so it's only a single request every N seconds, rather than tens (hundreds?) of them just after the cache expires during the period when it's empty.

In other words, we're thinking about replacing this:

{code}expireAfterWrite(validityPeriod, TimeUnit.MILLISECONDS){code}

with:

{code}refreshAfterWrite(refreshPeriod, TimeUnit.MILLISECONDS){code}

Default refreshPeriod could be the same as the validityPeriod, for example.

Are there any reasons that make this idea a bad one ("you misunderstood Guava's Cache" counts too!)?

[~iamaleksey], I've let myself to assign this issue directly to you, as you're the author of current solution.


> Refresh permissions cache in ClientState periodically to avoid cache miss stampede effect
> -----------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-6768
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6768
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Michał Michalski
>            Assignee: Aleksey Yeschenko
>              Labels: authentication
>
> h2. Background
> We want to password-protect Cassandra by using the built-in PasswordAuthenticator and PasswordAuthorizer. In general we are happy with this solution, but after reviewing the code we're a bit afraid of default  permissionsCache behaviour in org.apache.cassandra.service.ClientState.
> h2. Problem
> From what I understand, at the moment cache expires every N seconds (2 by default) and it gets repopulated when permissionsCache.get() is being  called. However, as we're talking about at least a few hundreds requests to Cassandra per second, we're afraid of the "stampede" effect once the cache expires and a number of queries will "trigger" its reload simultaneously during the short period of time when it will be empty.
> h2. Proposed Solution
> Therefore, instead of the current solution, we'd prefer this cache to be reloaded "in background" every N seconds, so it's only a single request every N seconds, rather than tens (hundreds?) of them just after the cache expires during the period when it's empty.
> In other words, we're thinking about two solutions (updated after comments from [~jjordan] and [~iamaleksey]):
> h3. Make the ClientState's permissionsCache configurable. 
> Let's define additional configurable variable called refreshPeriod and make it 0 by default (0 means no refresh - nothing changes in current code). If it's > 0, add the refreshAfterWrite to the existing code.
> This way we keep the defaults "safe", but add possibility to "tune" the cache when someone needs it. Any cons?
> h3. Make Authorizer responsible for its own cache
> It changes the IAuthorizer



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)