[jira] [Commented] (HDFS-16518) Cached KeyProvider in KeyProviderCache should be closed with ShutdownHookManager

["Owen O'Malley (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512012#comment-17512012 ] 

Owen O'Malley commented on HDFS-16518:
--------------------------------------

I don't understand why this is required. Obviously at jvm shutdown the cache will be discarded. The order of shutdown hooks isn't deterministic, so using this isn't a fix against other shutdown hooks using the cache.

Is there some other call to KeyProvider.close() that this should replace?

> Cached KeyProvider in KeyProviderCache should be closed with ShutdownHookManager
> --------------------------------------------------------------------------------
>
>                 Key: HDFS-16518
>                 URL: https://issues.apache.org/jira/browse/HDFS-16518
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs
>    Affects Versions: 2.10.0
>            Reporter: Lei Yang
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> We need to make sure the underlying KeyProvider used by multiple DFSClient instances is closed at one shot during jvm shutdown. Within the shutdownhook, we invalidate the cache and make sure they are all closed. The  cache has a removeListener hook which is called when cache entry is invalidated. 
> {code:java}
> Class KeyProviderCache
> ...
>  public KeyProviderCache(long expiryMs) {
>   cache = CacheBuilder.newBuilder()
>     .expireAfterAccess(expiryMs, TimeUnit.MILLISECONDS)
>     .removalListener(new RemovalListener<URI, KeyProvider>() {
>       @Override
>       public void onRemoval(
>           @Nonnull RemovalNotification<URI, KeyProvider> notification) {
>         try {
>           assert notification.getValue() != null;
>           notification.getValue().close();
>         } catch (Throwable e) {
>           LOG.error(
>               "Error closing KeyProvider with uri ["
>                   + notification.getKey() + "]", e);
>         }
>       }
>     })
>     .build(); 
> }{code}
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org