[jira] [Updated] (HIVE-13075) Metastore shuts down when no delegation token is found in ZooKeeper

["Piotr Wikieł (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Piotr Wikieł updated HIVE-13075:
--------------------------------
    Status: Patch Available  (was: Open)

> Metastore shuts down when no delegation token is found in ZooKeeper
> -------------------------------------------------------------------
>
>                 Key: HIVE-13075
>                 URL: https://issues.apache.org/jira/browse/HIVE-13075
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Piotr Wikieł
>            Priority: Minor
>
> {{ZooKeeperTokenStore}} looks [as follows|https://github.com/apache/hive/blob/branch-1.2/shims/common/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java#L397]:
> {code:java}
> @Override
> public DelegationTokenInformation getToken(DelegationTokenIdentifier tokenIdentifier) {
>   byte[] tokenBytes = zkGetData(getTokenPath(tokenIdentifier));
>   try {
>     return HiveDelegationTokenSupport.decodeDelegationTokenInformation(tokenBytes);
>   } catch (Exception ex) {
>     throw new TokenStoreException("Failed to decode token", ex);
>   }
> }
> {code}
> which is slightly different from [DBTokenStore implementation|https://github.com/apache/hive/blob/branch-1.2/shims/common/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java#L85] that is protected against {{tokenBytes==null}} because nullable {{tokenBytes}} causes NPE to be thrown in [HiveDelegationTokenSupport#decodeDelegationTokenInformation|https://github.com/apache/hive/blob/branch-1.2/shims/common/src/main/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java#L51]
> Furthermore, NPE thrown here causes [TokenStoreDelegationTokenSecretManager.ExpiredTokenRemover|https://github.com/apache/hive/blob/branch-1.2/shims/common/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java#L333] to catch it and exits MetaStore.
> {{null}} from {{[zkGetData()|https://github.com/apache/hive/blob/branch-1.2/shims/common/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java#L284]}} is possible during ZooKeeper failure or (and that was our case) when another metastore instance removes tokens during {{ExpiredTokenRemover}} run. There were two solutions of this problem:
>  * distributed lock in ZooKeeper acquired during one metastore instance's ExpiredTokenRemover run,
>  * simple null check
> I think null check is sufficient if it is in {{DBTokenStore}}.
> Patch will be attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)