[jira] [Updated] (YARN-5092) TestRMDelegationTokens fails intermittently

["Jason Lowe (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Lowe updated YARN-5092:
-----------------------------
    Attachment: YARN-5092.001.patch

There are two unrelated problems here.

The class cast exception is caused when the two tests run in a particular order.  The setup method configures the fair scheduler, but one of the tests ignores the setup conf and uses a default one.  Therefore one of the tests was running with the fair scheduler and the other with the capacity scheduler.  If the capacity scheduler test runs first, the QueueMetrics will be initialized with a CSQueueMetrics.  Later when the fair scheduler tries to wield the already existing queue metric it fails to cast it because it's the wrong type.  I fixed this by having both tests use the same base config, and I also cleared out the queue metrics in-between tests just for good measure.

The rolling master key failure is triggered because there's a small benign window of time in the AbstractDelegationTokenSecretManager where a master key can have its expiry updated but not in the state store yet.  The test occasionally catches this window, and because DelegationKey leverages the expiration date in its hashcode and equals methods, the contains method on the set of delegation keys fails to find it.  As [~daryn] pointed out to me offline, arguably DelegationKey should not be using the expiration date for hashcode/equals.  However there's tons of stuff using and deriving from DelegationKey, so it's somewhat of a risky change to remove it.  Instead I updated the unit test to check for a matching key ID in the state store rather than the contains method.

If we don't update the DelegationKey hashcode/equals then there should be a followup JIRA to fix the MemoryRMStateStore, as it currently leaks delegation keys as they roll.  The key's expiration date gets updated, and the state store cannot find them in the set of keys to remove them.  A simple fix is to store them by key ID like the other RM state store implementations and the secret manager itself already do.

> TestRMDelegationTokens fails intermittently 
> --------------------------------------------
>
>                 Key: YARN-5092
>                 URL: https://issues.apache.org/jira/browse/YARN-5092
>             Project: Hadoop YARN
>          Issue Type: Test
>          Components: test
>            Reporter: Rohith Sharma K S
>         Attachments: YARN-5092.001.patch
>
>
> In build [link|https://builds.apache.org/job/PreCommit-YARN-Build/11476/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt] , TestRMDelegationTokens fails for 2 test cases
> # TestRMDelegationTokens.testRMDTMasterKeyStateOnRollingMasterKey
> # TestRMDelegationTokens.testRemoveExpiredMasterKeyInRMStateStore



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org