You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Eric Badger (JIRA)" <ji...@apache.org> on 2016/09/27 21:53:20 UTC
[jira] [Updated] (HADOOP-12611)
TestZKSignerSecretProvider#testMultipleInit occasionally fail
[ https://issues.apache.org/jira/browse/HADOOP-12611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Badger updated HADOOP-12611:
---------------------------------
Attachment: HADOOP-12611.003.patch
Uploading a new patch that is built off of [~jojochuang]'s 002 patch. I made 3 main changes.
1. I explicitly call realRollSecret() so that we got rid of the race condition between rollSecret() being called and checking the secrets of each secretProvider.
2. I changed the verify() functions to use atLeast() instead of times(), because it is possible for rollSecret() to be called many times if the main code is slow and the scheduler thread is not.
3. I randomized the order in which realRollSecret() is called for each secretProvider.
In regards to change #3, I had an offline conversation with [~jlowe] that makes me wonder why we need this in the first place (though I was the one who suggested randomly selecting the order). Why are we relying on the test to give us randomness? If there are a finite amount of states like there are here, why don't we iterate over all of them instead of trying to randomly get all of the states over multiple runs? If we care that every permutation of the order of realRollSecret() works, then we should test every permutation. Randomly selecting the order means that it would take 6+ runs to go through every permutation. So if a change was made in the source that screwed up one of those permutations, we might not see it until a long ways down (and then it would be really hard to debug since we wouldn't know what order). [~rkanter], I think the better way to do this is to explicitly iterate through all of the permutations (or at least 2 permutations, 1 with each secretProvider "winning" the race).
> TestZKSignerSecretProvider#testMultipleInit occasionally fail
> -------------------------------------------------------------
>
> Key: HADOOP-12611
> URL: https://issues.apache.org/jira/browse/HADOOP-12611
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Wei-Chiu Chuang
> Assignee: Wei-Chiu Chuang
> Attachments: HADOOP-12611.001.patch, HADOOP-12611.002.patch, HADOOP-12611.003.patch
>
>
> https://builds.apache.org/job/Hadoop-Common-trunk/2053/testReport/junit/org.apache.hadoop.security.authentication.util/TestZKSignerSecretProvider/testMultipleInit/
> Error Message
> expected null, but was:<[B...@142bad79>
> Stacktrace
> java.lang.AssertionError: expected null, but was:<[B...@142bad79>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotNull(Assert.java:664)
> at org.junit.Assert.assertNull(Assert.java:646)
> at org.junit.Assert.assertNull(Assert.java:656)
> at org.apache.hadoop.security.authentication.util.TestZKSignerSecretProvider.testMultipleInit(TestZKSignerSecretProvider.java:149)
> I think the failure was introduced after HADOOP-12181
> This is likely where the root cause is:
> 2015-11-29 00:24:33,325 ERROR ZKSignerSecretProvider - An unexpected exception occurred while pulling data fromZooKeeper
> java.lang.IllegalStateException: instance must be started before calling this method
> at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> at org.apache.curator.framework.imps.CuratorFrameworkImpl.getData(CuratorFrameworkImpl.java:363)
> at org.apache.hadoop.security.authentication.util.ZKSignerSecretProvider.pullFromZK(ZKSignerSecretProvider.java:341)
> at org.apache.hadoop.security.authentication.util.ZKSignerSecretProvider.rollSecret(ZKSignerSecretProvider.java:264)
> at org.apache.hadoop.security.authentication.util.ZKSignerSecretProvider$$EnhancerByMockitoWithCGLIB$$575f06d8.CGLIB$rollSecret$2(<generated>)
> at org.apache.hadoop.security.authentication.util.ZKSignerSecretProvider$$EnhancerByMockitoWithCGLIB$$575f06d8$$FastClassByMockitoWithCGLIB$$6f94a716.invoke(<generated>)
> at org.mockito.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:216)
> at org.mockito.internal.creation.AbstractMockitoMethodProxy.invokeSuper(AbstractMockitoMethodProxy.java:10)
> at org.mockito.internal.invocation.realmethod.CGLIBProxyRealMethod.invoke(CGLIBProxyRealMethod.java:22)
> at org.mockito.internal.invocation.realmethod.FilteredCGLIBProxyRealMethod.invoke(FilteredCGLIBProxyRealMethod.java:27)
> at org.mockito.internal.invocation.Invocation.callRealMethod(Invocation.java:211)
> at org.mockito.internal.stubbing.answers.CallsRealMethods.answer(CallsRealMethods.java:36)
> at org.mockito.internal.MockHandler.handle(MockHandler.java:99)
> at org.mockito.internal.creation.MethodInterceptorFilter.intercept(MethodInterceptorFilter.java:47)
> at org.apache.hadoop.security.authentication.util.ZKSignerSecretProvider$$EnhancerByMockitoWithCGLIB$$575f06d8.rollSecret(<generated>)
> at org.apache.hadoop.security.authentication.util.RolloverSignerSecretProvider$1.run(RolloverSignerSecretProvider.java:97)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org