You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Xiao Chen (JIRA)" <ji...@apache.org> on 2016/01/08 23:32:39 UTC
[jira] [Commented] (HADOOP-12699) TestKMS#testKMSProvider
intermittently fails during 'test rollover draining'
[ https://issues.apache.org/jira/browse/HADOOP-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090080#comment-15090080 ]
Xiao Chen commented on HADOOP-12699:
------------------------------------
A sample failure is pasted here, but I don't think there's much information.
Error Message
{noformat}
Values should be different. Actual: k6@0
{noformat}
Stacktrace
{noformat}
java.lang.AssertionError: Values should be different. Actual: k6@0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failEquals(Assert.java:185)
at org.junit.Assert.assertNotEquals(Assert.java:161)
at org.junit.Assert.assertNotEquals(Assert.java:175)
at org.apache.hadoop.crypto.key.kms.server.TestKMS$2.call(TestKMS.java:649)
at org.apache.hadoop.crypto.key.kms.server.TestKMS$2.call(TestKMS.java:413)
at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:130)
at org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:112)
at org.apache.hadoop.crypto.key.kms.server.TestKMS.testKMSProvider(TestKMS.java:413)
{noformat}
Standard Output
{noformat}
Test KMS running at: http://localhost:50665/kms
2016-01-05 15:02:01,885 ERROR NIOServerCnxnFactory - Thread Thread[org.apache.hadoop.crypto.key.kms.ValueQueue_thread,5,main] died
java.lang.RuntimeException: java.io.IOException: java.io.IOException: Exeption while contacting value generator
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:334)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: java.io.IOException: Exeption while contacting value generator
at sun.reflect.GeneratedConstructorAccessor68.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:546)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:504)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.access$200(KMSClientProvider.java:84)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider$EncryptedQueueRefiller.fillQueueForKey(KMSClientProvider.java:135)
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:330)
... 3 more
2016-01-05 15:02:01,886 ERROR NIOServerCnxnFactory - Thread Thread[org.apache.hadoop.crypto.key.kms.ValueQueue_thread,5,main] died
java.lang.RuntimeException: java.lang.NullPointerException: No KeyVersion exists for key 'k1'
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:334)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException: No KeyVersion exists for key 'k1'
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:231)
at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension$DefaultCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:252)
at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:371)
at org.apache.hadoop.crypto.key.kms.server.EagerKeyGeneratorKeyProviderCryptoExtension$CryptoExtension$EncryptedQueueRefiller.fillQueueForKey(EagerKeyGeneratorKeyProviderCryptoExtension.java:77)
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:330)
... 3 more
2016-01-05 15:02:01,886 ERROR NIOServerCnxnFactory - Thread Thread[org.apache.hadoop.crypto.key.kms.ValueQueue_thread,5,main] died
java.lang.RuntimeException: java.lang.NullPointerException: No KeyVersion exists for key 'k1'
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:334)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException: No KeyVersion exists for key 'k1'
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:231)
at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension$DefaultCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:252)
at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:371)
at org.apache.hadoop.crypto.key.kms.server.EagerKeyGeneratorKeyProviderCryptoExtension$CryptoExtension$EncryptedQueueRefiller.fillQueueForKey(EagerKeyGeneratorKeyProviderCryptoExtension.java:77)
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:330)
... 3 more
2016-01-05 15:02:01,885 ERROR NIOServerCnxnFactory - Thread Thread[org.apache.hadoop.crypto.key.kms.ValueQueue_thread,5,main] died
java.lang.RuntimeException: java.lang.NullPointerException: No KeyVersion exists for key 'k1'
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:334)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException: No KeyVersion exists for key 'k1'
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:231)
at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension$DefaultCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:252)
at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:371)
at org.apache.hadoop.crypto.key.kms.server.EagerKeyGeneratorKeyProviderCryptoExtension$CryptoExtension$EncryptedQueueRefiller.fillQueueForKey(EagerKeyGeneratorKeyProviderCryptoExtension.java:77)
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:330)
... 3 more
2016-01-05 15:02:01,885 ERROR NIOServerCnxnFactory - Thread Thread[org.apache.hadoop.crypto.key.kms.ValueQueue_thread,5,main] died
java.lang.RuntimeException: java.io.IOException: java.io.IOException: Exeption while contacting value generator
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:334)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: java.io.IOException: Exeption while contacting value generator
at sun.reflect.GeneratedConstructorAccessor68.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:546)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:504)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.access$200(KMSClientProvider.java:84)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider$EncryptedQueueRefiller.fillQueueForKey(KMSClientProvider.java:135)
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:330)
... 3 more
2016-01-05 15:02:01,912 WARN LoadBalancingKMSClientProvider - KMS provider at [http://localhost:50665/kms/v1/] threw an IOException [java.util.concurrent.ExecutionException: java.io.IOException: org.apache.hadoop.security.authorize.AuthorizationException: User [jenkins] is not authorized to perform [GENERATE_EEK] on key with ACL name [k2]!!]!!
2016-01-05 15:02:01,912 WARN LoadBalancingKMSClientProvider - Aborting since the Request has failed with all KMS providers in the group. !!
2016-01-05 15:02:02,193 ERROR NIOServerCnxnFactory - Thread Thread[org.apache.hadoop.crypto.key.kms.ValueQueue_thread,5,main] died
java.lang.RuntimeException: java.net.ConnectException: Connection refused
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:334)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:763)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1324)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:519)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:504)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.access$200(KMSClientProvider.java:84)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider$EncryptedQueueRefiller.fillQueueForKey(KMSClientProvider.java:135)
at org.apache.hadoop.crypto.key.kms.ValueQueue$2.run(ValueQueue.java:330)
... 3 more
{noformat}
> TestKMS#testKMSProvider intermittently fails during 'test rollover draining'
> ----------------------------------------------------------------------------
>
> Key: HADOOP-12699
> URL: https://issues.apache.org/jira/browse/HADOOP-12699
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Xiao Chen
> Assignee: Xiao Chen
>
> I've seen several failures of testKMSProvider, all failed in the following snippet:
> {code}
> // test rollover draining
> KeyProviderCryptoExtension kpce = KeyProviderCryptoExtension.
> createKeyProviderCryptoExtension(kp);
> .....
> EncryptedKeyVersion ekv1 = kpce.generateEncryptedKey("k6");
> kpce.rollNewVersion("k6");
> EncryptedKeyVersion ekv2 = kpce.generateEncryptedKey("k6");
> Assert.assertNotEquals(ekv1.getEncryptionKeyVersionName(),
> ekv2.getEncryptionKeyVersionName());
> {code}
> with error message
> {quote}Values should be different. Actual: k6@0{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)