You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "wanzhai (JIRA)" <ji...@apache.org> on 2018/08/16 11:59:00 UTC
[jira] [Commented] (HADOOP-15519) KMS fails to read the existing
key metadata after upgrading to JDK 1.8u171
[ https://issues.apache.org/jira/browse/HADOOP-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582408#comment-16582408 ]
wanzhai commented on HADOOP-15519:
----------------------------------
I also encountered this error.But my hadoop version is 2.6.5
When I executed "hadoop key list -metadata",I got this:
{code:java}
Cannot list keys for KeyProvider: KMSClientProvider[http://IP:PORT/kms/v1/]: Can't recover key for key1 from keystore file:/root/kms.keystore
java.io.IOException: Can't recover key for key1 from keystore file:/root/kms.keystore
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:482)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:441)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.getKeysMetadata(KMSClientProvider.java:584)
at org.apache.hadoop.crypto.key.KeyShell$ListCommand.execute(KeyShell.java:289)
at org.apache.hadoop.crypto.key.KeyShell.run(KeyShell.java:79)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.crypto.key.KeyShell.main(KeyShell.java:513){code}
kms.log:
{code:java}
2018-08-15 03:03:42,889 WARN AuthenticationFilter - Authentication exception: Anonymous requests are disallowed
org.apache.hadoop.security.authentication.client.AuthenticationException: Anonymous requests are disallowed
at org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:183)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:347)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:509)
at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:748){code}
I replaced jdk8u171 and the error is gone.
I don't know if the error I encountered is related to this issue.
> KMS fails to read the existing key metadata after upgrading to JDK 1.8u171
> ---------------------------------------------------------------------------
>
> Key: HADOOP-15519
> URL: https://issues.apache.org/jira/browse/HADOOP-15519
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
> Affects Versions: 2.7.3
> Reporter: Vipin Rathor
> Priority: Critical
>
> Steps to reproduce are:
> a. Setup a KMS with any OpenJDK 1.8 before u171 and create few KMS keys.
> b. Update KMS to run with OpenJDK 1.8u171 JDK and keys can't be read anymore, as can be seen below
> {code:java}
> hadoop key list -metadata
> <keyname> : null
> {code}
> c. Going back to earlier JDK version fixes the issue.
>
> There are no direct error / stacktrace in kms.log when it is not able to read the key metadata. Only Java serialization INFO messages are printed, followed by this one empty line in log which just says:
> {code:java}
> ERROR RangerKeyStore -
> {code}
> In some cases, kms.log can also have these lines:
> {code:java}
> 2018-05-18 10:40:46,438 DEBUG RangerKmsAuthorizer - <== RangerKmsAuthorizer.assertAccess(null, rangerkms/node1.host.com@ENV.COM (auth:KERBEROS), GET_METADATA)
> 2018-05-18 10:40:46,598 INFO serialization - ObjectInputFilter REJECTED: class org.apache.hadoop.crypto.key.RangerKeyStoreProvider$KeyMetadata, array length: -1, nRefs: 1, depth: 1, bytes: 147, ex: n/a
> 2018-05-18 10:40:46,598 ERROR RangerKeyStore -
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org