You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Zsolt Venczel (JIRA)" <ji...@apache.org> on 2018/07/26 18:50:00 UTC
[jira] [Commented] (HDFS-13697) DFSClient should instantiate and
cache KMSClientProvider at creation time for consistent UGI handling
[ https://issues.apache.org/jira/browse/HDFS-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558750#comment-16558750 ]
Zsolt Venczel commented on HDFS-13697:
--------------------------------------
The latest patch contains:
* Revert of HDFS-7718 and HADOOP-13749.
* DFSClient creates and caches the KeyProvider at construction time.
* KMSClientProvider holds on to the UGI at creation time and also supports HADOOP-10698 efforts.
* HADOOP-11368 resolves the sslfactory truststore reloader thread leak
This patch does not cover the shared, periodic method for checking the truststore files.
If you agree it could be covered in a separate jira.
> DFSClient should instantiate and cache KMSClientProvider at creation time for consistent UGI handling
> -----------------------------------------------------------------------------------------------------
>
> Key: HDFS-13697
> URL: https://issues.apache.org/jira/browse/HDFS-13697
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Zsolt Venczel
> Assignee: Zsolt Venczel
> Priority: Major
> Attachments: HDFS-13697.01.patch, HDFS-13697.02.patch, HDFS-13697.03.patch, HDFS-13697.04.patch
>
>
> While calling KeyProviderCryptoExtension decryptEncryptedKey the call stack might not have doAs privileged execution call (in the DFSClient for example). This results in loosing the proxy user from UGI as UGI.getCurrentUser finds no AccessControllerContext and does a re-login for the login user only.
> This can cause the following for example: if we have set up the oozie user to be entitled to perform actions on behalf of example_user but oozie is forbidden to decrypt any EDEK (for security reasons), due to the above issue, example_user entitlements are lost from UGI and the following error is reported:
> {code}
> [0]
> SERVER[xxx] USER[example_user] GROUP[-] TOKEN[] APP[Test_EAR] JOB[0020905-180313191552532-oozie-oozi-W] ACTION[0020905-180313191552532-oozie-oozi-W@polling_dir_path] Error starting action [polling_dir_path]. ErrorType [ERROR], ErrorCode [FS014], Message [FS014: User [oozie] is not authorized to perform [DECRYPT_EEK] on key with ACL name [encrypted_key]!!]
> org.apache.oozie.action.ActionExecutorException: FS014: User [oozie] is not authorized to perform [DECRYPT_EEK] on key with ACL name [encrypted_key]!!
> at org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:463)
> at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:441)
> at org.apache.oozie.action.hadoop.FsActionExecutor.touchz(FsActionExecutor.java:523)
> at org.apache.oozie.action.hadoop.FsActionExecutor.doOperations(FsActionExecutor.java:199)
> at org.apache.oozie.action.hadoop.FsActionExecutor.start(FsActionExecutor.java:563)
> at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:232)
> at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
> at org.apache.oozie.command.XCommand.call(XCommand.java:286)
> at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
> at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.security.authorize.AuthorizationException: User [oozie] is not authorized to perform [DECRYPT_EEK] on key with ACL name [encrypted_key]!!
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
> at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:607)
> at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:565)
> at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:832)
> at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:209)
> at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:205)
> at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94)
> at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:205)
> at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388)
> at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1440)
> at org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1542)
> at org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1527)
> at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:408)
> at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:401)
> at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:401)
> at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:344)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:923)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:904)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:801)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:790)
> at org.apache.oozie.action.hadoop.FsActionExecutor.touchz(FsActionExecutor.java:519){code}
> The operation should have succeeded as [example_user] is the owner of the [encrypted_key]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org