You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2017/09/05 06:53:00 UTC
[jira] [Commented] (HADOOP-14808) Hadoop keychain
[ https://issues.apache.org/jira/browse/HADOOP-14808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153169#comment-16153169 ]
Hadoop QA commented on HADOOP-14808:
------------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m 3s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 2m 5s{color} | {color:orange} root: The patch generated 11 new + 206 unchanged - 3 fixed = 217 total (was 209) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 39s{color} | {color:red} hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 43s{color} | {color:red} hadoop-tools/hadoop-azure-datalake generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 17s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 43s{color} | {color:green} hadoop-azure-datalake in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 35s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 91m 59s{color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-common-project/hadoop-common |
| | Found reliance on default encoding in org.apache.hadoop.security.CredentialProviderKeychainLoader.load(Keychain, Configuration):in org.apache.hadoop.security.CredentialProviderKeychainLoader.load(Keychain, Configuration): String.getBytes() At CredentialProviderKeychainLoader.java:[line 59] |
| FindBugs | module:hadoop-tools/hadoop-azure-datalake |
| | Found reliance on default encoding in org.apache.hadoop.fs.adl.AzureCLIKeychainLoader.loadAccessTokensFile(Credentials, File):in org.apache.hadoop.fs.adl.AzureCLIKeychainLoader.loadAccessTokensFile(Credentials, File): String.getBytes() At AzureCLIKeychainLoader.java:[line 61] |
| Failed junit tests | hadoop.security.TestKDiag |
| | hadoop.security.alias.TestCredentialProviderFactory |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HADOOP-14808 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12885299/HADOOP-14808.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml |
| uname | Linux 6439c3bb1925 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ed162b7 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/13162/artifact/patchprocess/diff-checkstyle-root.txt |
| findbugs | https://builds.apache.org/job/PreCommit-HADOOP-Build/13162/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html |
| findbugs | https://builds.apache.org/job/PreCommit-HADOOP-Build/13162/artifact/patchprocess/new-findbugs-hadoop-tools_hadoop-azure-datalake.html |
| unit | https://builds.apache.org/job/PreCommit-HADOOP-Build/13162/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/13162/testReport/ |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure-datalake U: . |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/13162/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Hadoop keychain
> ---------------
>
> Key: HADOOP-14808
> URL: https://issues.apache.org/jira/browse/HADOOP-14808
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Affects Versions: 2.7.0
> Reporter: John Zhuge
> Assignee: John Zhuge
> Attachments: HADOOP-14808.001.patch
>
>
> Extend the idea from HADOOP-6520 "UGI should load tokens from the environment" to a generic lightweight "keychain" design. Load keys (secrets) into a keychain in UGI (secret map) at startup. YARN will distribute them securely into each container. The Hadoop code running in the container can then retrieve the credentials from UGI.
> The use case is Bring Your Own Key (BYOK) credentials for cloud connectors (adl, wasb, s3a, etc.), while Hadoop authentication is still Kerberos. No configuration change, no admin involved. It will support YARN applications initially, e.g., DistCp, Tera Suite, Spark-on-Yarn, etc.
> Implementation is surprisingly simple because almost all pieces are in place:
> * Retrieve secrets from UGI using {{conf.getPassword}} backed by the existing Credential Provider class {{UserProvider}}
> * Reuse Credential Provider classes and interface to define local permanent or transient credential store, e.g., {{LocalJavaKeyStoreProvider}}
> * New: create a new transient Credential Provider that logs into AAD with username/password or device code, and then put the Client ID and Refresh Token into the keychain
> * New: create a new permanent Credential Provider based on Hadoop configuration XML, for dev/testing purpose.
> Links
> * HADOOP-11766 Generic token authentication support for Hadoop
> * HADOOP-11744 Support OAuth2 in Hadoop
> * HADOOP-10959 A Kerberos based token authentication approach
> * HADOOP-9392 Token based authentication and Single Sign On
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org