You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Hadoop QA (Jira)" <ji...@apache.org> on 2021/04/14 15:35:00 UTC
[jira] [Commented] (YARN-10735) Unmanaged AM is won't populate
AMRMToken to ApplicationReport in secure cluster
[ https://issues.apache.org/jira/browse/YARN-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321095#comment-17321095 ]
Hadoop QA commented on YARN-10735:
----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 49s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m 45s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s{color} | {color:green}{color} | {color:green} trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 53s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 59s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 17m 10s{color} | {color:green}{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s{color} | {color:green}{color} | {color:green} trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 20m 36s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 2m 6s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 57s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green}{color} | {color:green} the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s{color} | {color:green}{color} | {color:green} the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 2s{color} | {color:green}{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s{color} | {color:green}{color} | {color:green} the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s{color} | {color:green}{color} | {color:green} the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 2m 10s{color} | {color:green}{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}101m 10s{color} | {color:green}{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 29s{color} | {color:green}{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}175m 41s{color} | {color:black}{color} | {color:black}{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/909/artifact/out/Dockerfile |
| JIRA Issue | YARN-10735 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/13023857/YARN-10735.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle spotbugs |
| uname | Linux 1abcc5e15e00 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 7d6f0ca0bd2 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/909/testReport/ |
| Max. process+thread count | 835 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
| Console output | https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/909/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Unmanaged AM is won't populate AMRMToken to ApplicationReport in secure cluster
> -------------------------------------------------------------------------------
>
> Key: YARN-10735
> URL: https://issues.apache.org/jira/browse/YARN-10735
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Wang, Xinglong
> Assignee: Wang, Xinglong
> Priority: Minor
> Attachments: YARN-10735.001.patch
>
>
> With kerberos enabled, NPE will be reported when launching UnmanagedAMLauncher.
> It is due to there is no AMRMToken is returned in ApplicationReport. After a while investigation, it turns out that RMAppImpl has a bad if condition inside createAndGetApplicationReport
> {code:java}
> 21/04/14 02:46:01 INFO unmanagedamlauncher.UnmanagedAMLauncher: Initializing Client
> 21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Starting Client
> 21/04/14 02:46:02 INFO client.AHSProxy: Connecting to Application History server at /0.0.0.0:10200
> 21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Setting up application submission context for ASM
> 21/04/14 02:46:02 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
> 21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Setting unmanaged AM
> 21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Submitting application to ASM
> 21/04/14 02:46:03 INFO impl.YarnClientImpl: Submitted application application_1618393442264_0002
> 21/04/14 02:46:04 INFO unmanagedamlauncher.UnmanagedAMLauncher: Got application report from ASM for, appId=2, appAttemptId=appattempt_1618393442264_0002_000001, clientToAMToken=Token { kind: YARN_CLIENT_TOKEN, service: }, appDiagnostics=AM container is launched, waiting for AM container to Register with RM, appMasterHost=N/A, appQueue=abc, appMasterRpcPort=-1, appStartTime=1618393562917, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=abc
> 21/04/14 02:46:04 INFO unmanagedamlauncher.UnmanagedAMLauncher: Launching AM with application attempt id appattempt_1618393442264_0002_000001
> 21/04/14 02:46:04 FATAL unmanagedamlauncher.UnmanagedAMLauncher: Error running Client
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.launchAM(UnmanagedAMLauncher.java:186)
> at org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.run(UnmanagedAMLauncher.java:354)
> at org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.main(UnmanagedAMLauncher.java:111)
> {code}
>
> {code:java}
> public ApplicationReport createAndGetApplicationReport(String clientUserName,
> boolean allowAccess) {
> ......
> if (currentAttempt != null &&
> currentAttempt.getAppAttemptState() == RMAppAttemptState.LAUNCHED) {
> if (getApplicationSubmissionContext().getUnmanagedAM() &&
> clientUserName != null && getUser().equals(clientUserName)) {
> Token<AMRMTokenIdentifier> token = currentAttempt.getAMRMToken();
> if (token != null) {
> amrmToken = BuilderUtils.newAMRMToken(token.getIdentifier(),
> token.getKind().toString(), token.getPassword(),
> token.getService().toString());
> }
> }
> }
> {code}
> clientUserName is fullName of a kerberos principle like abc@domain.com whereas getUser() will return the username recorded in RMAppImpl which is short name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org