You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Alejandro Abdelnur (Created) (JIRA)" <ji...@apache.org> on 2011/11/09 02:13:51 UTC
[jira] [Created] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
JobClient.getDelegationToken() returns NULL
-------------------------------------------
Key: MAPREDUCE-3380
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 0.23.0
Reporter: Alejandro Abdelnur
Priority: Blocker
Fix For: 0.23.1
The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148790#comment-13148790 ]
Mahadev konar commented on MAPREDUCE-3380:
------------------------------------------
Ok take that back. My misunderstanding.
Looks like we need something like DelegationTokenSecretManager (as in JT) in RM which needs to create delegation tokens for applications to be able to submit jobs in the RM. I am still not sure abt the JHS token. Will get more info from the Oozie folks.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Status: Patch Available (was: Open)
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Attachment: MAPREDUCE-3380.patch
An updated patch. Still preliminary. The patch adds delegation tokens to JHS as well. Still doing some more integration testing, adding unit tests where possible and adding some javadocs.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183776#comment-13183776 ]
Hadoop QA commented on MAPREDUCE-3380:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12510131/MAPREDUCE-3380-20120110.1.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1587//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1587//console
This message is automatically generated.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183813#comment-13183813 ]
Hudson commented on MAPREDUCE-3380:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1542 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1542/])
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1229855
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRClientProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRDelegationTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/client/MRClientProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/service/MRClientProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/MRClientProtocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/TestRPCFactories.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JHSDelegationTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/DelegationToken.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/DelegationTokenPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientRMSecurityInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMDelegationTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/RMDelegationTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Alejandro Abdelnur (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148201#comment-13148201 ]
Alejandro Abdelnur commented on MAPREDUCE-3380:
-----------------------------------------------
@Mahadev, the user running the Oozie server (unix UID for Pseudo auth, kerberos principal for Kerberos auth) is configured in Hadoop as a proxyuser. Then, users submit jobs to Oozie and Oozie does the corresponding doAs.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182171#comment-13182171 ]
Hadoop QA commented on MAPREDUCE-3380:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509817/MAPREDUCE-3380.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1565//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1565//console
This message is automatically generated.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Attachment: MAPREDUCE-3380.patch
Fixed the test cases.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "Alejandro Abdelnur (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated MAPREDUCE-3380:
------------------------------------------
Issue Type: Sub-task (was: Bug)
Parent: MAPREDUCE-3101
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "Alejandro Abdelnur (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147490#comment-13147490 ]
Alejandro Abdelnur commented on MAPREDUCE-3380:
-----------------------------------------------
John, yes, we need 2 JIRAs one to handle NULL delegation token (your patch) and one to get a delegation token (the intention of this JIRA). IMO both are blockers. thanks.
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Assignee: John George
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "Alejandro Abdelnur (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated MAPREDUCE-3380:
------------------------------------------
Affects Version/s: 0.24.0
Fix Version/s: 0.24.0
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3380:
-----------------------------------------------
Status: Open (was: Patch Available)
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184028#comment-13184028 ]
Hudson commented on MAPREDUCE-3380:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #135 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/135/])
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev) - Merging r1229855 from trunk.
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1229856
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRClientProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRDelegationTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/client/MRClientProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/service/MRClientProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/MRClientProtocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/TestRPCFactories.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JHSDelegationTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/DelegationToken.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/DelegationTokenPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientRMSecurityInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMDelegationTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/RMDelegationTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3380:
-----------------------------------------------
Attachment: MAPREDUCE-3380-20120110.1.txt
+1 overall. Cleaned the patch a bit.
- Adding extra check for Kerberos authentication when asking for delegation tokens
- Reusing the same record DelegationToken for both RM side and JHS side tokens
- Removed unnecessary changes to minimize the patch size and to make it tractable for review.
- There may be better ways to populate the history related delegation tokens but the current one to pass around info via conf should do the trick for now.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "Ahmed Radwan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147271#comment-13147271 ]
Ahmed Radwan commented on MAPREDUCE-3380:
-----------------------------------------
And if you are running locally, client will be LocalJobRunner (and not YarnRunner) and client.getDelegationToken() will also return null.
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148777#comment-13148777 ]
Mahadev konar commented on MAPREDUCE-3380:
------------------------------------------
@Vinod,
bq. To communicate to the JobHistory server, again we need some delegation token. I guess we can use the mapreduce JobToken itself but AM somehow needs to let the JHS know about it - through JobHistory file, or a direct communication protocol.
Do we really need the JHS token? As far as I understand the getDelgationtoken is used by a mapred task to submit another job to the cluster. Does it check for job status from JHS as well? In that case we just need to return the application token generated in AMLauncher (probably need to create that early on) no? Also, do we need the application token to be based on app attemptid? Can we just use the application id?
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-3380:
-----------------------------------
Fix Version/s: (was: 0.23.1)
(was: 0.24.0)
Assignee: (was: John George)
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "Ahmed Radwan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147264#comment-13147264 ]
Ahmed Radwan commented on MAPREDUCE-3380:
-----------------------------------------
John, 1 & 2 in your comment are the same. client in #1, is the YarnRunner which calls ResourceMgrDelegate.getDelegationToken() but it returns null because it is not implemented.
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181674#comment-13181674 ]
Hadoop QA commented on MAPREDUCE-3380:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509733/MAPREDUCE-3380.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The patch appears to cause tar ant target to fail.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed the unit tests build
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1559//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1559//console
This message is automatically generated.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Attachment: MAPREDUCE-3380.patch
Finally was able to get a secure oozie running and test this patch out. The patch works. The only issue is that oozie still reports the job as running although the launcher and action job complete. Will have to debug that with some oozie folks but as for the token this should be it.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "Ahmed Radwan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147273#comment-13147273 ]
Ahmed Radwan commented on MAPREDUCE-3380:
-----------------------------------------
I see you point John, thanks.
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Attachment: MAPREDUCE-3380_v1.patch
This a very preliminary patch. Lot of issues still need to be addressed. The patch currently just gets an application delegation token from RM. More work needs to be done to add the tokens by default to the client credentials when getdelegationtoken is called. Also, similar RPC needs to be added to jobhistory for the job history server to be a able to grant application delegation tokens.
Lot more testing needs to be done and also will add tests.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184022#comment-13184022 ]
Hudson commented on MAPREDUCE-3380:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #922 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/922/])
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1229855
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRClientProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRDelegationTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/client/MRClientProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/service/MRClientProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/MRClientProtocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/TestRPCFactories.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JHSDelegationTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/DelegationToken.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/DelegationTokenPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientRMSecurityInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMDelegationTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/RMDelegationTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181879#comment-13181879 ]
Hadoop QA commented on MAPREDUCE-3380:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509768/MAPREDUCE-3380.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1562//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1562//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1562//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1562//console
This message is automatically generated.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "John George (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147261#comment-13147261 ]
John George commented on MAPREDUCE-3380:
----------------------------------------
I ran into the same issue while starting oozie. To me it looks like there are two things here:
1. Cluster.getDelegationToken() throws NPE if client.getDelegationToken() returns null.
2. ResourceMgrDelegate.getDelegationToken() not implemented.
Alejandro - Are you not hitting #1?
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Attachment: MAPREDUCE-3380.patch
Updated patch with some fixes. Almost there, needs more javadoc and some more testing.
There are a couple of api's that I havent added in this patch - cancelToken, renewToken. I will be filing another jira to add those. Will also be adding non dummy renewers for tokens in a separate jira.
Ill try to just get the current feature set in the patch as per this jira, so that we can get oozie working and then open new jiras for the remaining work.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Attachment: MAPREDUCE-3380.patch
Fixed the findbugs issue.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148776#comment-13148776 ]
Mahadev konar commented on MAPREDUCE-3380:
------------------------------------------
@Vinod,
bq. To communicate to the JobHistory server, again we need some delegation token. I guess we can use the mapreduce JobToken itself but AM somehow needs to let the JHS know about it - through JobHistory file, or a direct communication protocol.
Do we really need the JHS token? As far as I understand the getDelgationtoken is used by a mapred task to submit another job to the cluster. Does it check for job status from JHS as well? In that case we just need to return the application token generated in AMLauncher (probably need to create that early on) no? Also, do we need the application token to be based on app attemptid? Can we just use the application id?
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183485#comment-13183485 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3380:
----------------------------------------------------
Looking at the patch for review/commit.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Component/s: mrv2
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-3380:
-----------------------------------
Attachment: MAPREDUCE-3380.branch-0.23.patch
Attaching a patch for the NPE - assuming based on Alejandro's comment that the NPE is the blocker and not implementation of the getDelegationToken by ResourceMgrDelegate. If not, let me know and I can file another JIRA for the NPE (or vice versa)
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Assignee: John George
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "John George (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148106#comment-13148106 ]
John George commented on MAPREDUCE-3380:
----------------------------------------
"Any user who started the tomcat service for oozie" is what the oozie guys told me.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "John George (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147269#comment-13147269 ]
John George commented on MAPREDUCE-3380:
----------------------------------------
Thanks for responding Ahmed. To me they are different because cluster.getDelegationToken() should not throw NPE even if a 'client' does not implement getDelegationToken(). I have a two line patch to fix the NPE, but would like to hear from other experts about implementing the getDelegationToken() itself.
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183790#comment-13183790 ]
Hudson commented on MAPREDUCE-3380:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1523 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1523/])
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1229855
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRClientProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRDelegationTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/client/MRClientProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/service/MRClientProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/MRClientProtocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/TestRPCFactories.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JHSDelegationTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/DelegationToken.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/DelegationTokenPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientRMSecurityInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMDelegationTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/RMDelegationTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3380:
-------------------------------------
Status: Open (was: Patch Available)
Mahadev, can you pls look at the multi-thread correctness findbugs warning?
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184063#comment-13184063 ]
Hudson commented on MAPREDUCE-3380:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #955 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/955/])
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1229855
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRClientProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRDelegationTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/client/MRClientProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/service/MRClientProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/MRClientProtocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/TestRPCFactories.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JHSDelegationTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/DelegationToken.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/DelegationTokenPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientRMSecurityInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMDelegationTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/RMDelegationTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183788#comment-13183788 ]
Hudson commented on MAPREDUCE-3380:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1596 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1596/])
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1229855
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRClientProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRDelegationTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/client/MRClientProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/service/MRClientProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/MRClientProtocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/TestRPCFactories.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JHSDelegationTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/DelegationToken.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/DelegationTokenPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientRMSecurityInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMDelegationTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/RMDelegationTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Amol Kekre (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181037#comment-13181037 ]
Amol Kekre commented on MAPREDUCE-3380:
---------------------------------------
any updates?
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Mona Chitnis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150846#comment-13150846 ]
Mona Chitnis commented on MAPREDUCE-3380:
-----------------------------------------
Adding another comment on the question
What user does Oozie run as?
the doAs logic handles the job to be submitted with any authenticated user. Then the user name for the hadoop job should be the same as the one with which the oozie job was submitted.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148048#comment-13148048 ]
Mahadev konar commented on MAPREDUCE-3380:
------------------------------------------
Vinod/John,
What user does Oozie run as?
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183792#comment-13183792 ]
Hudson commented on MAPREDUCE-3380:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #353 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/353/])
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev) - Merging r1229855 from trunk.
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1229856
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRClientProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRDelegationTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/client/MRClientProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/service/MRClientProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/MRClientProtocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/TestRPCFactories.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JHSDelegationTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/DelegationToken.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/DelegationTokenPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientRMSecurityInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMDelegationTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/RMDelegationTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "John George (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147772#comment-13147772 ]
John George commented on MAPREDUCE-3380:
----------------------------------------
Filed MR-3392 to track the NPE since this JIRA was originally filed to fix ResourceMgrDelegate.getDelegationToken(). Please ignore the first two patches.
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Assignee: John George
> Priority: Blocker
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks Vinod for the review and fixes.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183794#comment-13183794 ]
Hudson commented on MAPREDUCE-3380:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #363 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/363/])
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev) - Merging r1229855 from trunk.
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1229856
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRClientProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRDelegationTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/client/MRClientProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/service/MRClientProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/MRClientProtocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/TestRPCFactories.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JHSDelegationTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/DelegationToken.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/DelegationTokenPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientRMSecurityInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMDelegationTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/RMDelegationTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "John George (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147279#comment-13147279 ]
John George commented on MAPREDUCE-3380:
----------------------------------------
I still need to write tests etc. Will submit a patch with tests a little later.
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Assignee: John George
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184053#comment-13184053 ]
Hudson commented on MAPREDUCE-3380:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #157 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/157/])
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev) - Merging r1229855 from trunk.
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1229856
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRClientProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRDelegationTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/client/MRClientProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/service/MRClientProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/MRClientProtocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/TestRPCFactories.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JHSDelegationTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/DelegationToken.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/DelegationTokenPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientRMSecurityInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMDelegationTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/RMDelegationTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3380:
-----------------------------------------------
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Status: Patch Available (was: Open)
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Status: Patch Available (was: Open)
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) Token infrastructure for
running clients which are not kerberos authenticated
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183802#comment-13183802 ]
Hudson commented on MAPREDUCE-3380:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #374 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/374/])
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev) - Merging r1229855 from trunk.
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1229856
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRClientProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/MRDelegationTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/client/MRClientProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/impl/pb/service/MRClientProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/MRClientProtocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/TestRPCFactories.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JHSDelegationTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestYarnClientProtocolProvider.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetDelegationTokenResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetDelegationTokenResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/DelegationToken.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/DelegationTokenPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientRMSecurityInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMDelegationTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/RMTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/RMDelegationTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380-20120110.1.txt, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3380:
-----------------------------------------------
Affects Version/s: (was: 0.24.0)
Fix Version/s: 0.23.1
Summary: Token infrastructure for running clients which are not kerberos authenticated (was: JobClient.getDelegationToken() returns NULL)
This is a deep issue. I knew about this in the back of my mind, but put it on the back-burner because of more pressing tasks.
Client (the usual kerberos authenticated one) talks to RM (for submitting apps and getting application-status), gets a client-token which it then uses to connect to the AM, and the finally connects to the JobHistoryServer to obtain the status when the job completes.
To make oozie work (i.e. making a client that isn't kerberos authenticated), we need the following:
- To communicate to the RM, RM needs to support ApplicationDelegationTokens which the JobClient should then propagate to the Oozie action.
- To communicate to the AM, the present day client-token generated and given to the client by the RM should suffice.
- To communicate to the JobHistory server, again we need some delegation token. I guess we can use the mapreduce JobToken itself but AM somehow needs to let the JHS know about it - through JobHistory file, or a direct communication protocol.
We should perhaps split it into two separate tickets.
Modifying title to reflect the issues here.
I really am not sure about the timelines for fixing all this, but setting 0.23.1 tentatively.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-3380:
-----------------------------------
Attachment: MAPREDUCE-3380.branch-0.23.patch
Attaching a patch with unit test.
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Assignee: John George
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "Alejandro Abdelnur (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147277#comment-13147277 ]
Alejandro Abdelnur commented on MAPREDUCE-3380:
-----------------------------------------------
John, you are correct, there are 2 different things. #2 should be implemented properly to work in secure clusters. #1 should be fixed to allow working when there is no delegation token available. Currently we are hitting #1 when trying to run Oozie against 0.23
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar reassigned MAPREDUCE-3380:
----------------------------------------
Assignee: Mahadev konar
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3380) JobClient.getDelegationToken()
returns NULL
Posted by "John George (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George reassigned MAPREDUCE-3380:
--------------------------------------
Assignee: John George
> JobClient.getDelegationToken() returns NULL
> -------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Alejandro Abdelnur
> Assignee: John George
> Priority: Blocker
> Fix For: 0.24.0, 0.23.1
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3380) Token infrastructure for running
clients which are not kerberos authenticated
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3380:
-------------------------------------
Status: Open (was: Patch Available)
Ill address the test/javac/findbugs failures.
> Token infrastructure for running clients which are not kerberos authenticated
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3380
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.branch-0.23.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380.patch, MAPREDUCE-3380_v1.patch
>
>
> The JobClient.getDelegationToken() method is returning NULL, this makes Oozie fail when trying to get the delegation token to use it for starting a job.
> What is seems to be happing is that Jobclient.getDelegationToken() calls Cluster.getDelegationToken() that calls YarnRunner.getDelegationToken() that calls ResourceMgrDelegate.getDelegationToken(). And the last one is not implemented. (Thanks Ahmed for tracing this in MR2 code)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira