You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Daryn Sharp (Created) (JIRA)" <ji...@apache.org> on 2011/11/28 17:05:40 UTC
[jira] [Created] (MAPREDUCE-3475) JT can't renew its own tokens
JT can't renew its own tokens
-----------------------------
Key: MAPREDUCE-3475
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.20.205.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3475) JT can't renew its own tokens
Posted by "Matt Foley (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated MAPREDUCE-3475:
----------------------------------
Target Version/s: 1.0.0, 0.22.1 (was: 1.0.0)
Fix Version/s: 1.0.0
Taking Owen's comment for a +1, I have committed this patch to 1.0.0 and branch-1.
Leaving the jira open with the expectation that this same fix is needed in v0.22.
> JT can't renew its own tokens
> -----------------------------
>
> Key: MAPREDUCE-3475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 1.0.0
>
> Attachments: MAPREDUCE-3475-1.patch, MAPREDUCE-3475.patch
>
>
> When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
> After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3475) JT can't renew its own tokens
Posted by "Owen O'Malley (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158849#comment-13158849 ]
Owen O'Malley commented on MAPREDUCE-3475:
------------------------------------------
Well, that is a mistake. But since the filter matches the current behavior, that is fine. The patch looks fine to me.
> JT can't renew its own tokens
> -----------------------------
>
> Key: MAPREDUCE-3475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3475-1.patch, MAPREDUCE-3475.patch
>
>
> When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
> After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3475) JT can't renew its own tokens
Posted by "Owen O'Malley (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158838#comment-13158838 ]
Owen O'Malley commented on MAPREDUCE-3475:
------------------------------------------
Daryn,
Can you look for a match with either the owner or renewer? They should both be able to do renewals.
> JT can't renew its own tokens
> -----------------------------
>
> Key: MAPREDUCE-3475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3475-1.patch, MAPREDUCE-3475.patch
>
>
> When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
> After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3475) JT can't renew its own tokens
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated MAPREDUCE-3475:
-----------------------------------
Attachment: MAPREDUCE-3475-1.patch
A truly general solution will require touching all components. This patch fixes the issues for MR tokens by checking the token's renewer against the login user.
> JT can't renew its own tokens
> -----------------------------
>
> Key: MAPREDUCE-3475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3475-1.patch, MAPREDUCE-3475.patch
>
>
> When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
> After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3475) JT can't renew its own tokens
Posted by "Owen O'Malley (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158619#comment-13158619 ]
Owen O'Malley commented on MAPREDUCE-3475:
------------------------------------------
Actually, I think this is the wrong direction. In general the JT should renew its own tokens.
I'd suggest rather that the broken tokens from Oozie not be renewed by not trying to renew any tokens that don't have the proper user as either owner or renewer. That is a much more general fix that will handle a wider set of upstream problems.
> JT can't renew its own tokens
> -----------------------------
>
> Key: MAPREDUCE-3475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3475.patch
>
>
> When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
> After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3475) JT can't renew its own tokens
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158575#comment-13158575 ]
Hadoop QA commented on MAPREDUCE-3475:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505350/MAPREDUCE-3475.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1348//console
This message is automatically generated.
> JT can't renew its own tokens
> -----------------------------
>
> Key: MAPREDUCE-3475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3475.patch
>
>
> When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
> After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3475) JT can't renew its own tokens
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158844#comment-13158844 ]
Daryn Sharp commented on MAPREDUCE-3475:
----------------------------------------
Unfortunately the secret manager only looks at renewer. The owner isn't factored in for some reason...
> JT can't renew its own tokens
> -----------------------------
>
> Key: MAPREDUCE-3475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3475-1.patch, MAPREDUCE-3475.patch
>
>
> When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
> After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3475) JT can't renew its own tokens
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated MAPREDUCE-3475:
-----------------------------------
Attachment: MAPREDUCE-3475.patch
> JT can't renew its own tokens
> -----------------------------
>
> Key: MAPREDUCE-3475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3475.patch
>
>
> When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
> After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3475) JT can't renew its own tokens
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158791#comment-13158791 ]
Hadoop QA commented on MAPREDUCE-3475:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505394/MAPREDUCE-3475-1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1355//console
This message is automatically generated.
> JT can't renew its own tokens
> -----------------------------
>
> Key: MAPREDUCE-3475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3475-1.patch, MAPREDUCE-3475.patch
>
>
> When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
> After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3475) JT can't renew its own tokens
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated MAPREDUCE-3475:
-----------------------------------
Status: Patch Available (was: Open)
> JT can't renew its own tokens
> -----------------------------
>
> Key: MAPREDUCE-3475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3475.patch
>
>
> When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.
> After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira