You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Daryn Sharp (JIRA)" <ji...@apache.org> on 2011/08/12 22:04:27 UTC
[jira] [Created] (HADOOP-7541) Add issuer field to delegation
tokens
Add issuer field to delegation tokens
-------------------------------------
Key: HADOOP-7541
URL: https://issues.apache.org/jira/browse/HADOOP-7541
Project: Hadoop Common
Issue Type: New Feature
Components: security
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Fix For: 0.20.205.0
Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7541) Add issuer field to delegation
tokens
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HADOOP-7541:
--------------------------------
Attachment: HADOOP-7541.patch
> Add issuer field to delegation tokens
> -------------------------------------
>
> Key: HADOOP-7541
> URL: https://issues.apache.org/jira/browse/HADOOP-7541
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7541.patch
>
>
> Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7541) Add issuer field to delegation
tokens
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085108#comment-13085108 ]
Daryn Sharp commented on HADOOP-7541:
-------------------------------------
Yes, this and all the changes in the related jiras need to be ported to trunk. Currently I'm trying to break the overall delegation renewal changes into easily reviewable chunks. Given the importance of the 205 release, I'd like to focus on getting everything fully integrated. Then I'd like to simplify the trunk integration by having a jira per component with all required changes. Does that sound reasonable?
> Add issuer field to delegation tokens
> -------------------------------------
>
> Key: HADOOP-7541
> URL: https://issues.apache.org/jira/browse/HADOOP-7541
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7541.patch
>
>
> Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7541) Add issuer field to delegation
tokens
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084383#comment-13084383 ]
Daryn Sharp commented on HADOOP-7541:
-------------------------------------
The problem is that token renewal is very error-prone and fails in a number of cases, esp. involving hftp. Long running distcp jobs fail or hang when the renewal goes bad. The crux of the problem is tokens lack traceability to their origin, so the renewal code is based on assumptions, outright guesses, and intimate knowledge it ought not have of filesystems.
Any filesystem token including hftp, will essentially be renewed by:
{code}
FileSystem.get(token.getIssuer(), conf, ugi).renewDelegationToken(token);
{code}
The other suggestion is changing token types for every filesystem, having a static class with hardcoded conditionals(!) to do renewals. That will cause entirely unnecessary coupling of filesystems and mapreduce, causing recompilation of mapreduce to add a fs -- which leads to talk of service loaders to hide/break the coupling. The alternate is the one line above.
I don't know enough about the wire compat. I thought that allowing the field not to be present would avoid the problem. I suppose the alternative is to hijack the service field to hold both the service and the issuer.
> Add issuer field to delegation tokens
> -------------------------------------
>
> Key: HADOOP-7541
> URL: https://issues.apache.org/jira/browse/HADOOP-7541
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7541.patch
>
>
> Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7541) Add issuer field to delegation
tokens
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084455#comment-13084455 ]
Aaron T. Myers commented on HADOOP-7541:
----------------------------------------
Does a corresponding change also need to be made on trunk?
> Add issuer field to delegation tokens
> -------------------------------------
>
> Key: HADOOP-7541
> URL: https://issues.apache.org/jira/browse/HADOOP-7541
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7541.patch
>
>
> Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7541) Add issuer field to delegation
tokens
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085752#comment-13085752 ]
Daryn Sharp commented on HADOOP-7541:
-------------------------------------
Yes. This jira is really a subtask of MAPREDUCE-2764, except apparently subtasks cannot be a different component so I linked them. My focus is on 205 since the QE cycle is imminent but I will immediately merge to trunk when MAPREDUCE-2764 is completed -- hopefully this week. Since many of the subtasks of MAPREDUCE-2764 slice across common/hdfs/mapreduce, my hope is that I won't have up to 3*subtasks of jiras for trunk. Do you still consider this unacceptable?
> Add issuer field to delegation tokens
> -------------------------------------
>
> Key: HADOOP-7541
> URL: https://issues.apache.org/jira/browse/HADOOP-7541
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7541.patch
>
>
> Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7541) Add issuer field to delegation
tokens
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101339#comment-13101339 ]
Daryn Sharp commented on HADOOP-7541:
-------------------------------------
Can we re-target and consider this for trunk?
> Add issuer field to delegation tokens
> -------------------------------------
>
> Key: HADOOP-7541
> URL: https://issues.apache.org/jira/browse/HADOOP-7541
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: HADOOP-7541.patch
>
>
> Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7541) Add issuer field to delegation
tokens
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084346#comment-13084346 ]
Owen O'Malley commented on HADOOP-7541:
---------------------------------------
Adding fields to the delegation token will break wire compatibility, which is really difficult. What is the use case where the service field is insufficient?
> Add issuer field to delegation tokens
> -------------------------------------
>
> Key: HADOOP-7541
> URL: https://issues.apache.org/jira/browse/HADOOP-7541
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7541.patch
>
>
> Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7541) Add issuer field to delegation
tokens
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085167#comment-13085167 ]
Todd Lipcon commented on HADOOP-7541:
-------------------------------------
Are you implying this should go into 205 before trunk? The 0.23 release is also important, and having features/improvements in 205 that aren't in trunk is unacceptable IMO.
> Add issuer field to delegation tokens
> -------------------------------------
>
> Key: HADOOP-7541
> URL: https://issues.apache.org/jira/browse/HADOOP-7541
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7541.patch
>
>
> Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7541) Add issuer field to delegation
tokens
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-7541:
----------------------------------
Resolution: Won't Fix
Fix Version/s: (was: 0.20.205.0)
Status: Resolved (was: Patch Available)
> Add issuer field to delegation tokens
> -------------------------------------
>
> Key: HADOOP-7541
> URL: https://issues.apache.org/jira/browse/HADOOP-7541
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: HADOOP-7541.patch
>
>
> Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7541) Add issuer field to delegation
tokens
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HADOOP-7541:
--------------------------------
Release Note: Add getters/setters for an issuer. DFS token renew & cancel verify that it issued the token before taking action.
Status: Patch Available (was: Open)
> Add issuer field to delegation tokens
> -------------------------------------
>
> Key: HADOOP-7541
> URL: https://issues.apache.org/jira/browse/HADOOP-7541
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7541.patch
>
>
> Tokens currently lack traceability to its issuer. This complicates the ability to reliably renew tokens. Tokens should have an optional issuer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira