You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Kan Zhang (JIRA)" <ji...@apache.org> on 2010/03/14 05:29:27 UTC
[jira] Created: (HADOOP-6632) Support for using different Kerberos
keys for different instances of Hadoop services
Support for using different Kerberos keys for different instances of Hadoop services
------------------------------------------------------------------------------------
Key: HADOOP-6632
URL: https://issues.apache.org/jira/browse/HADOOP-6632
Project: Hadoop Common
Issue Type: Improvement
Reporter: Kan Zhang
Assignee: Kan Zhang
We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6632) Support for using different Kerberos
keys for different instances of Hadoop services
Posted by "Jitendra Nath Pandey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jitendra Nath Pandey updated HADOOP-6632:
-----------------------------------------
Attachment: HADOOP-6632-Y20S-22.patch
New patch for hadoop-20.
The hostname patter is changed to _HOST, and the renewer for delegation tokens is changed to shortnames.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6632) Support for using different Kerberos
keys for different instances of Hadoop services
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kan Zhang updated HADOOP-6632:
------------------------------
Status: Patch Available (was: Open)
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890240#action_12890240 ]
Hudson commented on HADOOP-6632:
--------------------------------
Integrated in Hadoop-Common-trunk #398 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/398/])
HADOOP-6632. Adds support for using different keytabs for different servers in a Hadoop cluster. In the earier implementation, all servers of a certain type \(like TaskTracker\), would have the same keytab and the same principal. Now the principal name is a pattern that has _HOST in it. Contributed by Kan Zhang & Jitendra Pandey.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Fix For: 0.22.0
>
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12926125#action_12926125 ]
Hudson commented on HADOOP-6632:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/])
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Fix For: 0.22.0
>
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6632) Support for using different Kerberos
keys for different instances of Hadoop services
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-6632:
--------------------------------
Attachment: 6632.mr.patch
A minor fix for the MR side to reuse filesystem handles
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: 6632.mr.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6632) Support for using different Kerberos
keys for different instances of Hadoop services
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kan Zhang updated HADOOP-6632:
------------------------------
Status: Patch Available (was: Open)
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: 6632.mr.patch, c6632-05.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890112#action_12890112 ]
Hudson commented on HADOOP-6632:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #331 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/331/])
HADOOP-6632. Adds support for using different keytabs for different servers in a Hadoop cluster. In the earier implementation, all servers of a certain type \(like TaskTracker\), would have the same keytab and the same principal. Now the principal name is a pattern that has _HOST in it. Contributed by Kan Zhang & Jitendra Pandey.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Fix For: 0.22.0
>
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890104#action_12890104 ]
Hadoop QA commented on HADOOP-6632:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12449900/c6632-07.patch
against trunk revision 965556.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/624/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/624/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/624/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/624/console
This message is automatically generated.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6632) Support for using different Kerberos
keys for different instances of Hadoop services
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kan Zhang updated HADOOP-6632:
------------------------------
Status: Open (was: Patch Available)
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6632) Support for using different Kerberos
keys for different instances of Hadoop services
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-6632:
--------------------------------
Status: Resolved (was: Patch Available)
Fix Version/s: 0.22.0
Resolution: Fixed
I just committed this. Thanks, Kan & Jitendra!
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Fix For: 0.22.0
>
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845021#action_12845021 ]
Kan Zhang commented on HADOOP-6632:
-----------------------------------
One error message we observed.
2010-03-03 07:33:50,542 INFO org.apache.hadoop.ipc.Server: IPC Server listener on
8020: readAndProcess threw exception javax.security.sasl.SaslException: GSS initia
te failed [Caused by GSSException: Failure unspecified at GSS-API level (Mechanism
level: Request is a replay (34))]. Count of bytes read: 0
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: Failure unspecified at GSS-API level
(Mechanism level: Request is a replay (34))]
at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:159)
at org.apache.hadoop.ipc.Server$Connection.saslReadAndProcess(Server.java:913)
at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1071)
at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:459)
at org.apache.hadoop.ipc.Server$Listener.run(Server.java:368)
Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34))
at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:741)
at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:323)
at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:267)
at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:137)
... 4 more
Caused by: KrbException: Request is a replay (34)
at sun.security.krb5.KrbApReq.authenticate(KrbApReq.java:299)
at sun.security.krb5.KrbApReq.<init>(KrbApReq.java:134)
at sun.security.jgss.krb5.InitSecContextToken.<init>(InitSecContextToken.java:79)
at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:724)
... 7 more
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888982#action_12888982 ]
Kan Zhang commented on HADOOP-6632:
-----------------------------------
The javadoc warnings are unrelated to this patch. Manually verified the feature on a single node cluster.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: 6632.mr.patch, c6632-05.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894158#action_12894158 ]
Todd Lipcon commented on HADOOP-6632:
-------------------------------------
It looks like the 6632.mr.patch portion was committed to ydist but not trunk - was this intentional?
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Fix For: 0.22.0
>
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6632) Support for using different Kerberos
keys for different instances of Hadoop services
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kan Zhang updated HADOOP-6632:
------------------------------
Attachment: c6632-07.patch
Uploading a new patch that simply merges with latest trunk changes. No semantic change from previous patch.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888873#action_12888873 ]
Hadoop QA commented on HADOOP-6632:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12449571/c6632-05.patch
against trunk revision 964134.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/615/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/615/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/615/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/615/console
This message is automatically generated.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: 6632.mr.patch, c6632-05.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6632) Support for using different Kerberos
keys for different instances of Hadoop services
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kan Zhang updated HADOOP-6632:
------------------------------
Attachment: HADOOP-6632-Y20S-18.patch
A patch for Yahoo 0.20S branch. Not for commit.
This patch assumes a service's Kerberos principal name is of the form "servicename/hostname@REALM", where servicename is the service type (for example, dn for DataNodes, and tt for TaskTrackers), hostname is the fully-qualified domain name of the host where the service is running on, and REALM is Kerberos realm that the service belongs to. To support the convenience of having the same conf on every host, a service's Kerberos principal name can be configured as "servicename/${FQDN}@REALM", where ${FQDN} will be substituted at runtime with the fully-qualified domain name of the host that the service is running on.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: HADOOP-6632-Y20S-18.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6632) Support for using different Kerberos
keys for different instances of Hadoop services
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kan Zhang updated HADOOP-6632:
------------------------------
Attachment: c6632-05.patch
A port for trunk.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Attachments: 6632.mr.patch, c6632-05.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894170#action_12894170 ]
Devaraj Das commented on HADOOP-6632:
-------------------------------------
Yes this was intentional. The mr patch seemed like a hack and that's why we didn't commit it to trunk, and instead raised MAPREDUCE-1824 to discuss that... BTW, the problem which the mr patch attempted to address would be significantly less once we have HADOOP-6706 committed that does retries in case of failures due to the false replay attack detection by the rpc servers. MAPREDUCE-1824 takes a low priority..
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Fix For: 0.22.0
>
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894192#action_12894192 ]
Todd Lipcon commented on HADOOP-6632:
-------------------------------------
Thanks, Deveraj. That makes sense.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Fix For: 0.22.0
>
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6632) Support for using different
Kerberos keys for different instances of Hadoop services
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890132#action_12890132 ]
Hudson commented on HADOOP-6632:
--------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #346 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/346/])
HDFS-1201. The HDFS component for HADOOP-6632. Contributed by Kan Zhang & Jitendra Pandey.
> Support for using different Kerberos keys for different instances of Hadoop services
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-6632
> URL: https://issues.apache.org/jira/browse/HADOOP-6632
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Kan Zhang
> Assignee: Kan Zhang
> Fix For: 0.22.0
>
> Attachments: 6632.mr.patch, c6632-05.patch, c6632-07.patch, HADOOP-6632-Y20S-18.patch, HADOOP-6632-Y20S-22.patch
>
>
> We tested using the same Kerberos key for all datanodes in a HDFS cluster or the same Kerberos key for all TaskTarckers in a MapRed cluster. But it doesn't work. The reason is that when datanodes try to authenticate to the namenode all at once, the Kerberos authenticators they send to the namenode may have the same timestamp and will be rejected as replay requests. This JIRA makes it possible to use a unique key for each service instance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.