You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Boris Shkolnik (JIRA)" <ji...@apache.org> on 2011/04/05 05:28:05 UTC
[jira] [Created] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
JobTracker should be able to renew delegation token over HTTP
-------------------------------------------------------------
Key: MAPREDUCE-2420
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.20.4
Reporter: Boris Shkolnik
Assignee: Boris Shkolnik
in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Benoy Antony (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoy Antony updated MAPREDUCE-2420:
------------------------------------
Attachment: (was: MR-2420-022.patch)
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.23.0
>
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.22-3.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated MAPREDUCE-2420:
--------------------------------------
Attachment: MR-2420.22-2.patch
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Jitendra Nath Pandey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026765#comment-13026765 ]
Jitendra Nath Pandey commented on MAPREDUCE-2420:
-------------------------------------------------
I have committed this. Thanks to Boris!
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.23.0
>
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated MAPREDUCE-2420:
--------------------------------------
Attachment: MR-2420.patch
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Benoy Antony (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoy Antony updated MAPREDUCE-2420:
------------------------------------
Attachment: MR-2420-022.patch
Patch for 22
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.23.0
>
> Attachments: MR-2420-022.patch, MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.22-3.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated MAPREDUCE-2420:
-------------------------------------------
Fix Version/s: (was: 0.23.0)
0.22.1
I just committed this to branch 0.22.1. Thank you Benoy.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.22.1
>
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.22-3.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Jitendra Nath Pandey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jitendra Nath Pandey resolved MAPREDUCE-2420.
---------------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.0
Hadoop Flags: [Reviewed]
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.23.0
>
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated MAPREDUCE-2420:
--------------------------------------
Attachment: MR-2420-1.patch
implemented some of Devaraj's comments.
also set stack to empty for artificially created exceptions (on the client side).
Agree with Devaraj that we need a more generic solution for all types of token.
Please open a Jira on this and put these requirements in.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420-1.patch, MR-2420.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Benoy Antony (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280538#comment-13280538 ]
Benoy Antony commented on MAPREDUCE-2420:
-----------------------------------------
I removed my patch for 22. The patch -MR-2420.22-3.patch is good.
The patch for 22 in HDFS-1860 has to be applied first.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.23.0
>
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.22-3.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016076#comment-13016076 ]
Boris Shkolnik commented on MAPREDUCE-2420:
-------------------------------------------
committed to branch-20-security.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420-1.patch, MR-2420.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016060#comment-13016060 ]
Devaraj Das commented on MAPREDUCE-2420:
----------------------------------------
This patch handles the case of HDFS token renewals, when the JobTracker's HDFS is running a different version of HDFS than the one the job is trying to use. An example of such a job is distcp (where it uses hftp to talk to a different source cluster to pull data to the cluster where distcp is running).
When the job is submitted, the client requests a delegation token over hftp and stuffs it in the job. Today, the NameNode doesn't distinguish between hftp and hdfs accesses, and issues HDFS tokens for both (and the token-kind field in the token has the value as 'HDFS'). Ideally, that should be fixed to have the token-kind as HFTP for hftp accesses. We should have the JobTracker handle all sorts of token renewals, and have a way in which it can look at a token and decide which protocol to use to talk to the server in question. This includes HDFS, HFTP, and also HIVE (where the protocol is thrift).
I think this patch is okay for the short term - the JobTracker falls back to hftp if it couldn't renew a token over hdfs. In the patch, there are a bunch of white space changes that aren't required. The string comparisons for exception messages and then instantiating a concrete exception could probably be replaced with a forname() on the string exception.
When we fix this issue in trunk, please make it more generic on lines similar to above.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated MAPREDUCE-2420:
--------------------------------------
Attachment: MR-2420.22-1.patch
preliminary patch for .22
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated MAPREDUCE-2420:
--------------------------------------
Attachment: MR-2420.22-3.patch
patch for previous version(.22)
using DFSConfigKeys.DFS_HTTPS_PORT_KEY instead of string value.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.23.0
>
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.22-3.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated MAPREDUCE-2420:
--------------------------------------
Attachment: MR2420-2.patch
patch for trunk
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.patch, MR2420-2.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026771#comment-13026771 ]
Hudson commented on MAPREDUCE-2420:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #651 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/651/])
MAPREDUCE-2420. JobTracker should be able to renew delegation token over HTTP. Contributed by Boris Shkolnik.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.23.0
>
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Jitendra Nath Pandey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026749#comment-13026749 ]
Jitendra Nath Pandey commented on MAPREDUCE-2420:
-------------------------------------------------
+1 for the patch.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Jitendra Nath Pandey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026693#comment-13026693 ]
Jitendra Nath Pandey commented on MAPREDUCE-2420:
-------------------------------------------------
Please use constant defined in DFSConfigKeys for dfs.https.port and for the default as well.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.patch, MR2420-2.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028773#comment-13028773 ]
Hudson commented on MAPREDUCE-2420:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #669 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/669/])
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.23.0
>
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.22-3.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026690#comment-13026690 ]
Boris Shkolnik commented on MAPREDUCE-2420:
-------------------------------------------
ran the ant test and it passed.
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.patch, MR2420-2.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289428#comment-13289428 ]
Hudson commented on MAPREDUCE-2420:
-----------------------------------
Integrated in Hadoop-Mapreduce-22-branch #104 (See [https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/104/])
MAPREDUCE-2420. JobTracker should be able to renew delegation token over HTTP. Contributed by Boris Shkolnik and Benoy Antony. (Revision 1346233)
Result = SUCCESS
shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346233
Files :
* /hadoop/common/branches/branch-0.22/mapreduce/CHANGES.txt
* /hadoop/common/branches/branch-0.22/mapreduce/src/java/org/apache/hadoop/mapreduce/security/token/DelegationTokenRenewal.java
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.22.1
>
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.22-3.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026636#comment-13026636 ]
Boris Shkolnik commented on MAPREDUCE-2420:
-------------------------------------------
+1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no new tests are needed for this patch.
[exec] Also please list what manual steps were performed to verify this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec]
No unit tests, but I did manual testing using discp copying from the same cluster or from 0.20 one
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.patch, MR2420-2.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2420) JobTracker should be able to
renew delegation token over HTTP
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated MAPREDUCE-2420:
--------------------------------------
Attachment: MR2420-3.patch
changed "dfs.https.port" to DFSConfigKeys....
> JobTracker should be able to renew delegation token over HTTP
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2420
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2420
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.4
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: MR-2420-1.patch, MR-2420.22-1.patch, MR-2420.22-2.patch, MR-2420.patch, MR2420-2.patch, MR2420-3.patch
>
>
> in case JobTracker has to talk to a NameNode running a different version (RPC version mismatch), Jobtracker should be able to fall back to HTTP renewal.
> Example of the case - running distcp between different versions using hfpt.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira