You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2010/09/24 20:29:33 UTC
[jira] Created: (MAPREDUCE-2089) JobToken should be passed over RPC
JobToken should be passed over RPC
----------------------------------
Key: MAPREDUCE-2089
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2089
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client, job submission, jobtracker, security, task, tasktracker
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.22.0
JobToken file is currently created by the JobTracker, and stored in the HDFS. Later on, the TaskTrackers pull that file from the HDFS as part of job localization. The HDFS operations can be avoided if the the jobToken file is passed over RPC (Jobclient -> JobTracker -> TaskTrackers).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2089) JobToken should be passed over RPC
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-2089:
-----------------------------------
Attachment: mr-2089.patch
Patch that passes unit tests and test-patch.
> JobToken should be passed over RPC
> ----------------------------------
>
> Key: MAPREDUCE-2089
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2089
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client, job submission, jobtracker, security, task, tasktracker
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: mr-2089.patch, mr-2089.patch
>
>
> JobToken file is currently created by the JobTracker, and stored in the HDFS. Later on, the TaskTrackers pull that file from the HDFS as part of job localization. The HDFS operations can be avoided if the the jobToken file is passed over RPC (Jobclient -> JobTracker -> TaskTrackers).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2089) JobToken should be passed over RPC
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-2089:
-----------------------------------
Status: Patch Available (was: Open)
> JobToken should be passed over RPC
> ----------------------------------
>
> Key: MAPREDUCE-2089
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2089
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client, job submission, jobtracker, security, task, tasktracker
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: mr-2089.patch
>
>
> JobToken file is currently created by the JobTracker, and stored in the HDFS. Later on, the TaskTrackers pull that file from the HDFS as part of job localization. The HDFS operations can be avoided if the the jobToken file is passed over RPC (Jobclient -> JobTracker -> TaskTrackers).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2089) JobToken should be passed over RPC
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-2089:
-----------------------------------
Attachment: mr-2089.patch
A patch for trunk.
> JobToken should be passed over RPC
> ----------------------------------
>
> Key: MAPREDUCE-2089
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2089
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client, job submission, jobtracker, security, task, tasktracker
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: mr-2089.patch
>
>
> JobToken file is currently created by the JobTracker, and stored in the HDFS. Later on, the TaskTrackers pull that file from the HDFS as part of job localization. The HDFS operations can be avoided if the the jobToken file is passed over RPC (Jobclient -> JobTracker -> TaskTrackers).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2089) JobToken should be passed over
RPC
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917106#action_12917106 ]
Devaraj Das commented on MAPREDUCE-2089:
----------------------------------------
The jobtoken is used for shuffle and it is also used for Pipes, for both the secure/unsecure cases. Can't skip it..
> JobToken should be passed over RPC
> ----------------------------------
>
> Key: MAPREDUCE-2089
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2089
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client, job submission, jobtracker, security, task, tasktracker
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: mr-2089.patch, mr-2089.patch
>
>
> JobToken file is currently created by the JobTracker, and stored in the HDFS. Later on, the TaskTrackers pull that file from the HDFS as part of job localization. The HDFS operations can be avoided if the the jobToken file is passed over RPC (Jobclient -> JobTracker -> TaskTrackers).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2089) JobToken should be passed over
RPC
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916394#action_12916394 ]
Vinod K V commented on MAPREDUCE-2089:
--------------------------------------
A related point I noted is that the Child task needs the token file irrespective of whether security is turned on or off. This can be skipped for no-security, can we do that here too?
> JobToken should be passed over RPC
> ----------------------------------
>
> Key: MAPREDUCE-2089
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2089
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client, job submission, jobtracker, security, task, tasktracker
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: mr-2089.patch, mr-2089.patch
>
>
> JobToken file is currently created by the JobTracker, and stored in the HDFS. Later on, the TaskTrackers pull that file from the HDFS as part of job localization. The HDFS operations can be avoided if the the jobToken file is passed over RPC (Jobclient -> JobTracker -> TaskTrackers).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2089) JobToken should be passed over
RPC
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915927#action_12915927 ]
Devaraj Das commented on MAPREDUCE-2089:
----------------------------------------
A note i forgot to put earlier - the trunk patches here are a port of Owen's original patch for Y20S.
> JobToken should be passed over RPC
> ----------------------------------
>
> Key: MAPREDUCE-2089
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2089
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client, job submission, jobtracker, security, task, tasktracker
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: mr-2089.patch, mr-2089.patch
>
>
> JobToken file is currently created by the JobTracker, and stored in the HDFS. Later on, the TaskTrackers pull that file from the HDFS as part of job localization. The HDFS operations can be avoided if the the jobToken file is passed over RPC (Jobclient -> JobTracker -> TaskTrackers).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.