You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinay Kumar Thota (JIRA)" <ji...@apache.org> on 2010/07/21 20:49:51 UTC
[jira] Created: (MAPREDUCE-1957) [Herriot] Test Job cache
directories cleanup after job completes.
[Herriot] Test Job cache directories cleanup after job completes.
-----------------------------------------------------------------
Key: MAPREDUCE-1957
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1957
Project: Hadoop Map/Reduce
Issue Type: Task
Components: test
Reporter: Vinay Kumar Thota
Assignee: Vinay Kumar Thota
Test the job cache directories cleanup after job completes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1957) [Herriot] Test Job cache
directories cleanup after job completes.
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated MAPREDUCE-1957:
------------------------------------------
Tags: herriot
> [Herriot] Test Job cache directories cleanup after job completes.
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1957
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1957
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1957-ydist-security.patch, 1957-ydist-security.patch, 1957-ydist-security.patch
>
>
> Test the job cache directories cleanup after job completes.Test covers the following scenarios.
> 1. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Wait till the job completes and verify whether the files and folders are cleaned up or not.
> 2. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Kill the job and verify whether the files and folders are cleaned up or not.
> 3. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Fail the job and verify whether the files and folders are cleaned up or not.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1957) [Herriot] Test Job cache
directories cleanup after job completes.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-1957:
-----------------------------------------
Attachment: 1957-ydist-security.patch
Addressed Balaji comments.
> [Herriot] Test Job cache directories cleanup after job completes.
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1957
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1957
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1957-ydist-security.patch, 1957-ydist-security.patch, 1957-ydist-security.patch
>
>
> Test the job cache directories cleanup after job completes.Test covers the following scenarios.
> 1. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Wait till the job completes and verify whether the files and folders are cleaned up or not.
> 2. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Kill the job and verify whether the files and folders are cleaned up or not.
> 3. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Fail the job and verify whether the files and folders are cleaned up or not.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1957) [Herriot] Test Job cache
directories cleanup after job completes.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-1957:
-----------------------------------------
Attachment: 1957-ydist-security.patch
Patch for yahoo dist security branch.
> [Herriot] Test Job cache directories cleanup after job completes.
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1957
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1957
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1957-ydist-security.patch
>
>
> Test the job cache directories cleanup after job completes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1957) [Herriot] Test Job cache
directories cleanup after job completes.
Posted by "Balaji Rajagopalan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891501#action_12891501 ]
Balaji Rajagopalan commented on MAPREDUCE-1957:
-----------------------------------------------
+1
> [Herriot] Test Job cache directories cleanup after job completes.
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1957
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1957
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1957-ydist-security.patch, 1957-ydist-security.patch, 1957-ydist-security.patch
>
>
> Test the job cache directories cleanup after job completes.Test covers the following scenarios.
> 1. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Wait till the job completes and verify whether the files and folders are cleaned up or not.
> 2. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Kill the job and verify whether the files and folders are cleaned up or not.
> 3. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Fail the job and verify whether the files and folders are cleaned up or not.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1957) [Herriot] Test Job cache
directories cleanup after job completes.
Posted by "Balaji Rajagopalan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891049#action_12891049 ]
Balaji Rajagopalan commented on MAPREDUCE-1957:
-----------------------------------------------
+ FinishTaskControlAction action = new FinishTaskControlAction(taskId);
+ if (ttClient != null ) {
+ ttClient.getProxy().sendAction(action);
+ String localDirs[] = ttClient.getMapredLocalDirs();
+ TaskAttemptID taskAttID = new TaskAttemptID(taskId, 0);
+ return createFilesInTaskDir(localDirs, jobId, taskAttID, ttClient);
The order of the create file and signal tasks for completion is wrong, you will have to create the files and then signal the tasks for completion. Other wise the code looks good, nice job.
> [Herriot] Test Job cache directories cleanup after job completes.
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1957
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1957
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1957-ydist-security.patch, 1957-ydist-security.patch
>
>
> Test the job cache directories cleanup after job completes.Test covers the following scenarios.
> 1. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Wait till the job completes and verify whether the files and folders are cleaned up or not.
> 2. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Kill the job and verify whether the files and folders are cleaned up or not.
> 3. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Fail the job and verify whether the files and folders are cleaned up or not.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1957) [Herriot] Test Job cache
directories cleanup after job completes.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-1957:
-----------------------------------------
Attachment: 1957-ydist-security.patch
> [Herriot] Test Job cache directories cleanup after job completes.
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1957
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1957
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1957-ydist-security.patch, 1957-ydist-security.patch
>
>
> Test the job cache directories cleanup after job completes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1957) [Herriot] Test Job cache
directories cleanup after job completes.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-1957:
-----------------------------------------
Description:
Test the job cache directories cleanup after job completes.Test covers the following scenarios.
1. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Wait till the job completes and verify whether the files and folders are cleaned up or not.
2. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Kill the job and verify whether the files and folders are cleaned up or not.
3. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Fail the job and verify whether the files and folders are cleaned up or not.
was:
Test the job cache directories cleanup after job completes.
> [Herriot] Test Job cache directories cleanup after job completes.
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1957
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1957
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1957-ydist-security.patch, 1957-ydist-security.patch
>
>
> Test the job cache directories cleanup after job completes.Test covers the following scenarios.
> 1. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Wait till the job completes and verify whether the files and folders are cleaned up or not.
> 2. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Kill the job and verify whether the files and folders are cleaned up or not.
> 3. Submit a job and create folders and files in work folder with non-writable permissions under task attempt id folder. Fail the job and verify whether the files and folders are cleaned up or not.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.