[jira] Updated: (HADOOP-4491) Per-job local data on the TaskTracker node should have right access-control

["Vinod K V (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vinod K V updated HADOOP-4491:
------------------------------

    Attachment: HADOOP-4491-20090703-common.txt
                HADOOP-4491-20090703.txt

Adding patches for quick review. Still a work in progress. Incorporated most of Devaraj's comments.

Two major changes from the previous patch:
 - Pulled _taskTracker/jobcache/attemp-id/work_ to _taskTracker/jobcache/task-work/attempt-id/work_. This is done to solve issues with jvm reuse. The work dir needs to be shared across tasks with jvm reuse and so cannot be finalized to be owned back by the TT on task finish. Pulling this out makes things clean, the original attempt-dir is finalized, and the task-work directory is cleaned up on jvm exit. TODO: task-work can have better name, perhaps jvmcache.

 - _log-dir/userlogs/attempt-id_ will still have 777 permissions with all the files inside created by linuxTaskController but owned by the user. This is done because
    -- attempt-id dir has to be writable by the child for log.tmp which is done periodically in synclogs.
    -- attempt-dir has to be readable by TT for log serving and writable by TT also by cleanup.

Long time, the correct solution is for this directory to be owned by the child, but cleaned up by TT using a LinuxTaskController binary lauch to swith user.

Pending items from Devaraj's comments:
 -  It will be nice to combine the APIs for creating files/directories and setting appropriate permissions all in one API
 - DiskChecker.java has lot of code to do with permissions handling to make it generic but not everything would be actually used. In fact, the approach taken in making some APIs generic is debatable. We might as well keep it simple for now and extend those APIs as and when required.
 - LocalDirAllocator.getSecureLocalPathForWrite could be renamed as getPrivateLocalPathForEWrite 

Extra:
 - Test TaskTracker.initializeJobDirs
 - Extra tests in taskcontroller.c
 - Test various scenarios of finalizeTaskDirs
 - Finish tests for jvm reuse

There are some minor TODO's also left to do.

> Per-job local data on the TaskTracker node should have right access-control
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-4491
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4491
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: mapred, security
>            Reporter: Arun C Murthy
>            Assignee: Vinod K V
>         Attachments: HADOOP-4491-20090623-common.1.txt, HADOOP-4491-20090623-mapred.1.txt, HADOOP-4491-20090703-common.txt, HADOOP-4491-20090703.txt
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.