You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinod K V (JIRA)" <ji...@apache.org> on 2009/07/09 11:50:14 UTC
[jira] Created: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Support in DistributedCache to share cache files with other users after HADOOP-4493
-----------------------------------------------------------------------------------
Key: MAPREDUCE-744
URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: tasktracker
Reporter: Vinod K V
HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Attachment: 744-4.patch
bq. 1. DistributedCache.get*Visibiliteies - are deprecated. Why do we add deprecated methods?
MAPREDUCE-982 needs to be fixed for me to not have the deprecated methods. But i will think about if we can avoid it in any way.
bq. 2. Do we need separate call for determineFile/ArchiveVisability. It can be determined while calling DistributedCache.addCacheArchive/File()
I don't have a strong preference for this either way but it seems clearer to me to keep it the way i currently have..
bq. 3. private static isPublic() should be called differently - something like haveOtherPermissions(Perm), to match its functionality. Some comments would help too.
Done
bq. 4. public methods should have javadoc comments (determineCacheVisibilities() in TrackerDistributedCacheManager)
Done
bq. 5. this line: File workDir = new File(new Path(TEST_ROOT_DIR, "workdir").toString()); can be changed
Done
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-4.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796743#action_12796743 ]
Hudson commented on MAPREDUCE-744:
----------------------------------
Integrated in Hadoop-Mapreduce-trunk #196 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/196/])
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-4.patch, 744-5.patch, 744-6.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated MAPREDUCE-744:
--------------------------------
Component/s: security
distributed-cache
Release Note: Fixed DistributedCache to support sharing of the local cache files with other users on the same TaskTracker. The cache files are checked at the client side for public/private access on the file system, and that information is passed in the configuration. The TaskTrackers look at the configuration for each file during task localization, and, if the file was public on the filesystem, they are localized to a common space for sharing by all users' tasks on the TaskTracker. Else the file is localized to the user's private directory on the local filesystem.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: distributed-cache, security, tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-4.patch, 744-5.patch, 744-6-y20.patch, 744-6.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Fix Version/s: 0.22.0
Assignee: Devaraj Das
Status: Patch Available (was: Open)
Passing through hudson
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Status: Patch Available (was: Open)
Retrying hudson
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789085#action_12789085 ]
Hadoop QA commented on MAPREDUCE-744:
-------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12427656/744-2.patch
against trunk revision 889085.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/317/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/317/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/317/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/317/console
This message is automatically generated.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated MAPREDUCE-744:
-------------------------------------
Hadoop Flags: [Reviewed]
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-4.patch, 744-5.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated MAPREDUCE-744:
---------------------------------------
Attachment: 744-6-y20.patch
Patch for older version of hadoop. Not for commit here.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-4.patch, 744-5.patch, 744-6-y20.patch, 744-6.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794507#action_12794507 ]
Boris Shkolnik commented on MAPREDUCE-744:
------------------------------------------
Looks good.
+1
One small nit:
TrackerDistributedCacheManager.isPublic and checkPermissionsofOther have empty @return javadoc. This may cause Javadoc warnings.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-4.patch, 744-5.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Attachment: 744-early.patch
Attaching a preliminary patch for review. What's done there is that the cache files are checked at the client side for public/private access, and that information (booleans - true for public, false for private) is passed in the configuration. The TaskTrackers look at the configuration for each file during localization, and, if the file was public they localize it to a common space. If not, then the file is localized to the user's private directory.
Testcase is not there yet.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Attachments: 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Attachment: 744-2.patch
Fixed the testcase issue to do with TestTrackerDistributedCacheManagerWithLinuxTaskController. The other failing tests are not related to this patch.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793743#action_12793743 ]
Boris Shkolnik commented on MAPREDUCE-744:
------------------------------------------
Some comments from review:
1. DistributedCache.get*Visibiliteies - are deprecated. Why do we add deprecated methods?
2. Do we need separate call for determineFile/ArchiveVisability. It can be determined while calling DistributedCache.addCacheArchive/File()
3. private static isPublic() should be called differently - something like haveOtherPermissions(Perm), to match its functionality. Some comments would help too.
4. public methods should have javadoc comments (determineCacheVisibilities() in TrackerDistributedCacheManager)
5. this line: File workDir = new File(new Path(TEST_ROOT_DIR, "workdir").toString());
can be changed to File workDir = new File(TEST_ROOT_DIR, "workdir") in Tasktrackerdistributedcachemanager
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Status: Open (was: Patch Available)
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Attachment: 744-6.patch
Thanks Boris for the review. The attached patch has the javadoc fix. All tests, except the ones known to fail (TestStreamingExitStatus, TestStreamingKeyValue), and test-patch passed. I will commit this.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-4.patch, 744-5.patch, 744-6.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Attachment: 744-3.patch
I realized that the check for whether a path on the hdfs is public or not should check for EXECUTE permission for Other user for the subdirs in the path. Attached patch fixes that.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Status: Open (was: Patch Available)
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Attachment: 744-1.patch
Attaching a more complete patch.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788686#action_12788686 ]
Hadoop QA commented on MAPREDUCE-744:
-------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12427588/744-1.patch
against trunk revision 889085.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/184/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/184/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/184/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/184/console
This message is automatically generated.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786279#action_12786279 ]
Devaraj Das commented on MAPREDUCE-744:
---------------------------------------
I should add that "public" means world-readable. The entire hierarchy of the cache file path is checked for that (starting from the leaf filename to "/").
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Attachments: 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated MAPREDUCE-744:
----------------------------------
Attachment: 744-5.patch
bq. 1. DistributedCache.get*Visibiliteies - are deprecated. Why do we add deprecated methods?
Ok it looks like we don't have to really add the methods in DistributedCache since the methods are used only by the framework. So i moved the accessors (getFileVisibilities/getArchiveVisibilities) to TrackerDistributedCacheManager where the setters are defined.
This seems like the right thing to do ..
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-4.patch, 744-5.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAPREDUCE-744) Support in DistributedCache to
share cache files with other users after HADOOP-4493
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das resolved MAPREDUCE-744.
-----------------------------------
Resolution: Fixed
I just committed this.
> Support in DistributedCache to share cache files with other users after HADOOP-4493
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-744
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: tasktracker
> Reporter: Vinod K V
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 744-1.patch, 744-2.patch, 744-3.patch, 744-4.patch, 744-5.patch, 744-6.patch, 744-early.patch
>
>
> HADOOP-4493 aims to completely privatize the files distributed to TT via DistributedCache. This jira issues focuses on sharing some/all of these files with all other users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.