You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Vinay Kumar Thota (JIRA)" <ji...@apache.org> on 2010/07/21 19:06:50 UTC
[jira] Created: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
[Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
--------------------------------------------------------------------------------------------------------------------
Key: HADOOP-6869
URL: https://issues.apache.org/jira/browse/HADOOP-6869
Project: Hadoop Common
Issue Type: Task
Components: test
Reporter: Vinay Kumar Thota
Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
It uses to create a file with full permissions.
2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
It uses to create a file with given permissions.
3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
It uses to create a file with full permissions.
4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892677#action_12892677 ]
Vinay Kumar Thota commented on HADOOP-6869:
-------------------------------------------
java doc warnings are not related to this patch.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, 6869-ydist-security.patch, HADOOP-6869.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated HADOOP-6869:
--------------------------------------
Attachment: 6869-ydist-security.patch
HADOOP-6869.patch
Addressed cos comments.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, 6869-ydist-security.patch, HADOOP-6869.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891486#action_12891486 ]
Vinay Kumar Thota commented on HADOOP-6869:
-------------------------------------------
Hi Cos, I do agree with you. However my requirement is creating the user define files and folders in task attempt id folder with different kind of permissions while job is running. Later while cleaning up, i just wanted make sure whether all the user defined files and folders are cleaned up or not for that job. So that I have defined the above methods for fulfilling my requirement. As my understand, i don't think so, the FileSystem is providing the no such straight forward methods(createNewFile and createNewFolder) like you mentioned.
In the four methods, the first method creates the files with full permission.Suppose if user wants to give his own permissions while creating file in that case he can use the second method.Same way for folder creation also.
bq. also you are writing some content into this file which seems to be an inadvertent side-effect. Doesn't look like a good idea
I will remove this part in the code.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HADOOP-6869:
---------------------------------------
Status: Patch Available (was: Open)
Affects Version/s: 0.21.0
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, 6869-ydist-security.patch, HADOOP-6869.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6869) Functionality to creating file or
folder on a remote daemon side
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HADOOP-6869:
---------------------------------------
Summary: Functionality to creating file or folder on a remote daemon side (was: [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.)
Hadoop Flags: [Reviewed]
Issue Type: New Feature (was: Task)
> Functionality to creating file or folder on a remote daemon side
> ----------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: New Feature
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, 6869-ydist-security.patch, HADOOP-6869.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891429#action_12891429 ]
Konstantin Boudnik commented on HADOOP-6869:
--------------------------------------------
In continuation of the comment above: since there's {{getFS()}} method which exposes filesystem API from a particular daemon then creating additional 4 wrappers around the standard API doesn't make much sense to me. These wrappers are too obvious and then will simply crowd Herriot API without adding much value.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891356#action_12891356 ]
Konstantin Boudnik commented on HADOOP-6869:
--------------------------------------------
- Looks good with one nit. I'd suggest to have methods with fewer arguments simply call ones with more args. E.g. in this case
{noformat}
public void createFile(String path, String fileName, boolean local) throws IOException {
createFile(path, fileName, null, local);
}
{noformat}
instead of duplicating their exact implementation. You have done similar in {{DaemonProtocolAspect}} already.
- Also, in {{DaemonProtocolAspect}} you already have a ref to the filesystem so you can just call its {{createFile}} methods instead of implementing your own logic
{noformat}
+ public void DaemonProtocol.createFile(String path, String fileName,
+ FsPermission permission, boolean local) throws IOException {
+ Path p = new Path(path);
+ FileSystem fs = getFS(p, local);
{noformat}
just call {{fs.createNewFile()}} to have new file created for you.
- also you are writing some content into this file which seems to be an inadvertent side-effect. Doesn't look like a good idea.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6869) Functionality to create file or
folder on a remote daemon side
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HADOOP-6869:
---------------------------------------
Tags: herriot
> Functionality to create file or folder on a remote daemon side
> --------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: New Feature
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.21.0
>
> Attachments: 6869-ydist-security.patch, 6869-ydist-security.patch, HADOOP-6869.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892672#action_12892672 ]
Hadoop QA commented on HADOOP-6869:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12450562/HADOOP-6869.patch
against trunk revision 979485.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/640/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/640/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/640/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/640/console
This message is automatically generated.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, 6869-ydist-security.patch, HADOOP-6869.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892902#action_12892902 ]
Konstantin Boudnik commented on HADOOP-6869:
--------------------------------------------
I'll commit it later today if won't hear otherwise.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, 6869-ydist-security.patch, HADOOP-6869.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892065#action_12892065 ]
Vinay Kumar Thota commented on HADOOP-6869:
-------------------------------------------
Cos,
Please look at my patch once again because I have not duplicated the exact implementation for create file. I have created a wrapper around the actual method with full permission like which i coded for create folder.Might be you have missed to look at that portion in my patch.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated HADOOP-6869:
--------------------------------------
Attachment: (was: 6869-ydist-security.patch)
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated HADOOP-6869:
--------------------------------------
Attachment: 6869-ydist-security.patch
Initial patch for yahoo dist security branch.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891719#action_12891719 ]
Konstantin Boudnik commented on HADOOP-6869:
--------------------------------------------
bq. my requirement is creating the user define files and folders in task attempt id folder with different kind of permissions while job is running
This reasoning sounds like an argument to move this functionality up the stack to the MR cluster concrete implementation. But let's suppose they are useful enough and let them be in the Common.
bq. In the four methods, the first method creates the files with full permission.Suppose if user wants to give his own permissions while creating file in that case he can use the second method.Same way for folder creation also.
I do understand that. What I have said is that you need a generic enough method which does most of needed functionality (i.e. create a file with specified permissions) and a wrapper around it which creates a file with all permissions. The implementation of the latter is essentially a call to the former with *all* permissions being passed. This is already done in this very patch for the implementation of {{createFolder(..)}} methods. Look at your own code.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated HADOOP-6869:
--------------------------------------
Attachment: HADOOP-6869.patch
Patch for trunk.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6869) Functionality to create file or
folder on a remote daemon side
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HADOOP-6869:
---------------------------------------
Summary: Functionality to create file or folder on a remote daemon side (was: Functionality to creating file or folder on a remote daemon side)
> Functionality to create file or folder on a remote daemon side
> --------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: New Feature
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.21.0
>
> Attachments: 6869-ydist-security.patch, 6869-ydist-security.patch, HADOOP-6869.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota reassigned HADOOP-6869:
-----------------------------------------
Assignee: Vinay Kumar Thota
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Balaji Rajagopalan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891050#action_12891050 ]
Balaji Rajagopalan commented on HADOOP-6869:
--------------------------------------------
+1
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated HADOOP-6869:
--------------------------------------
Attachment: 6869-ydist-security.patch
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6869) Functionality to create file or
folder on a remote daemon side
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HADOOP-6869:
---------------------------------------
Status: Resolved (was: Patch Available)
Fix Version/s: 0.21.0
Resolution: Fixed
I have just committed this. Thanks Vinay.
> Functionality to create file or folder on a remote daemon side
> --------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: New Feature
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.21.0
>
> Attachments: 6869-ydist-security.patch, 6869-ydist-security.patch, HADOOP-6869.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6869) [Herriot] Implement a functionality
for creating either file or folder in task attempt folder while job is
running.
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892657#action_12892657 ]
Konstantin Boudnik commented on HADOOP-6869:
--------------------------------------------
I am glad we have understood each other :)
+1 patch looks good. Let's verify. Also, please make sure that it works in real cluster setup because we can't verify it in Hudson yet.
> [Herriot] Implement a functionality for creating either file or folder in task attempt folder while job is running.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6869
> URL: https://issues.apache.org/jira/browse/HADOOP-6869
> Project: Hadoop Common
> Issue Type: Task
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 6869-ydist-security.patch, 6869-ydist-security.patch, HADOOP-6869.patch, HADOOP-6869.patch
>
>
> Functionality for creating either files or folders in task attempt folder while job is running. The functionality covers the following methods.
> 1. public void DaemonProtocol.createFile(String path, String fileName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 2. public void DaemonProtocol.createFile(String path, String fileName, FsPermission permission, boolean local) throws IOException;
> It uses to create a file with given permissions.
> 3. public void DaemonProtocol.createFolder(String path, String folderName, boolean local) throws IOException;
> It uses to create a file with full permissions.
> 4. public void DaemonProtocol.createFolder(String path, String folderName, FsPermission permission, boolean local) throws IOException;
> It uses to create a folder with given permissions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.